1
0
mirror of https://github.com/google/nomulus synced 2026-05-20 06:41:51 +00:00

Compare commits

...

19 Commits

Author SHA1 Message Date
gbrodman
7a4c109b36 Remove recursive load in DBMSS cache (#1286)
* Remove recursive load in DBMSS cache

This occurs because if we do a standard transaction, the JpaTxnManager
checks to see if we should be doing backups, which involves loading the
migration state schedule (causing the recursion). When starting the
transaction to load the schedule, we should explicitly
transactWithoutBackup so there's no need to check.

This wasn't hit in tests because we previously manually set the
replication to not occur in the JpaTransactionManagerExtension -- we
remove that and related setters.
2021-08-14 12:34:23 -06:00
Ben McIlwain
22b1b8d21a Add instructions for two-step DB schema updates (#1283)
* Add instructions for two-step DB schema updates

These expanded steps are required by the recent enabling of the SQL integration test suite.
2021-08-13 17:21:38 -04:00
Weimin Yu
5bbabadafd Generate string to uniquely identify a SqlEntity (#1271)
* Generate string to uniquely identify a SqlEntity

Add a method to SqlEntity that returns a string built from the entity's
primary key(s). This string can be used in logging.
2021-08-13 16:22:54 -04:00
Ben McIlwain
6c73161ff8 Add the domain DNS refresh request time field to the DB schema (#1280)
* Add the domain DNS refresh request time field to the DB schema

This isn't used yet, but it will eventually be the replacement for the dns-pull
task queue once we get further in the migration.

* Remove index
2021-08-13 15:32:18 -04:00
Rachel Guan
7faee04422 Modify class name to remove checkstyleTest warning (#1281) 2021-08-13 14:16:58 -04:00
Ben McIlwain
b340b2b5e9 Add tx/s instrumentation to replay action and re-enable it on sandbox (#1276) 2021-08-12 18:33:47 -04:00
gbrodman
7f733cd16d Store DatabaseMigrationSchedule in SQL instead of Datastore (#1269)
* Store DatabaseMigrationSchedule in SQL instead of Datastore

This requires messing around with some of the JPA unit test rule
creation since it requires saving / retrieving the schedule pretty much
always (which itself includes the hstore extension).
2021-08-12 15:57:31 -06:00
Ben McIlwain
60469479a4 Consolidate all remaining schema classes into model package (#1278) 2021-08-12 13:38:50 -04:00
Ben McIlwain
5158673f21 Consolidate all Registry/TLD-related classes into google.registry.model.tld (#1277) 2021-08-11 18:04:51 -04:00
gbrodman
743ca4106c Add SQL schema additions for DatabaseMigrationStateSchedule (#1274) 2021-08-10 16:46:07 -04:00
Ben McIlwain
2b99ee61d4 Load DatabaseMigrationStateSchedule in a more performant way (#1273)
This performs a direct load-by-key (the most efficient Datastore operation),
rather than attempting to load all entities by type using an ancestor query. The
existing implementation is possibly more error-prone as well, and might be
responsible for the "cross-group transaction need to be explicitly specified"
error we're seeing.
2021-08-10 14:47:59 -04:00
sarahcaseybot
28a1cc613c Remove PremiumList from Datastore schema (#1256)
* Remove PremiumList from Datastore schema

* Remove commented out code

* Change lastUpdateTime to creationTimestamp

* Remove extra file

* Remove currency unit from input data to parse

* Revert extra file

* Check currency in parse

* Create all PremiumEntries before saving them in bulk

* small fixes

* Fix merge conflict
2021-08-10 13:26:13 -04:00
sarahcaseybot
9811cdb85c Initialize data in cloudSqlOnly tests (#1266)
* Initialize data in cloudSqlOnly tests

* combine conditionals
2021-08-09 13:04:35 -04:00
Lai Jiang
761ae612fd Remove backported LocalStorageHelper (#1267)
* Remove backported LocalStorageHelper

The released version on Maven Central now contains the fix to the
serialization bug.
2021-08-06 21:10:32 -04:00
gbrodman
e2fa60a9c6 Use one SQL transaction per Datastore transaction in replay to SQL (#1268)
There was a subtle issue that we encountered in sandbox when using one
transaction per file that was difficult to replicate. Basically,
1. Save a domain with dsData
2. Save the domain without dsData
3. Save the domain with the same dsData as step 1
4. Delete literally any object

If one performs steps 2-4 in the same transaction, Hibernate will throw
an exception (cascade re-saving a cascade-deleted object). Note that
step 4 is in fact necessary to reproduce the issue, yay Hibernate.

We will test this and if one transaction per transaction is too slow,
we'll figure out ways to reduce the number of SQL transactions.
2021-08-06 16:05:36 -04:00
sarahcaseybot
b04dfbf740 Migrate invoicing pipeline to read from Cloud SQL (#1220)
* Save entities to Cloud SQL for tests

* Fix merge conflict

* Filter out non-real registrars and non-invoicing TLDs

* Add 1 month filter

* Handle cancellations

* Add to pipeline

* Use database in pipeline

* fix formatting

* Add a full pipeline test

* Fix repo ids in tests

* Move query to separate file

* Remove unused variables

* Remove unnecessary debugging remnant

* Reformat sql file

* Add jpql issue description

* Use DateTimeUtils

* Fix license header year

* Fix SQL formatting

* Use regex pattern

* Fix string building

* Add test for makeCloudSqlQuery

* Add clarifying comment
2021-08-06 15:56:04 -04:00
Weimin Yu
a1668ceafd Drop the KmsSecret table (#1258)
* Drop the KmsSecret table

Code using this table has been removed in PR 1252.
2021-08-04 23:23:58 -04:00
Lai Jiang
406d49ac99 Fix GCS bucket/subdir handling in IcannReportingStager (#1265)
After the migration to the new GCS API it becomes apparent that the
BlobId.of() method needs to take the bucket name (without any trailing
directories) as the first argument. I did a search on all occurrences of
"BlobId.of" in the code base and verified that it is only in the ICANN
reporting job that the API was misused.

<!-- Reviewable:start -->
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1265)
<!-- Reviewable:end -->
2021-08-04 14:01:04 -04:00
Weimin Yu
f7b82bc190 Allow db wipeouts in non-prod/sandbox enviroments (#1263)
* Allow db wipeouts in non-prod/sandbox enviroments
2021-08-03 17:41:10 -04:00
381 changed files with 5548 additions and 5656 deletions

4
.gitignore vendored
View File

@@ -112,3 +112,7 @@ core/**/registrar_bin*.js
core/**/registrar_dbg*.js
core/**/registrar_bin*.css
core/**/registrar_dbg*.css
# Appengine generated files
core/WEB-INF/appengine-generated/*.bin
core/WEB-INF/appengine-generated/*.xml

View File

@@ -100,6 +100,15 @@ public class DateTimeUtils {
return ZonedDateTime.ofInstant(instant, ZoneId.of(dateTime.getZone().getID()).normalized());
}
/**
* Converts a Joda {@link DateTime} object to an equivalent java.time {@link ZonedDateTime}
* object.
*/
public static ZonedDateTime toZonedDateTime(DateTime dateTime, ZoneId zoneId) {
java.time.Instant instant = java.time.Instant.ofEpochMilli(dateTime.getMillis());
return ZonedDateTime.ofInstant(instant, zoneId);
}
/**
* Converts a java.time {@link ZonedDateTime} object to an equivalent Joda {@link DateTime}
* object.

View File

@@ -28,7 +28,7 @@ com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-jackson2:1.30.10
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.31.5
com.google.api-client:google-api-client:1.32.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:1.5.5
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.105.5
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.105.5
@@ -56,10 +56,10 @@ com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
com.google.api.grpc:proto-google-common-protos:2.3.2
com.google.api.grpc:proto-google-iam-v1:1.0.14
com.google.api:api-common:1.10.3
com.google.api:api-common:1.10.4
com.google.api:gax-grpc:1.62.0
com.google.api:gax-httpjson:0.81.0
com.google.api:gax:1.65.0
com.google.api:gax-httpjson:0.83.0
com.google.api:gax:1.66.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20200916-1.30.10
@@ -76,7 +76,7 @@ com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-pubsub:v1-rev20200713-1.30.10
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.31.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
@@ -98,16 +98,16 @@ com.google.cloud:google-cloud-bigquery:1.122.2
com.google.cloud:google-cloud-bigquerystorage:1.5.5
com.google.cloud:google-cloud-bigtable:1.14.0
com.google.cloud:google-cloud-core-grpc:1.93.9
com.google.cloud:google-cloud-core-http:1.94.8
com.google.cloud:google-cloud-core:1.95.0
com.google.cloud:google-cloud-nio:0.123.2
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:1.95.4
com.google.cloud:google-cloud-nio:0.123.4
com.google.cloud:google-cloud-pubsub:1.110.0
com.google.cloud:google-cloud-pubsublite:0.7.0
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.cloud:google-cloud-storage:1.115.0
com.google.cloud:google-cloud-storage:1.118.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.code.gson:gson:2.8.7
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.7.1
@@ -138,8 +138,8 @@ com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.31.5
com.google.protobuf:protobuf-java-util:3.17.2
com.google.protobuf:protobuf-java:3.17.2
com.google.protobuf:protobuf-java-util:3.17.3
com.google.protobuf:protobuf-java:3.17.3
com.google.re2j:re2j:1.6
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
@@ -163,7 +163,7 @@ io.github.classgraph:classgraph:4.8.102
io.grpc:grpc-alts:1.36.0
io.grpc:grpc-api:1.36.0
io.grpc:grpc-auth:1.36.0
io.grpc:grpc-context:1.37.1
io.grpc:grpc-context:1.39.0
io.grpc:grpc-core:1.36.0
io.grpc:grpc-grpclb:1.36.0
io.grpc:grpc-netty-shaded:1.36.0

View File

@@ -27,7 +27,7 @@ com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-jackson2:1.30.10
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.31.5
com.google.api-client:google-api-client:1.32.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:1.5.5
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.105.5
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.105.5
@@ -55,10 +55,10 @@ com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
com.google.api.grpc:proto-google-common-protos:2.3.2
com.google.api.grpc:proto-google-iam-v1:1.0.14
com.google.api:api-common:1.10.3
com.google.api:api-common:1.10.4
com.google.api:gax-grpc:1.62.0
com.google.api:gax-httpjson:0.81.0
com.google.api:gax:1.65.0
com.google.api:gax-httpjson:0.83.0
com.google.api:gax:1.66.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20200916-1.30.10
@@ -75,7 +75,7 @@ com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-pubsub:v1-rev20200713-1.30.10
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.31.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
@@ -97,16 +97,16 @@ com.google.cloud:google-cloud-bigquery:1.122.2
com.google.cloud:google-cloud-bigquerystorage:1.5.5
com.google.cloud:google-cloud-bigtable:1.14.0
com.google.cloud:google-cloud-core-grpc:1.93.9
com.google.cloud:google-cloud-core-http:1.94.8
com.google.cloud:google-cloud-core:1.95.0
com.google.cloud:google-cloud-nio:0.123.2
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:1.95.4
com.google.cloud:google-cloud-nio:0.123.4
com.google.cloud:google-cloud-pubsub:1.110.0
com.google.cloud:google-cloud-pubsublite:0.7.0
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.cloud:google-cloud-storage:1.115.0
com.google.cloud:google-cloud-storage:1.118.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.code.gson:gson:2.8.7
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.7.1
@@ -136,8 +136,8 @@ com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.31.5
com.google.protobuf:protobuf-java-util:3.17.2
com.google.protobuf:protobuf-java:3.17.2
com.google.protobuf:protobuf-java-util:3.17.3
com.google.protobuf:protobuf-java:3.17.3
com.google.re2j:re2j:1.6
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
@@ -161,7 +161,7 @@ io.github.classgraph:classgraph:4.8.102
io.grpc:grpc-alts:1.36.0
io.grpc:grpc-api:1.36.0
io.grpc:grpc-auth:1.36.0
io.grpc:grpc-context:1.37.1
io.grpc:grpc-context:1.39.0
io.grpc:grpc-core:1.36.0
io.grpc:grpc-grpclb:1.36.0
io.grpc:grpc-netty-shaded:1.36.0

View File

@@ -32,7 +32,7 @@ com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-jackson2:1.30.10
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.31.5
com.google.api-client:google-api-client:1.32.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:1.5.5
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.105.5
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.105.5
@@ -60,10 +60,10 @@ com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
com.google.api.grpc:proto-google-common-protos:2.3.2
com.google.api.grpc:proto-google-iam-v1:1.0.14
com.google.api:api-common:1.10.3
com.google.api:api-common:1.10.4
com.google.api:gax-grpc:1.62.0
com.google.api:gax-httpjson:0.81.0
com.google.api:gax:1.65.0
com.google.api:gax-httpjson:0.83.0
com.google.api:gax:1.66.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20200916-1.30.10
@@ -80,7 +80,7 @@ com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-pubsub:v1-rev20200713-1.30.10
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.31.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
@@ -103,16 +103,16 @@ com.google.cloud:google-cloud-bigquery:1.122.2
com.google.cloud:google-cloud-bigquerystorage:1.5.5
com.google.cloud:google-cloud-bigtable:1.14.0
com.google.cloud:google-cloud-core-grpc:1.93.9
com.google.cloud:google-cloud-core-http:1.94.8
com.google.cloud:google-cloud-core:1.95.0
com.google.cloud:google-cloud-nio:0.123.2
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:1.95.4
com.google.cloud:google-cloud-nio:0.123.4
com.google.cloud:google-cloud-pubsub:1.110.0
com.google.cloud:google-cloud-pubsublite:0.7.0
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.cloud:google-cloud-storage:1.115.0
com.google.cloud:google-cloud-storage:1.118.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.code.gson:gson:2.8.7
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.7.1
@@ -143,8 +143,8 @@ com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.31.5
com.google.protobuf:protobuf-java-util:3.17.2
com.google.protobuf:protobuf-java:3.17.2
com.google.protobuf:protobuf-java-util:3.17.3
com.google.protobuf:protobuf-java:3.17.3
com.google.re2j:re2j:1.6
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
@@ -172,7 +172,7 @@ io.github.java-diff-utils:java-diff-utils:4.9
io.grpc:grpc-alts:1.36.0
io.grpc:grpc-api:1.36.0
io.grpc:grpc-auth:1.36.0
io.grpc:grpc-context:1.37.1
io.grpc:grpc-context:1.39.0
io.grpc:grpc-core:1.36.0
io.grpc:grpc-grpclb:1.36.0
io.grpc:grpc-netty-shaded:1.36.0

View File

@@ -32,7 +32,7 @@ com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-jackson2:1.30.10
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.31.5
com.google.api-client:google-api-client:1.32.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:1.5.5
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.105.5
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.105.5
@@ -60,10 +60,10 @@ com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
com.google.api.grpc:proto-google-common-protos:2.3.2
com.google.api.grpc:proto-google-iam-v1:1.0.14
com.google.api:api-common:1.10.3
com.google.api:api-common:1.10.4
com.google.api:gax-grpc:1.62.0
com.google.api:gax-httpjson:0.81.0
com.google.api:gax:1.65.0
com.google.api:gax-httpjson:0.83.0
com.google.api:gax:1.66.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20200916-1.30.10
@@ -80,7 +80,7 @@ com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-pubsub:v1-rev20200713-1.30.10
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.31.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
@@ -103,16 +103,16 @@ com.google.cloud:google-cloud-bigquery:1.122.2
com.google.cloud:google-cloud-bigquerystorage:1.5.5
com.google.cloud:google-cloud-bigtable:1.14.0
com.google.cloud:google-cloud-core-grpc:1.93.9
com.google.cloud:google-cloud-core-http:1.94.8
com.google.cloud:google-cloud-core:1.95.0
com.google.cloud:google-cloud-nio:0.123.2
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:1.95.4
com.google.cloud:google-cloud-nio:0.123.4
com.google.cloud:google-cloud-pubsub:1.110.0
com.google.cloud:google-cloud-pubsublite:0.7.0
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.cloud:google-cloud-storage:1.115.0
com.google.cloud:google-cloud-storage:1.118.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.code.gson:gson:2.8.7
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.7.1
@@ -143,8 +143,8 @@ com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.31.5
com.google.protobuf:protobuf-java-util:3.17.2
com.google.protobuf:protobuf-java:3.17.2
com.google.protobuf:protobuf-java-util:3.17.3
com.google.protobuf:protobuf-java:3.17.3
com.google.re2j:re2j:1.6
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
@@ -172,7 +172,7 @@ io.github.java-diff-utils:java-diff-utils:4.9
io.grpc:grpc-alts:1.36.0
io.grpc:grpc-api:1.36.0
io.grpc:grpc-auth:1.36.0
io.grpc:grpc-context:1.37.1
io.grpc:grpc-context:1.39.0
io.grpc:grpc-core:1.36.0
io.grpc:grpc-grpclb:1.36.0
io.grpc:grpc-netty-shaded:1.36.0

View File

@@ -36,6 +36,11 @@ import google.registry.gcs.GcsUtils;
import google.registry.model.common.DatabaseMigrationStateSchedule;
import google.registry.model.common.DatabaseMigrationStateSchedule.MigrationState;
import google.registry.model.common.DatabaseMigrationStateSchedule.ReplayDirection;
import google.registry.model.replay.DatastoreEntity;
import google.registry.model.replay.DatastoreOnlyEntity;
import google.registry.model.replay.NonReplicatedEntity;
import google.registry.model.replay.ReplaySpecializer;
import google.registry.model.replay.SqlReplayCheckpoint;
import google.registry.model.server.Lock;
import google.registry.model.translators.VKeyTranslatorFactory;
import google.registry.persistence.VKey;
@@ -44,11 +49,6 @@ import google.registry.request.Action.Method;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.schema.replay.DatastoreEntity;
import google.registry.schema.replay.DatastoreOnlyEntity;
import google.registry.schema.replay.NonReplicatedEntity;
import google.registry.schema.replay.ReplaySpecializer;
import google.registry.schema.replay.SqlReplayCheckpoint;
import google.registry.util.Clock;
import google.registry.util.RequestStatusChecker;
import java.io.IOException;
@@ -58,6 +58,7 @@ import javax.inject.Inject;
import javax.servlet.http.HttpServletResponse;
import org.joda.time.DateTime;
import org.joda.time.Duration;
import org.joda.time.Seconds;
/** Action that replays commit logs to Cloud SQL to keep it up to date. */
@Action(
@@ -97,7 +98,8 @@ public class ReplayCommitLogsToSqlAction implements Runnable {
@Override
public void run() {
MigrationState state = DatabaseMigrationStateSchedule.getValueAtTime(clock.nowUtc());
DateTime startTime = clock.nowUtc();
MigrationState state = DatabaseMigrationStateSchedule.getValueAtTime(startTime);
if (!state.getReplayDirection().equals(ReplayDirection.DATASTORE_TO_SQL)) {
String message =
String.format(
@@ -126,7 +128,7 @@ public class ReplayCommitLogsToSqlAction implements Runnable {
if (dryRun) {
resultMessage = executeDryRun();
} else {
resultMessage = replayFiles();
resultMessage = replayFiles(startTime);
}
response.setStatus(SC_OK);
response.setPayload(resultMessage);
@@ -160,10 +162,11 @@ public class ReplayCommitLogsToSqlAction implements Runnable {
fileBatch.stream().limit(10).collect(toImmutableList()));
}
private String replayFiles() {
DateTime replayTimeoutTime = clock.nowUtc().plus(REPLAY_TIMEOUT_DURATION);
private String replayFiles(DateTime startTime) {
DateTime replayTimeoutTime = startTime.plus(REPLAY_TIMEOUT_DURATION);
DateTime searchStartTime = jpaTm().transact(() -> SqlReplayCheckpoint.get().plusMillis(1));
int filesProcessed = 0;
int transactionsProcessed = 0;
// Starting from one millisecond after the last file we processed, search for and import files
// one hour at a time until we catch up to the current time or we hit the replay timeout (in
// which case the next run will pick up from where we leave off).
@@ -171,12 +174,12 @@ public class ReplayCommitLogsToSqlAction implements Runnable {
// We use hour-long batches because GCS supports filename prefix-based searches.
while (true) {
if (isAtOrAfter(clock.nowUtc(), replayTimeoutTime)) {
return String.format(
"Reached max execution time after replaying %d file(s).", filesProcessed);
return createResponseString(
"Reached max execution time", startTime, filesProcessed, transactionsProcessed);
}
if (isBeforeOrAt(clock.nowUtc(), searchStartTime)) {
return String.format(
"Caught up to current time after replaying %d file(s).", filesProcessed);
return createResponseString(
"Caught up to current time", startTime, filesProcessed, transactionsProcessed);
}
// Search through the end of the hour
DateTime searchEndTime =
@@ -189,29 +192,45 @@ public class ReplayCommitLogsToSqlAction implements Runnable {
searchStartTime.toString("yyyy-MM-dd HH"));
}
for (BlobInfo file : fileBatch) {
jpaTm().transact(() -> processFile(file));
transactionsProcessed += processFile(file);
filesProcessed++;
if (clock.nowUtc().isAfter(replayTimeoutTime)) {
return String.format(
"Reached max execution time after replaying %d file(s).", filesProcessed);
return createResponseString(
"Reached max execution time", startTime, filesProcessed, transactionsProcessed);
}
}
searchStartTime = searchEndTime.plusMillis(1);
}
}
private void processFile(BlobInfo metadata) {
private String createResponseString(
String msg, DateTime startTime, int filesProcessed, int transactionsProcessed) {
double tps =
(double) transactionsProcessed
/ (double) Seconds.secondsBetween(startTime, clock.nowUtc()).getSeconds();
return String.format(
"%s after replaying %d file(s) containing %d total transaction(s) (%.2f tx/s).",
msg, filesProcessed, transactionsProcessed, tps);
}
/**
* Replays the commit logs in the given commit log file and returns the number of transactions
* committed.
*/
private int processFile(BlobInfo metadata) {
try (InputStream input = gcsUtils.openInputStream(metadata.getBlobId())) {
// Load and process the Datastore transactions one at a time
ImmutableList<ImmutableList<VersionedEntity>> allTransactions =
CommitLogImports.loadEntitiesByTransaction(input);
allTransactions.forEach(this::replayTransaction);
allTransactions.forEach(
transaction -> jpaTm().transact(() -> replayTransaction(transaction)));
// if we succeeded, set the last-seen time
DateTime checkpoint = DateTime.parse(metadata.getName().substring(DIFF_FILE_PREFIX.length()));
SqlReplayCheckpoint.set(checkpoint);
jpaTm().transact(() -> SqlReplayCheckpoint.set(checkpoint));
logger.atInfo().log(
"Replayed %d transactions from commit log file %s with size %d B.",
allTransactions.size(), metadata.getName(), metadata.getSize());
return allTransactions.size();
} catch (IOException e) {
throw new RuntimeException(
"Errored out while replaying commit log file " + metadata.getName(), e);

View File

@@ -28,10 +28,10 @@ import com.googlecode.objectify.Key;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.EppResource;
import google.registry.model.ImmutableObject;
import google.registry.model.domain.RegistryLock;
import google.registry.model.eppcommon.Trid;
import google.registry.model.host.HostResource;
import google.registry.persistence.VKey;
import google.registry.schema.domain.RegistryLock;
import google.registry.util.AppEngineServiceUtils;
import google.registry.util.Retrier;
import javax.inject.Inject;

View File

@@ -21,8 +21,8 @@ import static google.registry.config.RegistryEnvironment.PRODUCTION;
import static google.registry.mapreduce.MapreduceRunner.PARAM_DRY_RUN;
import static google.registry.model.ResourceTransferUtils.updateForeignKeyIndexDeletionTime;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.model.registry.Registries.getTldsOfType;
import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_DELETE;
import static google.registry.model.tld.Registries.getTldsOfType;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static google.registry.request.RequestParameters.PARAM_TLDS;
@@ -47,8 +47,8 @@ import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.index.EppResourceIndex;
import google.registry.model.index.ForeignKeyIndex;
import google.registry.model.registry.Registry;
import google.registry.model.registry.Registry.TldType;
import google.registry.model.tld.Registry;
import google.registry.model.tld.Registry.TldType;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.Response;

View File

@@ -52,9 +52,9 @@ import google.registry.model.common.Cursor;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.Period;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.tld.Registry;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Parameter;

View File

@@ -28,16 +28,16 @@ import com.google.common.flogger.FluentLogger;
import com.google.common.net.MediaType;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.RegistryLock;
import google.registry.model.eppcommon.StatusValue;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarContact;
import google.registry.model.registry.RegistryLockDao;
import google.registry.model.tld.RegistryLockDao;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.schema.domain.RegistryLock;
import google.registry.tools.DomainLockUtils;
import google.registry.util.DateTimeUtils;
import google.registry.util.EmailMessage;

View File

@@ -22,7 +22,7 @@ import static javax.servlet.http.HttpServletResponse.SC_OK;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryEnvironment;
import google.registry.persistence.PersistenceModule.SchemaManagerConnection;
import google.registry.request.Action;
import google.registry.request.Response;
@@ -48,22 +48,18 @@ import javax.inject.Inject;
public class WipeOutCloudSqlAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
// As a short-lived class, hardcode allowed projects here instead of using config files.
private static final ImmutableSet<String> ALLOWED_PROJECTS =
ImmutableSet.of("domain-registry-qa");
private static final ImmutableSet<RegistryEnvironment> FORBIDDEN_ENVIRONMENTS =
ImmutableSet.of(RegistryEnvironment.PRODUCTION, RegistryEnvironment.SANDBOX);
private final String projectId;
private final Supplier<Connection> connectionSupplier;
private final Response response;
private final Retrier retrier;
@Inject
WipeOutCloudSqlAction(
@Config("projectId") String projectId,
@SchemaManagerConnection Supplier<Connection> connectionSupplier,
Response response,
Retrier retrier) {
this.projectId = projectId;
this.connectionSupplier = connectionSupplier;
this.response = response;
this.retrier = retrier;
@@ -73,9 +69,9 @@ public class WipeOutCloudSqlAction implements Runnable {
public void run() {
response.setContentType(PLAIN_TEXT_UTF_8);
if (!ALLOWED_PROJECTS.contains(projectId)) {
if (FORBIDDEN_ENVIRONMENTS.contains(RegistryEnvironment.get())) {
response.setStatus(SC_FORBIDDEN);
response.setPayload("Wipeout is not allowed in " + projectId);
response.setPayload("Wipeout is not allowed in " + RegistryEnvironment.get());
return;
}
@@ -90,11 +86,11 @@ public class WipeOutCloudSqlAction implements Runnable {
},
e -> !(e instanceof SQLException));
response.setStatus(SC_OK);
response.setPayload("Wiped out Cloud SQL in " + projectId);
response.setPayload("Wiped out Cloud SQL in " + RegistryEnvironment.get());
} catch (RuntimeException e) {
logger.atSevere().withCause(e).log("Failed to wipe out Cloud SQL data.");
response.setStatus(SC_INTERNAL_SERVER_ERROR);
response.setPayload("Failed to wipe out Cloud SQL in " + projectId);
response.setPayload("Failed to wipe out Cloud SQL in " + RegistryEnvironment.get());
}
}

View File

@@ -28,6 +28,7 @@ import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryEnvironment;
import google.registry.request.Action;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
@@ -49,9 +50,8 @@ public class WipeoutDatastoreAction implements Runnable {
private static final String PIPELINE_NAME = "bulk_delete_datastore_pipeline";
// As a short-lived class, hardcode allowed projects here instead of using config files.
private static final ImmutableSet<String> ALLOWED_PROJECTS =
ImmutableSet.of("domain-registry-qa");
private static final ImmutableSet<RegistryEnvironment> FORBIDDEN_ENVIRONMENTS =
ImmutableSet.of(RegistryEnvironment.PRODUCTION, RegistryEnvironment.SANDBOX);
private final String projectId;
private final String jobRegion;
@@ -80,9 +80,9 @@ public class WipeoutDatastoreAction implements Runnable {
public void run() {
response.setContentType(PLAIN_TEXT_UTF_8);
if (!ALLOWED_PROJECTS.contains(projectId)) {
if (FORBIDDEN_ENVIRONMENTS.contains(RegistryEnvironment.get())) {
response.setStatus(SC_FORBIDDEN);
response.setPayload("Wipeout is not allowed in " + projectId);
response.setPayload("Wipeout is not allowed in " + RegistryEnvironment.get());
return;
}

View File

@@ -23,6 +23,7 @@ import com.google.common.collect.Streams;
import google.registry.backup.AppEngineEnvironment;
import google.registry.beam.common.RegistryQuery.CriteriaQuerySupplier;
import google.registry.model.ofy.ObjectifyService;
import google.registry.model.replay.SqlEntity;
import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.persistence.transaction.TransactionManagerFactory;
import java.io.Serializable;
@@ -358,17 +359,17 @@ public final class RegistryJpaIO {
@ProcessElement
public void processElement(@Element KV<ShardedKey<Integer>, Iterable<T>> kv) {
try (AppEngineEnvironment env = new AppEngineEnvironment()) {
ImmutableList<Object> ofyEntities =
ImmutableList<Object> entities =
Streams.stream(kv.getValue())
.map(this.jpaConverter::apply)
// TODO(b/177340730): post migration delete the line below.
.filter(Objects::nonNull)
.collect(ImmutableList.toImmutableList());
try {
jpaTm().transact(() -> jpaTm().putAll(ofyEntities));
counter.inc(ofyEntities.size());
jpaTm().transact(() -> jpaTm().putAll(entities));
counter.inc(entities.size());
} catch (RuntimeException e) {
processSingly(ofyEntities);
processSingly(entities);
}
}
}
@@ -377,19 +378,22 @@ public final class RegistryJpaIO {
* Writes entities in a failed batch one by one to identify the first bad entity and throws a
* {@link RuntimeException} on it.
*/
private void processSingly(ImmutableList<Object> ofyEntities) {
for (Object ofyEntity : ofyEntities) {
private void processSingly(ImmutableList<Object> entities) {
for (Object entity : entities) {
try {
jpaTm().transact(() -> jpaTm().put(ofyEntity));
jpaTm().transact(() -> jpaTm().put(entity));
counter.inc();
} catch (RuntimeException e) {
throw new RuntimeException(toOfyKey(ofyEntity).toString(), e);
throw new RuntimeException(toEntityKeyString(entity), e);
}
}
}
private com.googlecode.objectify.Key<?> toOfyKey(Object ofyEntity) {
return com.googlecode.objectify.Key.create(ofyEntity);
private String toEntityKeyString(Object entity) {
if (entity instanceof SqlEntity) {
return ((SqlEntity) entity).getPrimaryKeyString();
}
return "Non-SqlEntity: " + String.valueOf(entity);
}
}
}

View File

@@ -32,8 +32,8 @@ import google.registry.model.host.HostResource;
import google.registry.model.poll.PollMessage;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarContact;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.tld.Registry;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import java.io.Serializable;
import java.util.Collection;

View File

@@ -39,9 +39,9 @@ import google.registry.backup.VersionedEntity;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.domain.DomainBase;
import google.registry.model.replay.DatastoreAndSqlEntity;
import google.registry.model.replay.SqlEntity;
import google.registry.model.reporting.HistoryEntry;
import google.registry.schema.replay.DatastoreAndSqlEntity;
import google.registry.schema.replay.SqlEntity;
import google.registry.tools.LevelDbLogReader;
import java.io.Serializable;
import java.util.Collection;

View File

@@ -14,18 +14,29 @@
package google.registry.beam.invoicing;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.beam.BeamUtils.getQueryFromFile;
import static org.apache.beam.sdk.values.TypeDescriptors.strings;
import google.registry.beam.common.RegistryJpaIO;
import google.registry.beam.common.RegistryJpaIO.Read;
import google.registry.beam.invoicing.BillingEvent.InvoiceGroupingKey;
import google.registry.beam.invoicing.BillingEvent.InvoiceGroupingKey.InvoiceGroupingKeyCoder;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.registrar.Registrar;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.reporting.billing.BillingModule;
import google.registry.util.DateTimeUtils;
import google.registry.util.DomainNameUtils;
import google.registry.util.SqlTemplate;
import java.io.Serializable;
import java.time.LocalDateTime;
import java.time.LocalTime;
import java.time.YearMonth;
import java.time.ZoneId;
import java.time.format.DateTimeFormatter;
import java.util.Optional;
import java.util.regex.Pattern;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.PipelineResult;
import org.apache.beam.sdk.coders.SerializableCoder;
@@ -58,6 +69,9 @@ public class InvoicingPipeline implements Serializable {
private static final DateTimeFormatter TIMESTAMP_FORMATTER =
DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.SSSSSS");
private static final Pattern SQL_COMMENT_REGEX =
Pattern.compile("^\\s*--.*\\n", Pattern.MULTILINE);
private final InvoicingPipelineOptions options;
InvoicingPipeline(InvoicingPipelineOptions options) {
@@ -71,21 +85,60 @@ public class InvoicingPipeline implements Serializable {
}
void setupPipeline(Pipeline pipeline) {
options.setIsolationOverride(TransactionIsolationLevel.TRANSACTION_READ_COMMITTED);
PCollection<BillingEvent> billingEvents =
pipeline.apply(
"Read BillingEvents from Bigquery",
BigQueryIO.read(BillingEvent::parseFromRecord)
.fromQuery(makeQuery(options.getYearMonth(), options.getProject()))
.withCoder(SerializableCoder.of(BillingEvent.class))
.usingStandardSql()
.withoutValidation()
.withTemplateCompatibility());
options.getDatabase().equals("DATASTORE")
? readFromBigQuery(options, pipeline)
: readFromCloudSql(options, pipeline);
saveInvoiceCsv(billingEvents, options);
saveDetailedCsv(billingEvents, options);
}
static PCollection<BillingEvent> readFromBigQuery(
InvoicingPipelineOptions options, Pipeline pipeline) {
return pipeline.apply(
"Read BillingEvents from Bigquery",
BigQueryIO.read(BillingEvent::parseFromRecord)
.fromQuery(makeQuery(options.getYearMonth(), options.getProject()))
.withCoder(SerializableCoder.of(BillingEvent.class))
.usingStandardSql()
.withoutValidation()
.withTemplateCompatibility());
}
static PCollection<BillingEvent> readFromCloudSql(
InvoicingPipelineOptions options, Pipeline pipeline) {
Read<Object[], BillingEvent> read =
RegistryJpaIO.read(
makeCloudSqlQuery(options.getYearMonth()), false, InvoicingPipeline::parseRow);
return pipeline.apply("Read BillingEvents from Cloud SQL", read);
}
private static BillingEvent parseRow(Object[] row) {
google.registry.model.billing.BillingEvent.OneTime oneTime =
(google.registry.model.billing.BillingEvent.OneTime) row[0];
Registrar registrar = (Registrar) row[1];
return BillingEvent.create(
oneTime.getId(),
DateTimeUtils.toZonedDateTime(oneTime.getBillingTime(), ZoneId.of("UTC")),
DateTimeUtils.toZonedDateTime(oneTime.getEventTime(), ZoneId.of("UTC")),
registrar.getClientId(),
registrar.getBillingIdentifier().toString(),
registrar.getPoNumber().orElse(""),
DomainNameUtils.getTldFromDomainName(oneTime.getTargetId()),
oneTime.getReason().toString(),
oneTime.getTargetId(),
oneTime.getDomainRepoId(),
Optional.ofNullable(oneTime.getPeriodYears()).orElse(0),
oneTime.getCost().getCurrencyUnit().toString(),
oneTime.getCost().getAmount().doubleValue(),
String.join(
" ", oneTime.getFlags().stream().map(Flag::toString).collect(toImmutableSet())));
}
/** Transform that converts a {@code BillingEvent} into an invoice CSV row. */
private static class GenerateInvoiceRows
extends PTransform<PCollection<BillingEvent>, PCollection<String>> {
@@ -171,6 +224,21 @@ public class InvoicingPipeline implements Serializable {
.build();
}
/** Create the Cloud SQL query for a given yearMonth at runtime. */
static String makeCloudSqlQuery(String yearMonth) {
YearMonth endMonth = YearMonth.parse(yearMonth).plusMonths(1);
String queryWithComments =
SqlTemplate.create(
getQueryFromFile(InvoicingPipeline.class, "cloud_sql_billing_events.sql"))
.put("FIRST_TIMESTAMP_OF_MONTH", yearMonth.concat("-01"))
.put(
"LAST_TIMESTAMP_OF_MONTH",
String.format("%d-%d-01", endMonth.getYear(), endMonth.getMonthValue()))
.build();
// Remove the comments from the query string
return SQL_COMMENT_REGEX.matcher(queryWithComments).replaceAll("");
}
public static void main(String[] args) {
PipelineOptionsFactory.register(InvoicingPipelineOptions.class);
InvoicingPipelineOptions options =

View File

@@ -30,6 +30,11 @@ public interface InvoicingPipelineOptions extends RegistryPipelineOptions {
void setInvoiceFilePrefix(String value);
@Description("The database to read data from.")
String getDatabase();
void setDatabase(String value);
@Description("The GCS bucket URL for invoices and detailed reports to be uploaded.")
String getBillingBucketUrl();

View File

@@ -30,7 +30,7 @@ import google.registry.model.common.Cursor;
import google.registry.model.rde.RdeMode;
import google.registry.model.rde.RdeNamingUtils;
import google.registry.model.rde.RdeRevision;
import google.registry.model.registry.Registry;
import google.registry.model.tld.Registry;
import google.registry.rde.DepositFragment;
import google.registry.rde.Ghostryde;
import google.registry.rde.PendingDeposit;

View File

@@ -60,7 +60,7 @@ import org.joda.time.Duration;
* <p>This class does not represent the total configuration of the Nomulus service. It's <b>only
* meant for settings that need to be configured <i>once</i></b>. Settings which may be subject to
* change in the future, should instead be retrieved from Datastore. The {@link
* google.registry.model.registry.Registry Registry} class is one such example of this.
* google.registry.model.tld.Registry Registry} class is one such example of this.
*
* <p>Note: Only settings that are actually configurable belong in this file. It's not a catch-all
* for anything widely used throughout the code base.
@@ -1394,8 +1394,8 @@ public final class RegistryConfig {
/**
* Returns the amount of time a domain label list should be cached in memory before expiring.
*
* @see google.registry.model.registry.label.ReservedList
* @see google.registry.model.registry.label.PremiumList
* @see google.registry.model.tld.label.ReservedList
* @see google.registry.model.tld.label.PremiumList
*/
public static Duration getDomainLabelListCacheDuration() {
return Duration.standardSeconds(CONFIG_SETTINGS.get().caching.domainLabelCachingSeconds);

View File

@@ -31,9 +31,9 @@ import static google.registry.cron.CronModule.FOR_EACH_TEST_TLD_PARAM;
import static google.registry.cron.CronModule.JITTER_SECONDS_PARAM;
import static google.registry.cron.CronModule.QUEUE_PARAM;
import static google.registry.cron.CronModule.RUN_IN_EMPTY_PARAM;
import static google.registry.model.registry.Registries.getTldsOfType;
import static google.registry.model.registry.Registry.TldType.REAL;
import static google.registry.model.registry.Registry.TldType.TEST;
import static google.registry.model.tld.Registries.getTldsOfType;
import static google.registry.model.tld.Registry.TldType.REAL;
import static google.registry.model.tld.Registry.TldType.TEST;
import static java.util.concurrent.TimeUnit.SECONDS;
import com.google.appengine.api.taskqueue.Queue;

View File

@@ -20,7 +20,7 @@ import static google.registry.dns.DnsConstants.DNS_PULL_QUEUE_NAME;
import static google.registry.dns.DnsConstants.DNS_TARGET_CREATE_TIME_PARAM;
import static google.registry.dns.DnsConstants.DNS_TARGET_NAME_PARAM;
import static google.registry.dns.DnsConstants.DNS_TARGET_TYPE_PARAM;
import static google.registry.model.registry.Registries.assertTldExists;
import static google.registry.model.tld.Registries.assertTldExists;
import static google.registry.request.RequestParameters.PARAM_TLD;
import static google.registry.util.DomainNameUtils.getTldFromDomainName;
import static java.util.concurrent.TimeUnit.MILLISECONDS;
@@ -38,7 +38,7 @@ import com.google.common.flogger.FluentLogger;
import com.google.common.net.InternetDomainName;
import com.google.common.util.concurrent.RateLimiter;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.model.registry.Registries;
import google.registry.model.tld.Registries;
import google.registry.util.Clock;
import google.registry.util.NonFinalForTesting;
import google.registry.util.SystemClock;

View File

@@ -19,7 +19,7 @@ import static com.google.common.base.Preconditions.checkState;
import com.google.common.collect.ImmutableMap;
import com.google.common.flogger.FluentLogger;
import google.registry.dns.writer.DnsWriter;
import google.registry.model.registry.Registry;
import google.registry.model.tld.Registry;
import java.util.Map;
import javax.inject.Inject;

View File

@@ -32,7 +32,7 @@ import google.registry.dns.DnsMetrics.ActionStatus;
import google.registry.dns.DnsMetrics.CommitStatus;
import google.registry.dns.DnsMetrics.PublishStatus;
import google.registry.dns.writer.DnsWriter;
import google.registry.model.registry.Registry;
import google.registry.model.tld.Registry;
import google.registry.request.Action;
import google.registry.request.HttpException.ServiceUnavailableException;
import google.registry.request.Parameter;

View File

@@ -47,8 +47,8 @@ import com.google.common.hash.HashFunction;
import com.google.common.hash.Hashing;
import google.registry.config.RegistryConfig.Config;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.model.registry.Registries;
import google.registry.model.registry.Registry;
import google.registry.model.tld.Registries;
import google.registry.model.tld.Registry;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;

View File

@@ -39,7 +39,7 @@ import google.registry.dns.writer.DnsWriterZone;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.secdns.DelegationSignerData;
import google.registry.model.host.HostResource;
import google.registry.model.registry.Registries;
import google.registry.model.tld.Registries;
import google.registry.util.Clock;
import google.registry.util.Concurrent;
import google.registry.util.Retrier;

View File

@@ -30,7 +30,7 @@ import google.registry.dns.writer.DnsWriterZone;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.secdns.DelegationSignerData;
import google.registry.model.host.HostResource;
import google.registry.model.registry.Registries;
import google.registry.model.tld.Registries;
import google.registry.util.Clock;
import java.io.IOException;
import java.net.Inet4Address;

View File

@@ -230,7 +230,7 @@
</cron>
<cron>
<url><![CDATA[/_dr/cron/fanout?queue=replay-commit-logs-to-sql&endpoint=/_dr/task/replayCommitLogsToSql&runInEmpty&dryRun=true]]></url>
<url><![CDATA[/_dr/cron/fanout?queue=replay-commit-logs-to-sql&endpoint=/_dr/task/replayCommitLogsToSql&runInEmpty]]></url>
<description>
Replays recent commit logs from Datastore to the SQL secondary backend.
</description>

View File

@@ -17,7 +17,7 @@ package google.registry.export;
import static com.google.common.base.Verify.verifyNotNull;
import static google.registry.mapreduce.inputs.EppResourceInputs.createEntityInput;
import static google.registry.model.EppResourceUtils.isActive;
import static google.registry.model.registry.Registries.getTldsOfType;
import static google.registry.model.tld.Registries.getTldsOfType;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
@@ -38,8 +38,8 @@ import google.registry.config.RegistryConfig.Config;
import google.registry.gcs.GcsUtils;
import google.registry.mapreduce.MapreduceRunner;
import google.registry.model.domain.DomainBase;
import google.registry.model.registry.Registry;
import google.registry.model.registry.Registry.TldType;
import google.registry.model.tld.Registry;
import google.registry.model.tld.Registry.TldType;
import google.registry.request.Action;
import google.registry.request.Response;
import google.registry.request.auth.Auth;

View File

@@ -30,14 +30,14 @@ import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import com.google.common.net.MediaType;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.registry.Registry;
import google.registry.model.registry.label.PremiumList.PremiumListEntry;
import google.registry.model.tld.Registry;
import google.registry.model.tld.label.PremiumList.PremiumEntry;
import google.registry.model.tld.label.PremiumListDao;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.RequestParameters;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.schema.tld.PremiumListDao;
import google.registry.storage.drive.DriveConnection;
import java.io.IOException;
import java.util.Optional;
@@ -143,8 +143,8 @@ public class ExportPremiumTermsAction implements Runnable {
PremiumListDao.getLatestRevision(premiumListName).isPresent(),
"Could not load premium list for " + tld);
SortedSet<String> premiumTerms =
Streams.stream(PremiumListDao.loadAllPremiumListEntries(premiumListName))
.map(PremiumListEntry::toString)
Streams.stream(PremiumListDao.loadAllPremiumEntries(premiumListName))
.map(PremiumEntry::toString)
.collect(ImmutableSortedSet.toImmutableSortedSet(String::compareTo));
return Joiner.on("\n")

View File

@@ -23,7 +23,7 @@ import static javax.servlet.http.HttpServletResponse.SC_OK;
import com.google.common.flogger.FluentLogger;
import com.google.common.net.MediaType;
import google.registry.model.registry.Registry;
import google.registry.model.tld.Registry;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.RequestParameters;

View File

@@ -16,10 +16,10 @@ package google.registry.export;
import com.google.common.base.Joiner;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.registry.Registry;
import google.registry.model.registry.label.ReservedList;
import google.registry.model.registry.label.ReservedList.ReservedListEntry;
import google.registry.model.registry.label.ReservedListDao;
import google.registry.model.tld.Registry;
import google.registry.model.tld.label.ReservedList;
import google.registry.model.tld.label.ReservedList.ReservedListEntry;
import google.registry.model.tld.label.ReservedListDao;
import java.util.Set;
import java.util.TreeSet;
import javax.inject.Inject;
@@ -48,7 +48,7 @@ public final class ExportUtils {
String.format("Reserved list %s does not exist", reservedListName)));
if (reservedList.getShouldPublish()) {
for (ReservedListEntry entry : reservedList.getReservedListEntries().values()) {
reservedTerms.add(entry.getLabel());
reservedTerms.add(entry.getDomainLabel());
}
}
}

View File

@@ -19,8 +19,8 @@ import static com.google.common.net.HttpHeaders.ACCESS_CONTROL_ALLOW_ORIGIN;
import static google.registry.flows.domain.DomainFlowUtils.validateDomainName;
import static google.registry.flows.domain.DomainFlowUtils.validateDomainNameWithIdnTables;
import static google.registry.flows.domain.DomainFlowUtils.verifyNotInPredelegation;
import static google.registry.model.registry.label.ReservationType.getTypeOfHighestSeverity;
import static google.registry.model.registry.label.ReservedList.getReservationTypes;
import static google.registry.model.tld.label.ReservationType.getTypeOfHighestSeverity;
import static google.registry.model.tld.label.ReservedList.getReservationTypes;
import static google.registry.monitoring.whitebox.CheckApiMetric.Availability.AVAILABLE;
import static google.registry.monitoring.whitebox.CheckApiMetric.Availability.REGISTERED;
import static google.registry.monitoring.whitebox.CheckApiMetric.Availability.RESERVED;
@@ -46,8 +46,8 @@ import google.registry.flows.domain.DomainFlowUtils.BadCommandForRegistryPhaseEx
import google.registry.flows.domain.DomainFlowUtils.InvalidIdnDomainLabelException;
import google.registry.model.domain.DomainBase;
import google.registry.model.index.ForeignKeyIndex;
import google.registry.model.registry.Registry;
import google.registry.model.registry.label.ReservationType;
import google.registry.model.tld.Registry;
import google.registry.model.tld.label.ReservationType;
import google.registry.monitoring.whitebox.CheckApiMetric;
import google.registry.monitoring.whitebox.CheckApiMetric.Availability;
import google.registry.request.Action;

View File

@@ -23,7 +23,7 @@ import google.registry.flows.domain.DomainPricingLogic;
import google.registry.flows.domain.FeesAndCredits;
import google.registry.model.ImmutableObject;
import google.registry.model.eppinput.EppInput;
import google.registry.model.registry.Registry;
import google.registry.model.tld.Registry;
import org.joda.time.DateTime;
/**

View File

@@ -28,8 +28,8 @@ import static google.registry.flows.domain.DomainFlowUtils.isValidReservedCreate
import static google.registry.flows.domain.DomainFlowUtils.validateDomainName;
import static google.registry.flows.domain.DomainFlowUtils.validateDomainNameWithIdnTables;
import static google.registry.flows.domain.DomainFlowUtils.verifyNotInPredelegation;
import static google.registry.model.registry.Registry.TldState.START_DATE_SUNRISE;
import static google.registry.model.registry.label.ReservationType.getTypeOfHighestSeverity;
import static google.registry.model.tld.Registry.TldState.START_DATE_SUNRISE;
import static google.registry.model.tld.label.ReservationType.getTypeOfHighestSeverity;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
@@ -67,10 +67,10 @@ import google.registry.model.eppoutput.CheckData.DomainCheckData;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.eppoutput.EppResponse.ResponseExtension;
import google.registry.model.index.ForeignKeyIndex;
import google.registry.model.registry.Registry;
import google.registry.model.registry.Registry.TldState;
import google.registry.model.registry.label.ReservationType;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.tld.Registry.TldState;
import google.registry.model.tld.label.ReservationType;
import google.registry.persistence.VKey;
import google.registry.util.Clock;
import java.util.HashSet;

View File

@@ -43,8 +43,8 @@ import google.registry.model.domain.token.AllocationTokenExtension;
import google.registry.model.eppinput.EppInput;
import google.registry.model.eppinput.ResourceCommand;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.tmch.ClaimsListDao;
import google.registry.util.Clock;
import java.util.HashSet;

View File

@@ -44,11 +44,11 @@ import static google.registry.flows.domain.DomainFlowUtils.verifyUnitIsYears;
import static google.registry.model.EppResourceUtils.createDomainRepoId;
import static google.registry.model.IdService.allocateId;
import static google.registry.model.eppcommon.StatusValue.SERVER_HOLD;
import static google.registry.model.registry.Registry.TldState.GENERAL_AVAILABILITY;
import static google.registry.model.registry.Registry.TldState.QUIET_PERIOD;
import static google.registry.model.registry.Registry.TldState.START_DATE_SUNRISE;
import static google.registry.model.registry.label.ReservationType.NAME_COLLISION;
import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_CREATE;
import static google.registry.model.tld.Registry.TldState.GENERAL_AVAILABILITY;
import static google.registry.model.tld.Registry.TldState.QUIET_PERIOD;
import static google.registry.model.tld.Registry.TldState.START_DATE_SUNRISE;
import static google.registry.model.tld.label.ReservationType.NAME_COLLISION;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static google.registry.util.DateTimeUtils.leapSafeAddYears;
@@ -104,14 +104,14 @@ import google.registry.model.index.ForeignKeyIndex;
import google.registry.model.poll.PendingActionNotificationResponse.DomainPendingActionNotificationResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.poll.PollMessage.Autorenew;
import google.registry.model.registry.Registry;
import google.registry.model.registry.Registry.TldState;
import google.registry.model.registry.Registry.TldType;
import google.registry.model.registry.label.ReservationType;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.tld.Registry.TldState;
import google.registry.model.tld.Registry.TldType;
import google.registry.model.tld.label.ReservationType;
import google.registry.tmch.LordnTaskUtils;
import java.util.Optional;
import javax.inject.Inject;

View File

@@ -85,11 +85,11 @@ import google.registry.model.eppinput.EppInput;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.poll.PendingActionNotificationResponse.DomainPendingActionNotificationResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.registry.Registry;
import google.registry.model.registry.Registry.TldType;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.tld.Registry.TldType;
import google.registry.model.transfer.TransferStatus;
import java.util.Collections;
import java.util.Optional;

View File

@@ -27,17 +27,17 @@ import static com.google.common.collect.Sets.intersection;
import static com.google.common.collect.Sets.union;
import static google.registry.model.domain.DomainBase.MAX_REGISTRATION_YEARS;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.model.registry.Registries.findTldForName;
import static google.registry.model.registry.Registries.getTlds;
import static google.registry.model.registry.Registry.TldState.GENERAL_AVAILABILITY;
import static google.registry.model.registry.Registry.TldState.PREDELEGATION;
import static google.registry.model.registry.Registry.TldState.QUIET_PERIOD;
import static google.registry.model.registry.Registry.TldState.START_DATE_SUNRISE;
import static google.registry.model.registry.label.ReservationType.ALLOWED_IN_SUNRISE;
import static google.registry.model.registry.label.ReservationType.FULLY_BLOCKED;
import static google.registry.model.registry.label.ReservationType.NAME_COLLISION;
import static google.registry.model.registry.label.ReservationType.RESERVED_FOR_ANCHOR_TENANT;
import static google.registry.model.registry.label.ReservationType.RESERVED_FOR_SPECIFIC_USE;
import static google.registry.model.tld.Registries.findTldForName;
import static google.registry.model.tld.Registries.getTlds;
import static google.registry.model.tld.Registry.TldState.GENERAL_AVAILABILITY;
import static google.registry.model.tld.Registry.TldState.PREDELEGATION;
import static google.registry.model.tld.Registry.TldState.QUIET_PERIOD;
import static google.registry.model.tld.Registry.TldState.START_DATE_SUNRISE;
import static google.registry.model.tld.label.ReservationType.ALLOWED_IN_SUNRISE;
import static google.registry.model.tld.label.ReservationType.FULLY_BLOCKED;
import static google.registry.model.tld.label.ReservationType.NAME_COLLISION;
import static google.registry.model.tld.label.ReservationType.RESERVED_FOR_ANCHOR_TENANT;
import static google.registry.model.tld.label.ReservationType.RESERVED_FOR_SPECIFIC_USE;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.pricing.PricingEngineProxy.isDomainPremium;
@@ -119,13 +119,13 @@ import google.registry.model.host.HostResource;
import google.registry.model.poll.PollMessage;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.Registrar.State;
import google.registry.model.registry.Registry;
import google.registry.model.registry.Registry.TldState;
import google.registry.model.registry.label.ReservationType;
import google.registry.model.registry.label.ReservedList;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.tld.Registry;
import google.registry.model.tld.Registry.TldState;
import google.registry.model.tld.label.ReservationType;
import google.registry.model.tld.label.ReservedList;
import google.registry.model.tmch.ClaimsListDao;
import google.registry.persistence.VKey;
import google.registry.tldconfig.idn.IdnLabelValidator;

View File

@@ -32,7 +32,7 @@ import google.registry.model.domain.fee.BaseFee.FeeType;
import google.registry.model.domain.fee.Fee;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.pricing.PremiumPricingEngine.DomainPrices;
import google.registry.model.registry.Registry;
import google.registry.model.tld.Registry;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.util.Optional;

View File

@@ -71,10 +71,10 @@ import google.registry.model.eppinput.EppInput;
import google.registry.model.eppinput.ResourceCommand;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import java.util.Optional;
import javax.inject.Inject;
import org.joda.money.Money;

View File

@@ -64,10 +64,10 @@ import google.registry.model.eppinput.EppInput;
import google.registry.model.eppinput.ResourceCommand;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import java.util.Optional;
import javax.inject.Inject;
import org.joda.money.Money;

View File

@@ -57,9 +57,9 @@ import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.eppcommon.AuthInfo;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.DomainTransferData;
import google.registry.model.transfer.TransferStatus;
import java.util.Optional;

View File

@@ -45,9 +45,9 @@ import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.AuthInfo;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.TransferStatus;
import java.util.Optional;
import javax.inject.Inject;

View File

@@ -47,9 +47,9 @@ import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.AuthInfo;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.TransferStatus;
import java.util.Optional;
import javax.inject.Inject;

View File

@@ -66,10 +66,10 @@ import google.registry.model.eppinput.EppInput;
import google.registry.model.eppinput.ResourceCommand;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.DomainTransferData;
import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
import google.registry.model.transfer.TransferResponse.DomainTransferResponse;

View File

@@ -32,7 +32,7 @@ import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.eppcommon.Trid;
import google.registry.model.poll.PendingActionNotificationResponse.DomainPendingActionNotificationResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.registry.Registry;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.DomainTransferData;
import google.registry.model.transfer.TransferData;
import google.registry.model.transfer.TransferData.TransferServerApproveEntity;

View File

@@ -76,8 +76,8 @@ import google.registry.model.eppcommon.StatusValue;
import google.registry.model.eppinput.EppInput;
import google.registry.model.eppinput.ResourceCommand;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import java.util.Optional;
import javax.inject.Inject;
import org.joda.time.DateTime;

View File

@@ -21,7 +21,7 @@ import com.google.common.net.InternetDomainName;
import google.registry.flows.EppException;
import google.registry.model.domain.DomainCommand;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.registry.Registry;
import google.registry.model.tld.Registry;
import org.joda.time.DateTime;
/**

View File

@@ -31,8 +31,8 @@ import google.registry.model.domain.DomainCommand;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.domain.token.AllocationToken.TokenStatus;
import google.registry.model.domain.token.AllocationToken.TokenType;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.tld.Registry;
import google.registry.persistence.VKey;
import java.util.List;
import java.util.Optional;

View File

@@ -16,7 +16,7 @@ package google.registry.flows.host;
import static google.registry.model.EppResourceUtils.isActive;
import static google.registry.model.EppResourceUtils.loadByForeignKey;
import static google.registry.model.registry.Registries.findTldForName;
import static google.registry.model.tld.Registries.findTldForName;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import static java.util.stream.Collectors.joining;

View File

@@ -17,7 +17,6 @@ package google.registry.model;
import com.google.common.collect.ImmutableSet;
import google.registry.model.billing.BillingEvent;
import google.registry.model.common.Cursor;
import google.registry.model.common.DatabaseMigrationStateSchedule;
import google.registry.model.common.EntityGroupRoot;
import google.registry.model.common.GaeUserIdConverter;
import google.registry.model.contact.ContactHistory;
@@ -39,17 +38,16 @@ import google.registry.model.poll.PollMessage;
import google.registry.model.rde.RdeRevision;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarContact;
import google.registry.model.registry.Registry;
import google.registry.model.registry.label.PremiumList;
import google.registry.model.registry.label.ReservedList;
import google.registry.model.replay.LastSqlTransaction;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.server.Lock;
import google.registry.model.server.ServerSecret;
import google.registry.model.tld.Registry;
import google.registry.model.tld.label.ReservedList;
import google.registry.model.tmch.ClaimsList;
import google.registry.model.tmch.ClaimsList.ClaimsListRevision;
import google.registry.model.tmch.ClaimsList.ClaimsListSingleton;
import google.registry.model.tmch.TmchCrl;
import google.registry.schema.replay.LastSqlTransaction;
/** Sets of classes of the Objectify-registered entities in use throughout the model. */
public final class EntityClasses {
@@ -73,7 +71,6 @@ public final class EntityClasses {
ContactHistory.class,
ContactResource.class,
Cursor.class,
DatabaseMigrationStateSchedule.class,
DomainBase.class,
DomainHistory.class,
EntityGroupRoot.class,
@@ -91,9 +88,6 @@ public final class EntityClasses {
PollMessage.class,
PollMessage.Autorenew.class,
PollMessage.OneTime.class,
PremiumList.class,
PremiumList.PremiumListEntry.class,
PremiumList.PremiumListRevision.class,
RdeRevision.class,
Registrar.class,
RegistrarContact.class,

View File

@@ -40,9 +40,9 @@ import google.registry.model.host.HostResource;
import google.registry.model.index.ForeignKeyIndex;
import google.registry.model.ofy.CommitLogManifest;
import google.registry.model.ofy.CommitLogMutation;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.HistoryEntryDao;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.DomainTransferData;
import google.registry.model.transfer.TransferData;
import google.registry.model.transfer.TransferStatus;

View File

@@ -18,8 +18,8 @@ import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.model.registry.Registry.TldState.GENERAL_AVAILABILITY;
import static google.registry.model.registry.Registry.TldState.START_DATE_SUNRISE;
import static google.registry.model.tld.Registry.TldState.GENERAL_AVAILABILITY;
import static google.registry.model.tld.Registry.TldState.START_DATE_SUNRISE;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
@@ -36,11 +36,11 @@ import google.registry.model.pricing.StaticPremiumListPricingEngine;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarAddress;
import google.registry.model.registrar.RegistrarContact;
import google.registry.model.registry.Registry;
import google.registry.model.registry.Registry.TldState;
import google.registry.model.registry.label.PremiumList;
import google.registry.model.tld.Registry;
import google.registry.model.tld.Registry.TldState;
import google.registry.model.tld.label.PremiumList;
import google.registry.model.tld.label.PremiumListDao;
import google.registry.persistence.VKey;
import google.registry.schema.tld.PremiumListDao;
import google.registry.util.CidrAddressBlock;
import java.util.Collection;
import java.util.Optional;

View File

@@ -46,13 +46,13 @@ import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.replay.DatastoreAndSqlEntity;
import google.registry.model.replay.DatastoreOnlyEntity;
import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
import google.registry.persistence.BillingVKey.BillingEventVKey;
import google.registry.persistence.BillingVKey.BillingRecurrenceVKey;
import google.registry.persistence.VKey;
import google.registry.persistence.WithLongVKey;
import google.registry.schema.replay.DatastoreAndSqlEntity;
import google.registry.schema.replay.DatastoreOnlyEntity;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;

View File

@@ -30,9 +30,9 @@ import google.registry.model.ImmutableObject;
import google.registry.model.UpdateAutoTimestamp;
import google.registry.model.annotations.InCrossTld;
import google.registry.model.common.Cursor.CursorId;
import google.registry.model.registry.Registry;
import google.registry.model.replay.DatastoreAndSqlEntity;
import google.registry.model.tld.Registry;
import google.registry.persistence.VKey;
import google.registry.schema.replay.DatastoreAndSqlEntity;
import java.io.Serializable;
import java.util.List;
import java.util.Optional;

View File

@@ -15,7 +15,7 @@
package google.registry.model.common;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.persistence.transaction.TransactionManagerFactory.ofyTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import com.google.common.annotations.VisibleForTesting;
@@ -24,14 +24,13 @@ import com.google.common.cache.CacheLoader;
import com.google.common.cache.LoadingCache;
import com.google.common.collect.ImmutableMultimap;
import com.google.common.collect.ImmutableSortedMap;
import com.googlecode.objectify.annotation.Embed;
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Mapify;
import google.registry.model.annotations.InCrossTld;
import google.registry.model.common.TimedTransitionProperty.TimeMapper;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryEnvironment;
import google.registry.model.common.TimedTransitionProperty.TimedTransition;
import google.registry.schema.replay.DatastoreOnlyEntity;
import google.registry.model.replay.SqlOnlyEntity;
import java.time.Duration;
import javax.persistence.Entity;
import javax.persistence.PersistenceException;
import org.joda.time.DateTime;
/**
@@ -41,9 +40,9 @@ import org.joda.time.DateTime;
* of access.
*/
@Entity
@InCrossTld
public class DatabaseMigrationStateSchedule extends CrossTldSingleton
implements DatastoreOnlyEntity {
public class DatabaseMigrationStateSchedule extends CrossTldSingleton implements SqlOnlyEntity {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
public enum PrimaryDatabase {
CLOUD_SQL,
@@ -92,12 +91,11 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton
}
}
@Embed
public static class MigrationStateTransition extends TimedTransition<MigrationState> {
private MigrationState migrationState;
@Override
protected MigrationState getValue() {
public MigrationState getValue() {
return migrationState;
}
@@ -181,7 +179,6 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton
MigrationStateTransition.class);
@VisibleForTesting
@Mapify(TimeMapper.class)
public TimedTransitionProperty<MigrationState, MigrationStateTransition> migrationTransitions =
TimedTransitionProperty.forMapify(
MigrationState.DATASTORE_ONLY, MigrationStateTransition.class);
@@ -197,7 +194,7 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton
/** Sets and persists to Datastore the provided migration transition schedule. */
public static void set(ImmutableSortedMap<DateTime, MigrationState> migrationTransitionMap) {
ofyTm().assertInTransaction();
jpaTm().assertInTransaction();
TimedTransitionProperty<MigrationState, MigrationStateTransition> transitions =
TimedTransitionProperty.make(
migrationTransitionMap,
@@ -207,7 +204,7 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton
MigrationState.DATASTORE_ONLY,
"migrationTransitionMap must start with DATASTORE_ONLY");
validateTransitionAtCurrentTime(transitions);
ofyTm().put(new DatabaseMigrationStateSchedule(transitions));
jpaTm().put(new DatabaseMigrationStateSchedule(transitions));
CACHE.invalidateAll();
}
@@ -224,13 +221,23 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton
/** Loads the currently-set migration schedule from Datastore, or the default if none exists. */
@VisibleForTesting
static TimedTransitionProperty<MigrationState, MigrationStateTransition> getUncached() {
return ofyTm()
.transactNew(
() ->
ofyTm()
return jpaTm()
.transactWithoutBackup(
() -> {
try {
return jpaTm()
.loadSingleton(DatabaseMigrationStateSchedule.class)
.map(s -> s.migrationTransitions)
.orElse(DEFAULT_TRANSITION_MAP));
.orElse(DEFAULT_TRANSITION_MAP);
} catch (PersistenceException e) {
if (!RegistryEnvironment.get().equals(RegistryEnvironment.UNITTEST)) {
throw e;
}
logger.atWarning().withCause(e).log(
"Error when retrieving migration schedule; this should only happen in tests.");
return DEFAULT_TRANSITION_MAP;
}
});
}
/**
@@ -241,8 +248,8 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton
*/
private static void validateTransitionAtCurrentTime(
TimedTransitionProperty<MigrationState, MigrationStateTransition> newTransitions) {
MigrationState currentValue = getUncached().getValueAtTime(ofyTm().getTransactionTime());
MigrationState nextCurrentValue = newTransitions.getValueAtTime(ofyTm().getTransactionTime());
MigrationState currentValue = getUncached().getValueAtTime(jpaTm().getTransactionTime());
MigrationState nextCurrentValue = newTransitions.getValueAtTime(jpaTm().getTransactionTime());
checkArgument(
VALID_STATE_TRANSITIONS.get(currentValue).contains(nextCurrentValue),
"Cannot transition from current state-as-of-now %s to new state-as-of-now %s",

View File

@@ -19,7 +19,7 @@ import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Id;
import google.registry.model.BackupGroupRoot;
import google.registry.schema.replay.DatastoreOnlyEntity;
import google.registry.model.replay.DatastoreOnlyEntity;
import javax.annotation.Nullable;
/**

View File

@@ -21,10 +21,10 @@ import com.googlecode.objectify.annotation.EntitySubclass;
import google.registry.model.EppResource;
import google.registry.model.ImmutableObject;
import google.registry.model.contact.ContactHistory.ContactHistoryId;
import google.registry.model.replay.DatastoreEntity;
import google.registry.model.replay.SqlEntity;
import google.registry.model.reporting.HistoryEntry;
import google.registry.persistence.VKey;
import google.registry.schema.replay.DatastoreEntity;
import google.registry.schema.replay.SqlEntity;
import java.io.Serializable;
import java.util.Optional;
import javax.annotation.Nullable;

View File

@@ -19,9 +19,9 @@ import com.googlecode.objectify.annotation.Entity;
import google.registry.model.EppResource.ForeignKeyedEppResource;
import google.registry.model.annotations.ExternalMessagingName;
import google.registry.model.annotations.ReportedOn;
import google.registry.model.replay.DatastoreAndSqlEntity;
import google.registry.persistence.VKey;
import google.registry.persistence.WithStringVKey;
import google.registry.schema.replay.DatastoreAndSqlEntity;
import javax.persistence.Access;
import javax.persistence.AccessType;
import org.joda.time.DateTime;

View File

@@ -21,9 +21,9 @@ import google.registry.model.annotations.ExternalMessagingName;
import google.registry.model.annotations.ReportedOn;
import google.registry.model.domain.secdns.DelegationSignerData;
import google.registry.model.host.HostResource;
import google.registry.model.replay.DatastoreAndSqlEntity;
import google.registry.persistence.VKey;
import google.registry.persistence.WithStringVKey;
import google.registry.schema.replay.DatastoreAndSqlEntity;
import java.util.Set;
import javax.persistence.Access;
import javax.persistence.AccessType;

View File

@@ -62,8 +62,9 @@ import google.registry.model.domain.secdns.DelegationSignerData;
import google.registry.model.eppcommon.StatusValue;
import google.registry.model.host.HostResource;
import google.registry.model.poll.PollMessage;
import google.registry.model.registry.Registry;
import google.registry.model.replay.ReplaySpecializer;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.DomainTransferData;
import google.registry.model.transfer.TransferStatus;
import google.registry.persistence.VKey;
@@ -374,7 +375,7 @@ public class DomainContent extends EppResource
/**
* Callback to delete grace periods and DelegationSignerData records prior to domain delete.
*
* <p>See {@link google.registry.schema.replay.ReplaySpecializer}.
* <p>See {@link ReplaySpecializer}.
*/
public static void beforeSqlDelete(VKey<DomainBase> key) {
// Delete all grace periods associated with the domain.

View File

@@ -28,11 +28,11 @@ import google.registry.model.domain.GracePeriod.GracePeriodHistory;
import google.registry.model.domain.secdns.DelegationSignerData;
import google.registry.model.domain.secdns.DomainDsDataHistory;
import google.registry.model.host.HostResource;
import google.registry.model.replay.DatastoreEntity;
import google.registry.model.replay.SqlEntity;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.HistoryEntry;
import google.registry.persistence.VKey;
import google.registry.schema.replay.DatastoreEntity;
import google.registry.schema.replay.SqlEntity;
import java.io.Serializable;
import java.util.HashSet;
import java.util.Optional;

View File

@@ -23,10 +23,10 @@ import com.googlecode.objectify.annotation.Embed;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.replay.DatastoreAndSqlEntity;
import google.registry.persistence.BillingVKey.BillingEventVKey;
import google.registry.persistence.BillingVKey.BillingRecurrenceVKey;
import google.registry.persistence.VKey;
import google.registry.schema.replay.DatastoreAndSqlEntity;
import javax.annotation.Nullable;
import javax.persistence.Access;
import javax.persistence.AccessType;

View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.domain;
package google.registry.model.domain;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
@@ -23,7 +23,7 @@ import google.registry.model.Buildable;
import google.registry.model.CreateAutoTimestamp;
import google.registry.model.ImmutableObject;
import google.registry.model.UpdateAutoTimestamp;
import google.registry.schema.replay.SqlOnlyEntity;
import google.registry.model.replay.SqlOnlyEntity;
import google.registry.util.DateTimeUtils;
import java.time.ZonedDateTime;
import java.util.Optional;
@@ -232,7 +232,7 @@ public final class RegistryLock extends ImmutableObject implements Buildable, Sq
return new Builder(clone(this));
}
/** Builder for {@link google.registry.schema.domain.RegistryLock}. */
/** Builder for {@link RegistryLock}. */
public static class Builder extends Buildable.Builder<RegistryLock> {
public Builder() {}

View File

@@ -17,7 +17,7 @@ package google.registry.model.domain.secdns;
import static google.registry.model.IdService.allocateId;
import google.registry.model.domain.DomainHistory;
import google.registry.schema.replay.SqlOnlyEntity;
import google.registry.model.replay.SqlOnlyEntity;
import javax.persistence.Access;
import javax.persistence.AccessType;
import javax.persistence.Column;

View File

@@ -46,11 +46,11 @@ import google.registry.model.annotations.ReportedOn;
import google.registry.model.common.TimedTransitionProperty;
import google.registry.model.common.TimedTransitionProperty.TimeMapper;
import google.registry.model.common.TimedTransitionProperty.TimedTransition;
import google.registry.model.replay.DatastoreAndSqlEntity;
import google.registry.model.reporting.HistoryEntry;
import google.registry.persistence.DomainHistoryVKey;
import google.registry.persistence.VKey;
import google.registry.persistence.WithStringVKey;
import google.registry.schema.replay.DatastoreAndSqlEntity;
import java.util.Optional;
import java.util.Set;
import javax.annotation.Nullable;

View File

@@ -21,10 +21,10 @@ import com.googlecode.objectify.annotation.EntitySubclass;
import google.registry.model.EppResource;
import google.registry.model.ImmutableObject;
import google.registry.model.host.HostHistory.HostHistoryId;
import google.registry.model.replay.DatastoreEntity;
import google.registry.model.replay.SqlEntity;
import google.registry.model.reporting.HistoryEntry;
import google.registry.persistence.VKey;
import google.registry.schema.replay.DatastoreEntity;
import google.registry.schema.replay.SqlEntity;
import java.io.Serializable;
import java.util.Optional;
import javax.annotation.Nullable;

View File

@@ -19,9 +19,9 @@ import com.googlecode.objectify.annotation.Entity;
import google.registry.model.EppResource.ForeignKeyedEppResource;
import google.registry.model.annotations.ExternalMessagingName;
import google.registry.model.annotations.ReportedOn;
import google.registry.model.replay.DatastoreAndSqlEntity;
import google.registry.persistence.VKey;
import google.registry.persistence.WithStringVKey;
import google.registry.schema.replay.DatastoreAndSqlEntity;
import javax.persistence.Access;
import javax.persistence.AccessType;

View File

@@ -25,7 +25,7 @@ import com.googlecode.objectify.annotation.Parent;
import google.registry.model.BackupGroupRoot;
import google.registry.model.EppResource;
import google.registry.model.annotations.ReportedOn;
import google.registry.schema.replay.DatastoreOnlyEntity;
import google.registry.model.replay.DatastoreOnlyEntity;
/** An index that allows for quick enumeration of all EppResource entities (e.g. via map reduce). */
@ReportedOn

View File

@@ -24,7 +24,7 @@ import com.googlecode.objectify.annotation.Id;
import google.registry.model.EppResource;
import google.registry.model.ImmutableObject;
import google.registry.model.annotations.VirtualEntity;
import google.registry.schema.replay.DatastoreOnlyEntity;
import google.registry.model.replay.DatastoreOnlyEntity;
/** A virtual entity to represent buckets to which EppResourceIndex objects are randomly added. */
@Entity

View File

@@ -47,9 +47,9 @@ import google.registry.model.annotations.ReportedOn;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.DomainBase;
import google.registry.model.host.HostResource;
import google.registry.model.replay.DatastoreOnlyEntity;
import google.registry.persistence.VKey;
import google.registry.persistence.transaction.CriteriaQueryBuilder;
import google.registry.schema.replay.DatastoreOnlyEntity;
import google.registry.util.NonFinalForTesting;
import java.util.Collection;
import java.util.Comparator;

View File

@@ -33,7 +33,7 @@ import google.registry.model.Buildable;
import google.registry.model.ImmutableObject;
import google.registry.model.annotations.NotBackedUp;
import google.registry.model.annotations.NotBackedUp.Reason;
import google.registry.schema.replay.DatastoreOnlyEntity;
import google.registry.model.replay.DatastoreOnlyEntity;
import google.registry.util.NonFinalForTesting;
import java.util.Random;
import java.util.function.Supplier;

View File

@@ -27,7 +27,7 @@ import com.googlecode.objectify.annotation.Parent;
import google.registry.model.ImmutableObject;
import google.registry.model.annotations.NotBackedUp;
import google.registry.model.annotations.NotBackedUp.Reason;
import google.registry.schema.replay.DatastoreOnlyEntity;
import google.registry.model.replay.DatastoreOnlyEntity;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;

View File

@@ -23,7 +23,7 @@ import com.googlecode.objectify.annotation.Id;
import google.registry.model.ImmutableObject;
import google.registry.model.annotations.NotBackedUp;
import google.registry.model.annotations.NotBackedUp.Reason;
import google.registry.schema.replay.DatastoreOnlyEntity;
import google.registry.model.replay.DatastoreOnlyEntity;
import org.joda.time.DateTime;
/** Singleton parent entity for all commit log checkpoints. */

View File

@@ -25,7 +25,7 @@ import com.googlecode.objectify.annotation.Parent;
import google.registry.model.ImmutableObject;
import google.registry.model.annotations.NotBackedUp;
import google.registry.model.annotations.NotBackedUp.Reason;
import google.registry.schema.replay.DatastoreOnlyEntity;
import google.registry.model.replay.DatastoreOnlyEntity;
import java.util.LinkedHashSet;
import java.util.Set;
import org.joda.time.DateTime;

View File

@@ -29,7 +29,7 @@ import com.googlecode.objectify.annotation.Parent;
import google.registry.model.ImmutableObject;
import google.registry.model.annotations.NotBackedUp;
import google.registry.model.annotations.NotBackedUp.Reason;
import google.registry.schema.replay.DatastoreOnlyEntity;
import google.registry.model.replay.DatastoreOnlyEntity;
/** Representation of a saved entity in a {@link CommitLogManifest} (not deletes). */
@Entity

View File

@@ -36,12 +36,12 @@ import google.registry.model.annotations.InCrossTld;
import google.registry.model.contact.ContactHistory;
import google.registry.model.domain.DomainHistory;
import google.registry.model.host.HostHistory;
import google.registry.model.replay.DatastoreEntity;
import google.registry.model.replay.SqlEntity;
import google.registry.model.reporting.HistoryEntry;
import google.registry.persistence.VKey;
import google.registry.persistence.transaction.QueryComposer;
import google.registry.persistence.transaction.TransactionManager;
import google.registry.schema.replay.DatastoreEntity;
import google.registry.schema.replay.SqlEntity;
import java.util.List;
import java.util.NoSuchElementException;
import java.util.Optional;

View File

@@ -325,7 +325,7 @@ public class Ofy {
}
/** Execute some work in a transactionless context. */
<R> R doTransactionless(Supplier<R> work) {
public <R> R doTransactionless(Supplier<R> work) {
try {
com.googlecode.objectify.ObjectifyService.push(
com.googlecode.objectify.ObjectifyService.ofy().transactionless());

View File

@@ -22,9 +22,9 @@ import com.google.common.collect.ImmutableMap;
import com.googlecode.objectify.Key;
import google.registry.config.RegistryEnvironment;
import google.registry.model.UpdateAutoTimestamp;
import google.registry.model.replay.DatastoreEntity;
import google.registry.model.replay.ReplaySpecializer;
import google.registry.persistence.VKey;
import google.registry.schema.replay.DatastoreEntity;
import google.registry.schema.replay.ReplaySpecializer;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.ConcurrentLinkedQueue;

View File

@@ -43,6 +43,7 @@ import google.registry.model.host.HostResource;
import google.registry.model.poll.PendingActionNotificationResponse.ContactPendingActionNotificationResponse;
import google.registry.model.poll.PendingActionNotificationResponse.DomainPendingActionNotificationResponse;
import google.registry.model.poll.PendingActionNotificationResponse.HostPendingActionNotificationResponse;
import google.registry.model.replay.DatastoreAndSqlEntity;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
import google.registry.model.transfer.TransferResponse;
@@ -50,7 +51,6 @@ import google.registry.model.transfer.TransferResponse.ContactTransferResponse;
import google.registry.model.transfer.TransferResponse.DomainTransferResponse;
import google.registry.persistence.VKey;
import google.registry.persistence.WithLongVKey;
import google.registry.schema.replay.DatastoreAndSqlEntity;
import java.util.List;
import java.util.Optional;
import javax.annotation.Nullable;

View File

@@ -18,8 +18,8 @@ import static com.google.common.base.Preconditions.checkNotNull;
import static google.registry.util.DomainNameUtils.getTldFromDomainName;
import com.google.common.net.InternetDomainName;
import google.registry.model.registry.Registry;
import google.registry.schema.tld.PremiumListDao;
import google.registry.model.tld.Registry;
import google.registry.model.tld.label.PremiumListDao;
import java.util.Optional;
import javax.inject.Inject;
import org.joda.money.Money;

View File

@@ -27,9 +27,9 @@ import com.googlecode.objectify.annotation.Ignore;
import google.registry.model.BackupGroupRoot;
import google.registry.model.ImmutableObject;
import google.registry.model.rde.RdeRevision.RdeRevisionId;
import google.registry.model.replay.NonReplicatedEntity;
import google.registry.persistence.VKey;
import google.registry.persistence.converter.LocalDateConverter;
import google.registry.schema.replay.NonReplicatedEntity;
import java.io.Serializable;
import java.util.Optional;
import javax.persistence.Column;

View File

@@ -31,7 +31,7 @@ import static google.registry.config.RegistryConfig.getDefaultRegistrarWhoisServ
import static google.registry.model.CacheUtils.memoizeWithShortExpiration;
import static google.registry.model.common.EntityGroupRoot.getCrossTldKey;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.model.registry.Registries.assertTldsExist;
import static google.registry.model.tld.Registries.assertTldsExist;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.persistence.transaction.TransactionManagerUtil.transactIfJpaTm;
@@ -77,9 +77,9 @@ import google.registry.model.annotations.InCrossTld;
import google.registry.model.annotations.ReportedOn;
import google.registry.model.common.EntityGroupRoot;
import google.registry.model.registrar.Registrar.BillingAccountEntry.CurrencyMapper;
import google.registry.model.registry.Registry;
import google.registry.model.replay.DatastoreAndSqlEntity;
import google.registry.model.tld.Registry;
import google.registry.persistence.VKey;
import google.registry.schema.replay.DatastoreAndSqlEntity;
import google.registry.util.CidrAddressBlock;
import java.security.cert.CertificateParsingException;
import java.util.Comparator;

View File

@@ -30,6 +30,7 @@ import static google.registry.util.PasswordUtils.SALT_SUPPLIER;
import static google.registry.util.PasswordUtils.hashPassword;
import static java.util.stream.Collectors.joining;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Enums;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedSet;
@@ -48,8 +49,8 @@ import google.registry.model.Jsonifiable;
import google.registry.model.annotations.InCrossTld;
import google.registry.model.annotations.ReportedOn;
import google.registry.model.registrar.RegistrarContact.RegistrarPocId;
import google.registry.model.replay.DatastoreAndSqlEntity;
import google.registry.persistence.VKey;
import google.registry.schema.replay.DatastoreAndSqlEntity;
import java.io.Serializable;
import java.util.Arrays;
import java.util.Map;
@@ -396,7 +397,8 @@ public class RegistrarContact extends ImmutableObject
}
/** Class to represent the composite primary key for {@link RegistrarContact} entity. */
static class RegistrarPocId extends ImmutableObject implements Serializable {
@VisibleForTesting
public static class RegistrarPocId extends ImmutableObject implements Serializable {
String emailAddress;
@@ -405,7 +407,8 @@ public class RegistrarContact extends ImmutableObject
// Hibernate requires this default constructor.
private RegistrarPocId() {}
RegistrarPocId(String emailAddress, String registrarId) {
@VisibleForTesting
public RegistrarPocId(String emailAddress, String registrarId) {
this.emailAddress = emailAddress;
this.registrarId = registrarId;
}

View File

@@ -1,371 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.registry.label;
import static com.google.common.base.Charsets.US_ASCII;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableMap.toImmutableMap;
import static com.google.common.hash.Funnels.stringFunnel;
import static com.google.common.hash.Funnels.unencodedCharsFunnel;
import static google.registry.model.IdService.allocateId;
import static google.registry.persistence.transaction.QueryComposer.Comparator.EQ;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableMap;
import com.google.common.hash.BloomFilter;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Id;
import com.googlecode.objectify.annotation.Ignore;
import com.googlecode.objectify.annotation.Parent;
import google.registry.model.Buildable;
import google.registry.model.ImmutableObject;
import google.registry.model.annotations.InCrossTld;
import google.registry.model.annotations.ReportedOn;
import google.registry.model.registry.Registry;
import google.registry.schema.replay.DatastoreOnlyEntity;
import google.registry.schema.replay.NonReplicatedEntity;
import google.registry.schema.tld.PremiumEntry;
import google.registry.schema.tld.PremiumListDao;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import javax.annotation.Nullable;
import javax.persistence.Column;
import javax.persistence.Index;
import javax.persistence.PostLoad;
import javax.persistence.PostPersist;
import javax.persistence.PostUpdate;
import javax.persistence.PrePersist;
import javax.persistence.PreRemove;
import javax.persistence.Table;
import javax.persistence.Transient;
import org.joda.money.CurrencyUnit;
import org.joda.money.Money;
/**
* A premium list entity that is used to check domain label prices.
*
* <p>Note that the primary key of this entity is {@link #revisionId}, which is auto-generated by
* the database. So, if a retry of insertion happens after the previous attempt unexpectedly
* succeeds, we will end up with having two exact same premium lists that differ only by revisionId.
* This is fine though, because we only use the list with the highest revisionId.
*/
@ReportedOn
@Entity
@javax.persistence.Entity
@Table(indexes = {@Index(columnList = "name", name = "premiumlist_name_idx")})
public final class PremiumList extends BaseDomainLabelList<Money, PremiumList.PremiumListEntry>
implements NonReplicatedEntity {
/** Stores the revision key for the set of currently used premium list entry entities. */
@Transient Key<PremiumListRevision> revisionKey;
@Ignore
@Column(nullable = false)
CurrencyUnit currency;
/**
* Mapping from unqualified domain names to their prices.
*
* <p>This field requires special treatment since we want to lazy load it. We have to remove it
* from the immutability contract so we can modify it after construction and we have to handle the
* database processing on our own so we can detach it after load.
*/
@Ignore @ImmutableObject.Insignificant @Transient ImmutableMap<String, BigDecimal> labelsToPrices;
@Ignore
@Column(nullable = false)
BloomFilter<String> bloomFilter;
/** Virtual parent entity for premium list entry entities associated with a single revision. */
@ReportedOn
@Entity
@InCrossTld
public static class PremiumListRevision extends ImmutableObject implements DatastoreOnlyEntity {
@Parent Key<PremiumList> parent;
@Id long revisionId;
/**
* A Bloom filter that is used to determine efficiently and quickly whether a label might be
* premium.
*
* <p>If the label might be premium, then the premium list entry must be loaded by key and
* checked for existence. Otherwise, we know it's not premium, and no Datastore load is
* required.
*/
private BloomFilter<String> probablePremiumLabels;
/**
* Get the Bloom filter.
*
* <p>Note that this is not a copy, but the mutable object itself, because copying would be
* expensive. You probably should not modify the filter unless you know what you're doing.
*/
public BloomFilter<String> getProbablePremiumLabels() {
return probablePremiumLabels;
}
/**
* The maximum size of the Bloom filter.
*
* <p>Trying to set it any larger will throw an error, as we know it won't fit into a Datastore
* entity. We use 90% of the 1 MB Datastore limit to leave some wriggle room for the other
* fields and miscellaneous entity serialization overhead.
*/
private static final int MAX_BLOOM_FILTER_BYTES = 900000;
/** Returns a new PremiumListRevision for the given key and premium list map. */
@VisibleForTesting
public static PremiumListRevision create(PremiumList parent, Set<String> premiumLabels) {
PremiumListRevision revision = new PremiumListRevision();
revision.parent = Key.create(parent);
revision.revisionId = allocateId();
// All premium list labels are already punycoded, so don't perform any further character
// encoding on them.
revision.probablePremiumLabels =
BloomFilter.create(unencodedCharsFunnel(), premiumLabels.size());
premiumLabels.forEach(revision.probablePremiumLabels::put);
try {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
revision.probablePremiumLabels.writeTo(bos);
checkArgument(
bos.size() <= MAX_BLOOM_FILTER_BYTES,
"Too many premium labels were specified; Bloom filter exceeds max entity size");
} catch (IOException e) {
throw new IllegalStateException("Could not serialize premium labels Bloom filter", e);
}
return revision;
}
}
@VisibleForTesting
public Key<PremiumListRevision> getRevisionKey() {
return revisionKey;
}
/** Returns the {@link CurrencyUnit} used for this list. */
public CurrencyUnit getCurrency() {
return currency;
}
/**
* Returns a {@link Map} of domain labels to prices.
*
* <p>Note that this is lazily loaded and thus must be called inside a transaction. You generally
* should not be using this anyway as it's inefficient to load all of the PremiumEntry rows if you
* don't need them. To check prices, use {@link PremiumListDao#getPremiumPrice} instead.
*/
public synchronized ImmutableMap<String, BigDecimal> getLabelsToPrices() {
if (labelsToPrices == null) {
labelsToPrices =
jpaTm()
.createQueryComposer(PremiumEntry.class)
.where("revisionId", EQ, revisionId)
.stream()
.collect(
toImmutableMap(
PremiumEntry::getDomainLabel,
// Set the correct amount of precision for the premium list's currency.
entry -> convertAmountToMoney(entry.getPrice()).getAmount()));
}
return labelsToPrices;
}
/**
* Converts a raw {@link BigDecimal} amount to a {@link Money} by applying the list's currency.
*/
public Money convertAmountToMoney(BigDecimal amount) {
return Money.of(currency, amount.setScale(currency.getDecimalPlaces(), RoundingMode.HALF_EVEN));
}
/**
* Returns a Bloom filter to determine whether a label might be premium, or is definitely not.
*
* <p>If the domain label might be premium, then the next step is to check for the existence of a
* corresponding row in the PremiumListEntry table. Otherwise, we know for sure it's not premium,
* and no DB load is required.
*/
public BloomFilter<String> getBloomFilter() {
return bloomFilter;
}
/**
* A premium list entry entity, persisted to Datastore. Each instance represents the price of a
* single label on a given TLD.
*/
@ReportedOn
@Entity
@InCrossTld
public static class PremiumListEntry extends DomainLabelEntry<Money, PremiumListEntry>
implements Buildable, DatastoreOnlyEntity {
@Parent
Key<PremiumListRevision> parent;
Money price;
@Override
public Money getValue() {
return price;
}
@Override
public String toString() {
// Don't include the comment so that we can use this when exporting the premium list
return String.format("%s,%s", label, price);
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
}
/** A builder for constructing {@link PremiumListEntry} objects, since they are immutable. */
public static class Builder extends DomainLabelEntry.Builder<PremiumListEntry, Builder> {
public Builder() {}
private Builder(PremiumListEntry instance) {
super(instance);
}
public Builder setParent(Key<PremiumListRevision> parentKey) {
getInstance().parent = parentKey;
return this;
}
public Builder setPrice(Money price) {
getInstance().price = price;
return this;
}
}
}
@Override
@Nullable
PremiumListEntry createFromLine(String originalLine) {
List<String> lineAndComment = splitOnComment(originalLine);
if (lineAndComment.isEmpty()) {
return null;
}
String line = lineAndComment.get(0);
String comment = lineAndComment.get(1);
List<String> parts = Splitter.on(',').trimResults().splitToList(line);
checkArgument(parts.size() == 2, "Could not parse line in premium list: %s", originalLine);
return new PremiumListEntry.Builder()
.setLabel(parts.get(0))
.setPrice(Money.parse(parts.get(1)))
.setComment(comment)
.build();
}
@Override
public boolean refersToList(Registry registry, String name) {
return Objects.equals(registry.getPremiumListName().orElse(null), name);
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
}
/** A builder for constructing {@link PremiumList} objects, since they are immutable. */
public static class Builder extends BaseDomainLabelList.Builder<PremiumList, Builder> {
public Builder() {}
private Builder(PremiumList instance) {
super(instance);
}
public Builder setRevision(Key<PremiumListRevision> revision) {
getInstance().revisionKey = revision;
return this;
}
public Builder setCurrency(CurrencyUnit currency) {
getInstance().currency = currency;
return this;
}
public Builder setLabelsToPrices(Map<String, BigDecimal> labelsToPrices) {
getInstance().labelsToPrices = ImmutableMap.copyOf(labelsToPrices);
return this;
}
@Override
public PremiumList build() {
if (getInstance().labelsToPrices != null) {
// ASCII is used for the charset because all premium list domain labels are stored
// punycoded.
getInstance().bloomFilter =
BloomFilter.create(stringFunnel(US_ASCII), getInstance().labelsToPrices.size());
getInstance()
.labelsToPrices
.keySet()
.forEach(label -> getInstance().bloomFilter.put(label));
}
return super.build();
}
}
@PrePersist
void prePersist() {
lastUpdateTime = creationTime;
}
@PostLoad
void postLoad() {
creationTime = lastUpdateTime;
}
@PreRemove
void preRemove() {
jpaTm()
.query("DELETE FROM PremiumEntry WHERE revision_id = :revisionId")
.setParameter("revisionId", revisionId)
.executeUpdate();
}
/**
* Hibernate hook called on the insert of a new PremiumList. Stores the associated {@link
* PremiumEntry}'s.
*
* <p>We need to persist the list entries, but only on the initial insert (not on update) since
* the entries themselves never get changed, so we only annotate it with {@link PostPersist}, not
* {@link PostUpdate}.
*/
@PostPersist
void postPersist() {
// If the price map is loaded, persist it too.
if (labelsToPrices != null) {
labelsToPrices.entrySet().stream()
.forEach(
entry ->
jpaTm()
.insert(PremiumEntry.create(revisionId, entry.getValue(), entry.getKey())));
}
}
}

View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.replay;
package google.registry.model.replay;
import java.util.Optional;

View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.replay;
package google.registry.model.replay;
import java.util.Optional;

View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.replay;
package google.registry.model.replay;
import java.util.Optional;

View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.replay;
package google.registry.model.replay;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;

View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.replay;
package google.registry.model.replay;
import java.util.Optional;

View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.replay;
package google.registry.model.replay;
import google.registry.persistence.VKey;
import java.lang.reflect.InvocationTargetException;

View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.replay;
package google.registry.model.replay;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;

View File

@@ -12,7 +12,9 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.replay;
package google.registry.model.replay;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import java.util.Optional;
@@ -29,4 +31,19 @@ public interface SqlEntity {
/** A method that will ber called before the object is saved to SQL in asynchronous replay. */
default void beforeSqlSaveOnReplay() {}
/* Returns this entity's primary key field(s) in a string. */
default String getPrimaryKeyString() {
return jpaTm()
.transact(
() ->
String.format(
"%s_%s",
this.getClass().getSimpleName(),
jpaTm()
.getEntityManager()
.getEntityManagerFactory()
.getPersistenceUnitUtil()
.getIdentifier(this)));
}
}

View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.replay;
package google.registry.model.replay;
import java.util.Optional;

View File

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.replay;
package google.registry.model.replay;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;

View File

@@ -22,7 +22,7 @@ import com.googlecode.objectify.annotation.Embed;
import com.googlecode.objectify.annotation.Ignore;
import google.registry.model.Buildable;
import google.registry.model.ImmutableObject;
import google.registry.schema.replay.DatastoreAndSqlEntity;
import google.registry.model.replay.DatastoreAndSqlEntity;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.EnumType;

View File

@@ -46,9 +46,9 @@ import google.registry.model.host.HostBase;
import google.registry.model.host.HostHistory;
import google.registry.model.host.HostHistory.HostHistoryId;
import google.registry.model.host.HostResource;
import google.registry.model.replay.DatastoreEntity;
import google.registry.model.replay.SqlEntity;
import google.registry.persistence.VKey;
import google.registry.schema.replay.DatastoreEntity;
import google.registry.schema.replay.SqlEntity;
import java.util.Optional;
import java.util.Set;
import javax.annotation.Nullable;

View File

@@ -21,7 +21,7 @@ import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.google.common.collect.ImmutableSet;
import google.registry.model.Buildable;
import google.registry.model.ImmutableObject;
import google.registry.schema.replay.SqlOnlyEntity;
import google.registry.model.replay.SqlOnlyEntity;
import google.registry.util.DomainNameUtils;
import java.util.Set;
import javax.persistence.Column;

Some files were not shown because too many files have changed in this diff Show More