1
0
mirror of https://github.com/google/nomulus synced 2026-06-09 16:33:02 +00:00

Compare commits

...

52 Commits

Author SHA1 Message Date
sarahcaseybot 642405375b Stop writing ClaimsList to Datastore (#1169)
* Stop writing ClaimsList to Datastore

* Fix some failing tests

* Rename ClaimsListShard to ClaimsList
2021-05-20 15:44:40 -04:00
Lai Jiang 02eb7cfcc3 Switch from using raw HistoryEntries to typed subclasses thereof (#1150)
HistoryEntry is used to record all histories (contact, domain, host) in
Datastore. In SQL it is now split into three subclasses (and thus
tables): ContactHistory, DomainHistory and HostHistory. Its builder is
genericized as a result which led to a lot of compiler warnings for the
use of a raw HistoryEntry in the existing code base.

This PR cleans things up by replacing all the explicit use of
raw HistoryEntry with the corresponding subclass and also adds some
guardrails to prevent the use of raw HistoryEntry accidentally.

Note that because DomainHistory includes nsHosts and gracePeriodHistory,
both of which are assigned a roid from ofy when built, the assigned roids for
resources after history entries are built are incremented compared to
when only HistoryEntrys are built (before this PR) in
RdapDomainSearchActionTest.

Also added a convenient tm().updateAll() varargs method.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1150)
<!-- Reviewable:end -->
2021-05-20 11:58:41 -04:00
Michael Muller f7dca7fa96 Make PremiumList.labelsToPrices "insignificant" (#1167)
* Make PremiumList.labelsToPrices "insignificant"

Add the ImmutableObject.Insignificant annotation to labelsToPrices and also
mark it as Transient.  In order to do lazy-loads on this field, we need to do
so explicitly: doing otherwise breaks the immutability contract and prevents
detaching the object upon load.

Note that this is an expedient solution to this problem, but not the optimal
one.  Ideally, the disassociation between PremiumList and its PremiumEntry's
would be more explicit.  However, breaking labelsToPrices out would at minimum
require reworking the Create/UpdatePremiumList commands, which currently rely
on passing around a self-contained PremiumList object, both from the parser
interfaces and to the database.

If this approach is acceptable, we can apply it to ReservedList and ClaimsList
as well (though it may be easier to break the association in those cases).

* Fix premium list "delete" to support a test

* Fix a few more tests

* Changes for review (updated javadocs)

* Minor fixes

* Updated getLablesToPrices() comment

* Format fixes, fixed PremiumEntry interfaces

PremiumEntry can now be SQL only.
2021-05-20 11:21:37 -04:00
gbrodman a7e8ae5a2c Add loadOnlyOf method to tm() (#1162)
* Add loadOnlyOf method to tm()

In addition there's a bit of a refator of SqlReplayCheckpoint to make it
more in line with the other singletons. This method is useful for the
singleton classes where we expect at most one entity to exist, e.g.
ServerSecret.
2021-05-20 10:59:01 -04:00
Michael Muller dc7f21ca68 Convert most poll message queries to QueryComposer (#1151)
* Convert most poll message queries to QueryComposer

* Add unit test and a better exception for datastore

* Remove datastorePollMessageQuery from PollFlowUtils

* Reformatted.

* Improved test equality checks

* Changes for review

* Converted concatenated string to String.format()
2021-05-19 15:58:20 -04:00
Weimin Yu e96873f2d0 Support text-based JPQL query for BEAM (#1168)
* Support text-based JPQL query for BEAM
2021-05-19 14:45:04 -04:00
Lai Jiang b5f05405a0 Fix linter warnings (#1165) 2021-05-18 18:30:01 -04:00
gbrodman f702f2670b Use a flatMap in StaticPremiumPricingEngine (#1166)
* Use a flatMap in StaticPremiumPricingEngine
2021-05-18 12:20:04 -04:00
sarahcaseybot 21aeedae11 Fix NullPointerException in StaticPremiumPricingEngine (#1164)
* Fix NullPointerException in StaticPremiumPricingEngine

* Make getPremiumList return optional

* add isPresent checks
2021-05-18 10:55:27 -04:00
sarahcaseybot c1f0c29134 Stop writing ReservedList to Datastore (#1163) 2021-05-17 17:46:21 -04:00
gbrodman 16641e05a1 Update GCL dependency to avoid security alert (#1139)
* Update GCL dependency to avoid security alert

This required a few changes in addition to the dependency update.

- a few transitive / required dependency updates as well
- updating soyutils_usegoog.js and adding checks.js because they're
necessary as part of the Soy compilation process
- Using a trustedResourceUri in the buildSrc Soy compilation instead of
a string
- changing the arguments to the Soy-to-Java compiler to comply with the
new version
- Moving all Soy UI files to be in the registrar directory. This was
not the case before due to previous thinking that we'd have separate
admin and registrar consoles -- this is no longer the case so it's no
longer necessary. This necessitated various refactorings and reference
changes.
  - The new soy-to-javascript compiler requires this, as it removes the
  "deps" param that we were previously using to say "use the general UI
  utils as dependencies for the registrar-console files".
- Creating a SQL environment and loading test data in the test server
main method -- previously, the local test server did not work.
- Fix some JS code that was referencing now-deleted library functions
- Removal of the Karma tests, as the karma-closure library hasn't been
updated since 2018 and it no longer works. We never noticed any errors
from the Karma tests, we never change the JS, and we have the
Java+Selenium screenshot differ tests to test the UI anyway.
2021-05-17 13:21:26 -04:00
Ben McIlwain bf1c34cc3b Add sanity checks to history entry construction (#1156)
* Add sanity checks to history entry construction

* Add more missing setClientId() calls and delete scrap tool

* Merge branch 'master' into synthetic-requestedby

* Set more client IDs

* Merge branch 'master' into synthetic-requestedby
2021-05-14 19:54:35 -04:00
sarahcaseybot 93dc812ea2 Stop writing PremiumList to Datastore (#1160)
* Stop writing PremiumList to Datastore

* Fix formatting

* Format fix

* Rename the DAO

* Fix merge conflicts and add comment
2021-05-14 16:13:05 -04:00
Weimin Yu e09138645f Fix RegistryJpaIO.Read problem with large data (#1161)
* Fix RegistryJpaIO.Read problem with large data

The read connector needs to detach loaded entities. This 
is now the default behavior in QueryComposer

Also removed the 'transaction mode' property from the Read connector.
There are no obvious use cases for non-transaction query, and
implementation is not straightforward with the current code base.

Also changed the return type of QueryComposer.list() to ImmutableList.
2021-05-14 15:19:12 -04:00
gbrodman 238deb25ec Clean up some SqlEntity classes (#1158)
* Clean up some SqlEntity classes

This started as having a better check for when to run the
ReplayCommitLogsToSqlAction but that'll require a bit more thought, and
this is a fairly simple PR that can be split out.
2021-05-14 11:25:11 -04:00
Ben McIlwain 6ce2926c6d Remove final vestiges of domain applications (#1153)
* Remove final vestiges of domain applications
2021-05-14 10:39:25 -04:00
Rachel Guan 27f431b9cf Change premium list command to be based off of mutating command (#1123)
* Change premium list command to be based off of mutating command

* Modify test cases and add comments for better readability

* Fix typo
2021-05-14 08:40:03 -04:00
gbrodman 2bb0e7305d Convert even more classes to auditedOfy() (#1157)
* Convert even more classes to auditedOfy()

This covers almost all of the classes in the second round of the sheet.
There are still some classes that need conversion but this is the vast
majority of them.

https://docs.google.com/spreadsheets/d/1aFEFuyH6vVW6b-h71O9f5CuUc6Y7YjZ2kdRL3lwXcVk/edit?resourcekey=0-guwZVKfSH-pntER1tUit6w#gid=1355213322
for notes
2021-05-13 14:12:13 -04:00
Lai Jiang 10757863ce Reorder steps (#1159) 2021-05-13 13:15:46 -04:00
gbrodman 02079010c6 Add mapreduce action to create synthetic history entries (#1125)
* Add mapreduce action to create synthetic history entries

RDE and zone file generation require being able to tell what objects
looked like in the past (though not beyond 30 days, or whatever the
Datastore retention period is set to). In Datastore, to answer this we
look at commit logs, and in SQL we will look at the History objects
stored for each EPP resource. This action can be run once while in
Datastore-primary-SQL-secondary to make sure that every EPP resource has
at least one history entry for which the resource-at-this-time field is
filled out in the SQL world.
2021-05-13 11:48:19 -04:00
Lai Jiang 4246e7e4e0 Add indexes on contacts in the Domain table (#1145)
These indexes are used to find if a contact is linked to a domain in
during a contact delete.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1145)
<!-- Reviewable:end -->
2021-05-13 10:47:35 -04:00
Lai Jiang 9f21989f13 Remove the logic to add full certificate in the headers (#1143)
<!-- Reviewable:start -->
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1143)
<!-- Reviewable:end -->
2021-05-12 20:52:16 -04:00
gbrodman 2073f5b59f Populate the host in HostHistory objects in Host flows (#1129)
* Populate the host in HostHistory objects in Host flows
2021-05-12 19:11:30 -04:00
Weimin Yu 66ac000ef4 Fix the JPA Read connector for large data (#1155)
* Fix the JPA Read connector for large data

Allow result set streaming by setting the fetchSize on JDBC statements.
Many JDBC drivers by default buffers the entire result set, causing
delays in first result and/or out of memory errors.

Also fixed a entity instantiation problem exposed in production runs.

Lastly, removed incorrect comments.
2021-05-12 19:07:38 -04:00
Rachel Guan 85bac9834f Add stageEntityChange() method to display difference when creating a reserved list (#1149)
* Add stageEntityChange() method to display difference before execution when creating a reserved list
2021-05-12 17:32:57 -04:00
Weimin Yu 484e30cd80 Restore a fix for flaky test (#1154)
* Restore a fix for flaky test

Restore a speculative fix for the flakiness in
DeleteExpiredDomainsActionTest. Although we identified a bug and fixed
it in a previous commit, it may not be the only bug. The removed fix may
still be necessary.
2021-05-12 16:03:42 -04:00
gbrodman af67356aa0 Convert more ofy() to auditedOfy() calls (#1152)
A couple of these use the QueryComposer interface to avoid branching.

In addition, we enforce the Datastore restriction that there can be at
most 1 field with an inequality query, see https://cloud.google.com/appengine/docs/standard/go111/datastore/query-restrictions#inequality_filters_are_limited_to_at_most_one_property
2021-05-12 15:06:19 -04:00
Rachel Guan 8c9a2b5f4a Fix typo in comment of premium list example file (#1148)
* Fix typo in comment of premium list example file
2021-05-11 18:25:05 -04:00
gbrodman 0d67ea3a6e Combine the two Lock classes into one class (#1126)
* Combine the two Lock classes into one class

This allows us to remove the DAO and to just treat locks the same as we
would treat any other object -- generically grabbing them from the
transaction manager.

We do not need to be concerned about the changeover between Datastore
and SQL because we assume that any such changeover will require
sufficient downtime that any currently-valid acquired locks will expire
during the downtime. Otherwise, we could get into a situation where an
action has acquired a particular lock in Datastore but not SQL.
2021-05-11 16:37:40 -04:00
Rachel Guan 5b56e8b71b Create key based on the change type (#1147)
* Create key based on the change type
2021-05-11 15:24:35 -04:00
Weimin Yu 6eba8aa1c4 Fix timestamp inversion bug (#1144)
* Fix timestamp inversion bug

Set the number of commitLog buckets to 1 in CommitLog replay tests to
expose all timestamp inversion problems due to replay. Fixed
PollAckFlowTest which is related to this problem.

Also fixed a few tests that failed to advance the fake clock when they
should, using the following approaches:

- If DatabaseHelper used but clock is not injected, do it. This
  allows us to remove some unnecessary manual clock advances.
- Manually advance the clock where convenient.
- Enable clock autoIncrement mode when calling production classes that
  performs multiple transactions.

We should consider making 1-bucket the default setting for tests. This
is left to another PR.
2021-05-11 14:51:10 -04:00
Lai Jiang 8d18450e56 Update README.md (#1146)
<!-- Reviewable:start -->
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1146)
<!-- Reviewable:end -->
2021-05-11 13:40:07 -04:00
sarahcaseybot 65be65fb24 Always use Cloud SQL as primary for ClaimsList (#1127)
* Always use Cloud SQL as primary for ClaimsList

* Add a test back
2021-05-10 16:47:34 -04:00
Weimin Yu 984f1118e3 Make secretmanager primary storage for keyring (#1124)
* Make secretmanager primary storage for keyring

Also removed the migrate_kms_keyring command.
2021-05-10 11:11:26 -04:00
gbrodman 0bcb142bc9 Add an auditedOfy marker method for allow-listed ofy() calls (#1138)
* Add an auditedOfy marker method for allow-listed ofy() calls

This will allow us to make sure that every usage of ofy() has been
hand-examined and specifically allowed.
2021-05-10 10:55:28 -04:00
Lai Jiang d93a4e562a Delete hosts synchronously when using SQL (#1141)
Also put some common logic in helper funcions in ContactDeleteFlowTest
to reduce clutter.
2021-05-10 10:22:01 -04:00
Lai Jiang 420a579e01 Fix flaky Spec11PipelineTest (#1133) 2021-05-07 15:01:11 -04:00
Lai Jiang 1ec96b66e2 Perform synchronous contact delete in SQL (#1137)
In SQL the contact of a domain is an indexed field and therefore we can
find linked domains synchronously, without the need for MapReduce.

The delete logic is mostly lifted from DeleteContactsAndHostsAction, but
because everything happens in a transaction we do not need to recheck a
lot of the preconditions that were necessary to ensure that the async
delete request still meets the conditions that when the request was
enqueued.
2021-05-07 10:48:51 -04:00
gbrodman 51a7ba249e Populate the contact in ContactHistory objects created in Contact flows (#1111)
* Populate the contact in ContactHistory objects created in Contact flows

Minimal interesting changes here
- a bit of reconstruction in ContactHistory to get the repo ID from the
key
- making the History revision ID Long instead of long so that it can be
null in non-built intermediate entities
- adding a copyFrom(HistoryEntry.Builder) method in HistoryEntry.Builder
so that we don't need to allocate quite as many unnecessary IDs, i.e.
removing the .build() lines in provideContactHistory and
provideDomainHistory
2021-05-06 14:38:55 -04:00
Lai Jiang 5120397607 Upload the GCB delete job yaml file to GCS (#1135)
<!-- Reviewable:start -->
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1135)
<!-- Reviewable:end -->
2021-05-05 21:43:51 -04:00
sarahcaseybot 038825f254 Always use Cloud SQL as primary for Reserved and Premium Lists (#1113)
* Always use Cloud SQL as primary for Reserved and Premium Lists

* small typos

* Add a state check

* Add test for bloom filter

* fix import
2021-05-05 17:24:06 -04:00
Weimin Yu b38574a9fc Add a BEAM read connector for JPA entities (#1132)
* Add a BEAM read connector for JPA entities

Added a Read connector to load JPA entities from Cloud SQL.

Also attempted a fix to the null threadfactory problem.
2021-05-05 15:45:03 -04:00
Lai Jiang 3f6ec8f1b0 Re-enable tests in RC build (#1130)
There has been a case where the CI was broken on Friday and no one
noticied or fixed it and a RC build was built with broken tests.
The tests were disabled due to unknown test failures that have since
been fixed.

Also update the machine type used by GCB to be more powerful. This is
necessary for the tests to past because N1_HIGHCPU_8 is RAM constraint
and the tests crashes. I updated all jobs to use the new type which
hopefully will make the build faster as well.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1130)
<!-- Reviewable:end -->
2021-05-05 13:53:21 -04:00
gbrodman 65fb0c6cff Update Karma version to avoid security hole in dependency (#1134)
This also forces the karma test to use the Gradle-installed version of
node instead of the global version. The global version installed on the
Kokoro machines is too old to function with some of the newer libraries.
2021-05-05 13:50:45 -04:00
Lai Jiang e63085fb6a Add a GCB job to delete stopped GAE versions (#1128) 2021-05-05 11:27:46 -04:00
gbrodman b5363e9457 Populate the domain in DomainHistory objects created in Domain flows (#1106)
Unfortunately, much of the time there's a bit of a circular dependency
in the object creation, e.g. the Domain object stores references to the
billing events which store references to the history object which
contains the Domain object. As a result, we allocate the history
object's ID before creating it, so that it can be referenced in the
other objects that store that reference, e.g. billing events.

In addition, we add a utility copyFrom method in HistoryEntry.Builder to
avoid unnecessary ID allocations.
2021-05-04 19:09:27 -04:00
Ben McIlwain cb16df235a Remove unnecessary MockitoExtension from Spec11PipelineTest (#1115)
* Remove unnecessary MockitoExtension from Spec11PipelineTest

This is kind of a shot in the dark here, but this is one of the obvious
differences between this test class (which frequently experiences flakes) and
the other pipeline test classes which do not.

It's also possible we were getting the wrong runner if the test framework was
incorrectly detecting an App Engine runtime environment, so I added an assert
that will make it very clear if this is the cause of any failures.
2021-05-04 18:38:24 -04:00
Lai Jiang d285edef3d Fix a few linter warnings (#1122) 2021-05-04 13:35:31 -04:00
Weimin Yu 509c0dcd17 Handle bad production data when migrating to SQL (#1120)
* Handle bad production data when migrating to SQL

Ignore or fix bad entites when populating SQL with production data in
Datastore. These are mostly inconsistent foreign keys.

See b/185954992 for details.
2021-05-03 16:09:43 -04:00
sarahcaseybot ce18bf0690 Use FakeClock to prevent Expired Certificate Violations (#1121)
* Use FakeClock to prevent Expired Certificate Violations

* Format fixes

* Make CertificateChecker static
2021-05-03 15:10:26 -04:00
Lai Jiang 8d63cbfca0 Remove enforcement date from the SslServerInitializer (#1117)
The enforcement data has passed and ICANN has confirmed that their web
WHOIS prober conforms to our requirements.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1117)
<!-- Reviewable:end -->
2021-04-30 15:44:03 -04:00
Lai Jiang eb6a1fe1ed Remove Pipeline as a field in pipeline classes (#1119)
In tests we use a TestPipelineExtension which does some static
initialization that should not be repeated the same JVM. In our
XXXPipeline classes we save the pipeline as a field and usually write lambdas
that are pass to the pipeline. Because lambdas are effectively anonymous inner
classes they are bound to their enclosing instances. When they get serialized
during pipeline execution, their enclosing classes also do. This might result
in undefined behavior when multiple lambdas in the same XXXPipeline are used
on the same JVM (such as in tests) where the static initialization may be done
multiple times if different class loaders are used. This is very
unlikely to happen but as a best practice we still remove them as
fields.
2021-04-30 14:32:33 -04:00
468 changed files with 6590 additions and 10433 deletions
+1
View File
@@ -1,4 +1,5 @@
python/
node_modules/
**/build/
**/out/
.*/
+3 -1
View File
@@ -318,7 +318,7 @@ subprojects {
// expose to users.
if (project.name != 'docs') {
javadocSource << project.sourceSets.main.allJava
javadocClasspath << project.sourceSets.main.compileClasspath
javadocClasspath << project.sourceSets.main.runtimeClasspath
javadocClasspath << "${buildDir}/generated/sources/annotationProcessor/java/main"
javadocDependentTasks << project.tasks.compileJava
}
@@ -457,6 +457,8 @@ task javaIncrementalFormatApply {
task javadoc(type: Javadoc) {
source javadocSource
classpath = files(javadocClasspath)
// Exclude the misbehaving generated-by-Soy Java files
exclude "**/*SoyInfo.java"
destinationDir = file("${buildDir}/docs/javadoc")
options.encoding = "UTF-8"
// In a lot of places we don't write @return so suppress warnings about that.
+1
View File
@@ -72,6 +72,7 @@ dependencies {
compile deps['com.google.auth:google-auth-library-credentials']
compile deps['com.google.auth:google-auth-library-oauth2-http']
compile deps['com.google.auto.value:auto-value-annotations']
compile deps['com.google.common.html.types:types']
compile deps['com.google.cloud:google-cloud-core']
compile deps['com.google.cloud:google-cloud-storage']
compile deps['com.google.guava:guava']
@@ -20,12 +20,12 @@ com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
@@ -34,10 +34,11 @@ com.google.http-client:google-http-client:1.39.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.ibm.icu:icu4j:57.1
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
@@ -47,7 +48,6 @@ io.opencensus:opencensus-contrib-http-util:0.28.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.validation:validation-api:1.0.0.GA
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.13
@@ -55,9 +55,9 @@ org.apache.httpcomponents:httpcore:4.4.14
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:6.0
org.ow2.asm:asm-util:6.0
org.ow2.asm:asm:6.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.threeten:threetenbp:1.5.0
@@ -20,12 +20,12 @@ com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
@@ -34,10 +34,11 @@ com.google.http-client:google-http-client:1.39.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.ibm.icu:icu4j:57.1
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
@@ -47,7 +48,6 @@ io.opencensus:opencensus-contrib-http-util:0.28.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.validation:validation-api:1.0.0.GA
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.13
@@ -55,9 +55,9 @@ org.apache.httpcomponents:httpcore:4.4.14
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:6.0
org.ow2.asm:asm-util:6.0
org.ow2.asm:asm:6.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.threeten:threetenbp:1.5.0
@@ -20,12 +20,12 @@ com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
@@ -34,10 +34,11 @@ com.google.http-client:google-http-client:1.39.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.ibm.icu:icu4j:57.1
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
@@ -47,7 +48,6 @@ io.opencensus:opencensus-contrib-http-util:0.28.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.validation:validation-api:1.0.0.GA
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.13
@@ -55,9 +55,9 @@ org.apache.httpcomponents:httpcore:4.4.14
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:6.0
org.ow2.asm:asm-util:6.0
org.ow2.asm:asm:6.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.threeten:threetenbp:1.5.0
@@ -20,12 +20,12 @@ com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
@@ -34,10 +34,11 @@ com.google.http-client:google-http-client:1.39.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.ibm.icu:icu4j:57.1
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
@@ -47,7 +48,6 @@ io.opencensus:opencensus-contrib-http-util:0.28.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.validation:validation-api:1.0.0.GA
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.13
@@ -55,9 +55,9 @@ org.apache.httpcomponents:httpcore:4.4.14
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:6.0
org.ow2.asm:asm-util:6.0
org.ow2.asm:asm:6.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.threeten:threetenbp:1.5.0
@@ -20,12 +20,12 @@ com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
@@ -34,10 +34,11 @@ com.google.http-client:google-http-client:1.39.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.ibm.icu:icu4j:57.1
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
@@ -47,7 +48,6 @@ io.opencensus:opencensus-contrib-http-util:0.28.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.validation:validation-api:1.0.0.GA
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.13
@@ -55,9 +55,9 @@ org.apache.httpcomponents:httpcore:4.4.14
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:6.0
org.ow2.asm:asm-util:6.0
org.ow2.asm:asm:6.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.threeten:threetenbp:1.5.0
@@ -20,12 +20,12 @@ com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
@@ -34,10 +34,11 @@ com.google.http-client:google-http-client:1.39.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.ibm.icu:icu4j:57.1
@@ -49,7 +50,6 @@ io.opencensus:opencensus-contrib-http-util:0.28.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.validation:validation-api:1.0.0.GA
junit:junit:4.13.1
net.bytebuddy:byte-buddy-agent:1.10.19
net.bytebuddy:byte-buddy:1.10.19
@@ -70,9 +70,9 @@ org.junit:junit-bom:5.6.2
org.mockito:mockito-core:3.7.7
org.objenesis:objenesis:3.1
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:6.0
org.ow2.asm:asm-util:6.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:9.0
org.threeten:threetenbp:1.5.0
@@ -20,12 +20,12 @@ com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
@@ -34,10 +34,11 @@ com.google.http-client:google-http-client:1.39.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.ibm.icu:icu4j:57.1
@@ -49,7 +50,6 @@ io.opencensus:opencensus-contrib-http-util:0.28.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.validation:validation-api:1.0.0.GA
junit:junit:4.13.1
net.bytebuddy:byte-buddy-agent:1.10.19
net.bytebuddy:byte-buddy:1.10.19
@@ -70,9 +70,9 @@ org.junit:junit-bom:5.6.2
org.mockito:mockito-core:3.7.7
org.objenesis:objenesis:3.1
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:6.0
org.ow2.asm:asm-util:6.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:9.0
org.threeten:threetenbp:1.5.0
@@ -20,12 +20,12 @@ com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
@@ -34,10 +34,11 @@ com.google.http-client:google-http-client:1.39.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.ibm.icu:icu4j:57.1
@@ -49,7 +50,6 @@ io.opencensus:opencensus-contrib-http-util:0.28.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.validation:validation-api:1.0.0.GA
junit:junit:4.13.1
net.bytebuddy:byte-buddy-agent:1.10.19
net.bytebuddy:byte-buddy:1.10.19
@@ -70,9 +70,9 @@ org.junit:junit-bom:5.6.2
org.mockito:mockito-core:3.7.7
org.objenesis:objenesis:3.1
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:6.0
org.ow2.asm:asm-util:6.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:9.0
org.threeten:threetenbp:1.5.0
@@ -20,12 +20,12 @@ com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
@@ -34,10 +34,11 @@ com.google.http-client:google-http-client:1.39.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.ibm.icu:icu4j:57.1
@@ -49,7 +50,6 @@ io.opencensus:opencensus-contrib-http-util:0.28.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.validation:validation-api:1.0.0.GA
junit:junit:4.13.1
net.bytebuddy:byte-buddy-agent:1.10.19
net.bytebuddy:byte-buddy:1.10.19
@@ -70,9 +70,9 @@ org.junit:junit-bom:5.6.2
org.mockito:mockito-core:3.7.7
org.objenesis:objenesis:3.1
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:6.0
org.ow2.asm:asm-util:6.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:9.0
org.threeten:threetenbp:1.5.0
@@ -23,6 +23,7 @@ import static google.registry.gradle.plugin.GcsPluginUtils.toByteArraySupplier;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSetMultimap;
import com.google.common.html.types.TrustedResourceUrls;
import com.google.template.soy.SoyFileSet;
import com.google.template.soy.tofu.SoyTofu;
import google.registry.gradle.plugin.ProjectData.TaskData;
@@ -118,7 +119,7 @@ final class CoverPageGenerator {
builder.put("projectState", state.toString());
builder.put("title", title);
builder.put("cssFiles", ImmutableSet.of("css/style.css"));
builder.put("cssFiles", ImmutableSet.of(TrustedResourceUrls.fromConstant("css/style.css")));
builder.put("invocation", getInvocation());
builder.put("tasksByState", getTasksByStateSoyData());
return builder.build();
@@ -16,7 +16,7 @@
{template .coverPage}
{@param title: string}
{@param cssFiles: list<string>}
{@param cssFiles: list<trusted_resource_uri>}
{@param projectState: string}
{@param invocation: string}
{@param tasksByState: map<string, list<[uniqueName: string, description: string, log: string, reports: map<string, string>]>>}
@@ -22,6 +22,7 @@ import google.registry.util.Clock;
import java.util.concurrent.atomic.AtomicLong;
import javax.annotation.concurrent.ThreadSafe;
import org.joda.time.DateTime;
import org.joda.time.Duration;
import org.joda.time.ReadableDuration;
import org.joda.time.ReadableInstant;
@@ -81,4 +82,14 @@ public final class FakeClock implements Clock {
public void setTo(ReadableInstant time) {
currentTimeMillis.set(time.getMillis());
}
/** Invokes {@link #setAutoIncrementStep} with one millisecond-step. */
public FakeClock setAutoIncrementByOneMilli() {
return setAutoIncrementStep(Duration.millis(1));
}
/** Disables the auto-increment mode. */
public FakeClock disableAutoIncrement() {
return setAutoIncrementStep(Duration.ZERO);
}
}
+2 -1
View File
@@ -81,7 +81,7 @@ PRESUBMITS = {
".git", "/build/", "/generated/", "/generated_tests/",
"node_modules/", "JUnitBackports.java", "registrar_bin.",
"registrar_dbg.", "google-java-format-diff.py",
"nomulus.golden.sql", "soyutils_usegoog.js"
"nomulus.golden.sql", "soyutils_usegoog.js", "javascript/checks.js"
}, REQUIRED):
"File did not include the license header.",
@@ -213,6 +213,7 @@ PRESUBMITS = {
"RdapDomainSearchAction.java",
"RdapNameserverSearchAction.java",
"RdapSearchActionBase.java",
"RegistryQuery",
},
):
"The first String parameter to EntityManager.create(Native)Query "
+17 -55
View File
@@ -330,7 +330,6 @@ dependencies {
testCompile deps['org.junit.platform:junit-platform-suite-api']
testCompile deps['org.mockito:mockito-core']
testCompile deps['org.mockito:mockito-junit-jupiter']
testCompile 'org.checkerframework:checker-qual:3.9.1'
runtime deps['org.postgresql:postgresql']
// Indirect dependency found by undeclared-dependency check. Such
@@ -433,12 +432,9 @@ task soyToJava {
// Relative paths of soy directories.
def spec11SoyDir = "google/registry/reporting/spec11/soy"
def toolsSoyDir = "google/registry/tools/soy"
def uiSoyDir = "google/registry/ui/soy"
def registrarSoyDir = "google/registry/ui/soy/registrar"
def soyRelativeDirs = [
spec11SoyDir, toolsSoyDir, uiSoyDir, registrarSoyDir,
]
def soyRelativeDirs = [spec11SoyDir, toolsSoyDir, registrarSoyDir]
soyRelativeDirs.each {
inputs.dir "${resourcesSourceDir}/${it}"
outputs.dir "${generatedDir}/${it}"
@@ -452,7 +448,8 @@ task soyToJava {
"--outputDirectory", "${outputDirectory}",
"--javaClassNameSource", "filename",
"--allowExternalCalls", "true",
"--srcs", "${soyFiles.join(',')}"
"--srcs", "${soyFiles.join(',')}",
"--compileTimeGlobalsFile", "${resourcesSourceDir}/google/registry/ui/globals.txt"
}
}
@@ -469,14 +466,6 @@ task soyToJava {
dir: "${resourcesSourceDir}/${registrarSoyDir}",
include: ['**/*.soy']))
soyToJava('google.registry.ui.soy',
"${generatedDir}/${uiSoyDir}",
files {
file("${resourcesSourceDir}/${uiSoyDir}").listFiles()
}.filter {
it.name.endsWith(".soy")
})
soyToJava('google.registry.reporting.spec11.soy',
"${generatedDir}/${spec11SoyDir}",
fileTree(
@@ -485,42 +474,24 @@ task soyToJava {
}
}
task soyToJS {
def rootSoyDirectory = "${resourcesSourceDir}/google/registry/ui/soy"
def outputSoyDirectory = "${generatedDir}/google/registry/ui/soy"
task soyToJS(type: JavaExec) {
def rootSoyDirectory = "${resourcesSourceDir}/google/registry/ui/soy/registrar"
def outputSoyDirectory = "${generatedDir}/google/registry/ui/soy/registrar"
inputs.dir rootSoyDirectory
outputs.dir outputSoyDirectory
ext.soyToJS = { outputDirectory, soyFiles , deps->
javaexec {
main = "com.google.template.soy.SoyToJsSrcCompiler"
classpath configurations.soy
def inputSoyFiles = files {
file("${rootSoyDirectory}").listFiles()
}.filter {
it.name.endsWith(".soy")
}
args "--outputPathFormat", "${outputDirectory}/{INPUT_FILE_NAME}.js",
classpath configurations.soy
main = "com.google.template.soy.SoyToJsSrcCompiler"
args "--outputPathFormat", "${outputSoyDirectory}/{INPUT_FILE_NAME}.js",
"--allowExternalCalls", "false",
"--srcs", "${soyFiles.join(',')}",
"--shouldProvideRequireSoyNamespaces", "true",
"--srcs", "${inputSoyFiles.join(',')}",
"--compileTimeGlobalsFile", "${resourcesSourceDir}/google/registry/ui/globals.txt"
if (deps != "") {
args "--deps", "${deps.join(',')}"
}
}
}
doLast {
def rootSoyFiles =
fileTree(
dir: "${rootSoyDirectory}",
include: ['*.soy'])
soyToJS("${outputSoyDirectory}", rootSoyFiles, "")
soyToJS("${outputSoyDirectory}/registrar",
files {
file("${rootSoyDirectory}/registrar").listFiles()
}.filter {
it.name.endsWith(".soy")
}, rootSoyFiles)
}
}
task stylesheetsToJavascript {
@@ -603,8 +574,8 @@ task compileProdJS(type: JavaExec) {
closureArgs << "--generate_exports"
// manually include all the required js files
closureArgs << "--js=${nodeModulesDir}/google-closure-library/**.js"
closureArgs << "--js=${jsDir}/soyutils_usegoog.js"
closureArgs << "--js=${nodeModulesDir}/google-closure-library/**/*.js"
closureArgs << "--js=${jsDir}/*.js"
closureArgs << "--js=${cssSourceDir}/registrar_bin.css.js"
closureArgs << "--js=${jsSourceDir}/**.js"
closureArgs << "--js=${externsDir}/json.js"
@@ -631,15 +602,6 @@ compileProdJS.dependsOn processResources
compileProdJS.dependsOn processTestResources
compileProdJS.dependsOn soyToJS
task karmaTest(type: Exec) {
dependsOn ':npmInstall'
workingDir rootProject.projectDir
executable 'node_modules/karma/bin/karma'
args('start', "${project.projectDir}/karma.conf.js")
}
test.dependsOn karmaTest
// Make testing artifacts available to be depended up on by other projects.
// TODO: factor out google.registry.testing to be a separate project.
task testJar(type: Jar) {
@@ -1,15 +1,4 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
args4j:args4j:2.0.26
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.7
com.google.errorprone:error_prone_annotations:2.3.1
com.google.guava:guava:25.1-jre
com.google.j2objc:j2objc-annotations:1.1
com.google.javascript:closure-compiler-externs:v20190301
com.google.javascript:closure-compiler:v20190301
com.google.jsinterop:jsinterop-annotations:1.0.0
com.google.protobuf:protobuf-java:3.0.2
org.checkerframework:checker-qual:2.0.0
org.codehaus.mojo:animal-sniffer-annotations:1.14
com.google.javascript:closure-compiler:v20210505
@@ -105,9 +105,10 @@ com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
@@ -135,7 +136,7 @@ com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.2
com.google.protobuf:protobuf-java:3.15.2
com.google.re2j:re2j:1.6
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
@@ -104,9 +104,10 @@ com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
@@ -133,7 +134,7 @@ com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.14.0
com.google.protobuf:protobuf-java:3.15.2
com.google.re2j:re2j:1.6
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
@@ -110,9 +110,10 @@ com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
@@ -140,7 +141,7 @@ com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.2
com.google.protobuf:protobuf-java:3.15.2
com.google.re2j:re2j:1.6
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
@@ -110,9 +110,10 @@ com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
@@ -140,7 +141,7 @@ com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.2
com.google.protobuf:protobuf-java:3.15.2
com.google.re2j:re2j:1.6
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
@@ -105,9 +105,10 @@ com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
@@ -135,7 +136,7 @@ com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.2
com.google.protobuf:protobuf-java:3.15.2
com.google.re2j:re2j:1.6
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
@@ -104,9 +104,10 @@ com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
@@ -133,7 +134,7 @@ com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.14.0
com.google.protobuf:protobuf-java:3.15.2
com.google.re2j:re2j:1.6
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
@@ -109,9 +109,10 @@ com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
@@ -139,7 +140,7 @@ com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.2
com.google.protobuf:protobuf-java:3.15.2
com.google.re2j:re2j:1.6
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
@@ -109,9 +109,10 @@ com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
@@ -139,7 +140,7 @@ com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.2
com.google.protobuf:protobuf-java:3.15.2
com.google.re2j:re2j:1.6
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
@@ -109,9 +109,10 @@ com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
@@ -139,7 +140,7 @@ com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.2
com.google.protobuf:protobuf-java:3.15.2
com.google.re2j:re2j:1.6
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
@@ -110,9 +110,10 @@ com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
@@ -140,7 +141,7 @@ com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.2
com.google.protobuf:protobuf-java:3.15.2
com.google.re2j:re2j:1.6
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
+9 -9
View File
@@ -5,25 +5,25 @@ aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.3.4
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:5.0.1
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.protobuf:protobuf-java:3.13.0
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.ibm.icu:icu4j:57.1
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.validation:validation-api:1.0.0.GA
org.checkerframework:checker-qual:3.5.0
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:6.0
org.ow2.asm:asm-util:6.0
org.ow2.asm:asm:6.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
@@ -106,9 +106,10 @@ com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
@@ -138,7 +139,7 @@ com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.2
com.google.protobuf:protobuf-java:3.15.2
com.google.re2j:re2j:1.6
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.googlecode.charts4j:charts4j:1.3
@@ -105,9 +105,10 @@ com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
@@ -136,7 +137,7 @@ com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.14.0
com.google.protobuf:protobuf-java:3.15.2
com.google.re2j:re2j:1.6
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.googlecode.charts4j:charts4j:1.3
@@ -111,9 +111,10 @@ com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
@@ -143,7 +144,7 @@ com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.2
com.google.protobuf:protobuf-java:3.15.2
com.google.re2j:re2j:1.6
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.googlecode.charts4j:charts4j:1.3
@@ -111,9 +111,10 @@ com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:2.0.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.5.1
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
@@ -143,7 +144,7 @@ com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.2
com.google.protobuf:protobuf-java:3.15.2
com.google.re2j:re2j:1.6
com.google.template:soy:2018-03-14
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.googlecode.charts4j:charts4j:1.3
-75
View File
@@ -1,75 +0,0 @@
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
process.env.CHROME_BIN = require('puppeteer').executablePath()
module.exports = function(config) {
config.set({
basePath: '..',
browsers: ['ChromeHeadlessNoSandbox'],
customLaunchers: {
ChromeHeadlessNoSandbox: {
base: 'ChromeHeadless',
flags: ['--no-sandbox']
}
},
frameworks: ['jasmine', 'closure'],
singleRun: true,
autoWatch: false,
files: [
'node_modules/google-closure-library/closure/goog/base.js',
'core/src/test/javascript/**/*_test.js',
{
pattern: 'core/src/test/javascript/**/!(*_test).js',
included: false
},
{
pattern: 'core/src/main/javascript/**/*.js',
included: false
},
{
pattern: 'core/build/generated/sources/custom/java/main/**/*.soy.js',
included: false
},
{
pattern: 'node_modules/google-closure-library/closure/goog/deps.js',
included: false,
served: false
},
{
pattern: 'node_modules/google-closure-library/closure/goog/**/*.js',
included: false
},
{
pattern: 'core/build/resources/main/google/registry/ui/assets/images/*.png',
included: false
},
{
pattern: 'core/build/resources/main/google/registry/ui/assets/images/icons/svg/*.svg',
included: false
}
],
preprocessors: {
'node_modules/google-closure-library/closure/goog/deps.js': ['closure', 'closure-deps'],
'node_modules/google-closure-library/closure/goog/base.js': ['closure'],
'node_modules/google-closure-library/closure/**/*.js': ['closure'],
'core/src/*/javascript/**/*.js': ['closure'],
'core/build/generated/sources/custom/java/main/**/*.soy.js': ['closure'],
},
proxies: {
"/assets/": "/base/core/build/resources/main/google/registry/ui/assets/"
}
});
};
@@ -14,7 +14,7 @@
package google.registry.backup;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import com.google.appengine.api.datastore.EntityTranslator;
import com.google.common.collect.AbstractIterator;
@@ -45,7 +45,7 @@ public class BackupUtils {
* {@link OutputStream} in delimited protocol buffer format.
*/
static void serializeEntity(ImmutableObject entity, OutputStream stream) throws IOException {
EntityTranslator.convertToPb(ofy().save().toEntity(entity)).writeDelimitedTo(stream);
EntityTranslator.convertToPb(auditedOfy().save().toEntity(entity)).writeDelimitedTo(stream);
}
/**
@@ -61,11 +61,12 @@ public class BackupUtils {
@Override
protected ImmutableObject computeNext() {
EntityProto proto = new EntityProto();
if (proto.parseDelimitedFrom(input)) { // False means end of stream; other errors throw.
return ofy().load().fromEntity(EntityTranslator.createFromPb(proto));
if (proto.parseDelimitedFrom(input)) { // False means end of stream; other errors throw.
return auditedOfy().load().fromEntity(EntityTranslator.createFromPb(proto));
}
return endOfData();
}};
}
};
}
public static ImmutableList<ImmutableObject> deserializeEntities(byte[] bytes) {
@@ -18,7 +18,7 @@ import static com.google.appengine.api.taskqueue.QueueFactory.getQueue;
import static com.google.appengine.api.taskqueue.TaskOptions.Builder.withUrl;
import static google.registry.backup.ExportCommitLogDiffAction.LOWER_CHECKPOINT_TIME_PARAM;
import static google.registry.backup.ExportCommitLogDiffAction.UPPER_CHECKPOINT_TIME_PARAM;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
@@ -64,8 +64,7 @@ public final class CommitLogCheckpointAction implements Runnable {
final CommitLogCheckpoint checkpoint = strategy.computeCheckpoint();
logger.atInfo().log(
"Generated candidate checkpoint for time: %s", checkpoint.getCheckpointTime());
tm()
.transact(
tm().transact(
() -> {
DateTime lastWrittenTime = CommitLogCheckpointRoot.loadRoot().getLastWrittenTime();
if (isBeforeOrAt(checkpoint.getCheckpointTime(), lastWrittenTime)) {
@@ -73,7 +72,7 @@ public final class CommitLogCheckpointAction implements Runnable {
"Newer checkpoint already written at time: %s", lastWrittenTime);
return;
}
ofy()
auditedOfy()
.saveWithoutBackup()
.entities(
checkpoint, CommitLogCheckpointRoot.create(checkpoint.getCheckpointTime()));
@@ -17,7 +17,7 @@ package google.registry.backup;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.mapreduce.MapreduceRunner.PARAM_DRY_RUN;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static java.lang.Boolean.FALSE;
import static java.lang.Boolean.TRUE;
@@ -75,9 +75,17 @@ public final class DeleteOldCommitLogsAction implements Runnable {
@Inject MapreduceRunner mrRunner;
@Inject Response response;
@Inject Clock clock;
@Inject @Config("commitLogDatastoreRetention") Duration maxAge;
@Inject @Parameter(PARAM_DRY_RUN) boolean isDryRun;
@Inject DeleteOldCommitLogsAction() {}
@Inject
@Config("commitLogDatastoreRetention")
Duration maxAge;
@Inject
@Parameter(PARAM_DRY_RUN)
boolean isDryRun;
@Inject
DeleteOldCommitLogsAction() {}
@Override
public void run() {
@@ -138,12 +146,12 @@ public final class DeleteOldCommitLogsAction implements Runnable {
// If it isn't a Key<CommitLogManifest> then it should be an EppResource, which we need to
// load to emit the revisions.
//
Object object = ofy().load().key(key).now();
Object object = auditedOfy().load().key(key).now();
checkNotNull(object, "Received a key to a missing object. key: %s", key);
checkState(
object instanceof EppResource,
"Received a key to an object that isn't EppResource nor CommitLogManifest."
+ " Key: %s object type: %s",
+ " Key: %s object type: %s",
key,
object.getClass().getName());
@@ -224,8 +232,7 @@ public final class DeleteOldCommitLogsAction implements Runnable {
* OK to delete this manifestKey. If even one source returns "false" (meaning "it's not OK to
* delete this manifest") then it won't be deleted.
*/
static class DeleteOldCommitLogsReducer
extends Reducer<Key<CommitLogManifest>, Boolean, Void> {
static class DeleteOldCommitLogsReducer extends Reducer<Key<CommitLogManifest>, Boolean, Void> {
private static final long serialVersionUID = -4918760187627937268L;
@@ -241,12 +248,12 @@ public final class DeleteOldCommitLogsAction implements Runnable {
}
public abstract Status status();
public abstract int numDeleted();
static DeletionResult create(Status status, int numDeleted) {
return
new AutoValue_DeleteOldCommitLogsAction_DeleteOldCommitLogsReducer_DeletionResult(
status, numDeleted);
return new AutoValue_DeleteOldCommitLogsAction_DeleteOldCommitLogsReducer_DeletionResult(
status, numDeleted);
}
}
@@ -257,8 +264,7 @@ public final class DeleteOldCommitLogsAction implements Runnable {
@Override
public void reduce(
final Key<CommitLogManifest> manifestKey,
ReducerInput<Boolean> canDeleteVerdicts) {
final Key<CommitLogManifest> manifestKey, ReducerInput<Boolean> canDeleteVerdicts) {
ImmutableMultiset<Boolean> canDeleteMultiset = ImmutableMultiset.copyOf(canDeleteVerdicts);
if (canDeleteMultiset.count(TRUE) > 1) {
getContext().incrementCounter("commit log manifests incorrectly mapped multiple times");
@@ -267,47 +273,54 @@ public final class DeleteOldCommitLogsAction implements Runnable {
getContext().incrementCounter("commit log manifests referenced multiple times");
}
if (canDeleteMultiset.contains(FALSE)) {
getContext().incrementCounter(
canDeleteMultiset.contains(TRUE)
? "old commit log manifests still referenced"
: "new (or nonexistent) commit log manifests referenced");
getContext().incrementCounter(
"EPP resource revisions handled",
canDeleteMultiset.count(FALSE));
getContext()
.incrementCounter(
canDeleteMultiset.contains(TRUE)
? "old commit log manifests still referenced"
: "new (or nonexistent) commit log manifests referenced");
getContext()
.incrementCounter("EPP resource revisions handled", canDeleteMultiset.count(FALSE));
return;
}
DeletionResult deletionResult = tm().transactNew(() -> {
CommitLogManifest manifest = ofy().load().key(manifestKey).now();
// It is possible that the same manifestKey was run twice, if a shard had to be restarted
// or some weird failure. If this happens, we want to exit immediately.
// Note that this can never happen in dryRun.
if (manifest == null) {
return DeletionResult.create(DeletionResult.Status.ALREADY_DELETED, 0);
}
// Doing a sanity check on the date. This is the only place we use the CommitLogManifest,
// so maybe removing this test will improve performance. However, unless it's proven that
// the performance boost is significant (and we've tested this enough to be sure it never
// happens)- the safty of "let's not delete stuff we need from prod" is more important.
if (manifest.getCommitTime().isAfter(deletionThreshold)) {
return DeletionResult.create(DeletionResult.Status.AFTER_THRESHOLD, 0);
}
Iterable<Key<CommitLogMutation>> commitLogMutationKeys = ofy().load()
.type(CommitLogMutation.class)
.ancestor(manifestKey)
.keys()
.iterable();
ImmutableList<Key<?>> keysToDelete = ImmutableList.<Key<?>>builder()
.addAll(commitLogMutationKeys)
.add(manifestKey)
.build();
// Normally in a dry run we would log the entities that would be deleted, but those can
// number in the millions so we skip the logging.
if (!isDryRun) {
ofy().deleteWithoutBackup().keys(keysToDelete);
}
return DeletionResult.create(DeletionResult.Status.SUCCESS, keysToDelete.size());
});
DeletionResult deletionResult =
tm().transactNew(
() -> {
CommitLogManifest manifest = auditedOfy().load().key(manifestKey).now();
// It is possible that the same manifestKey was run twice, if a shard had to be
// restarted or some weird failure. If this happens, we want to exit
// immediately. Note that this can never happen in dryRun.
if (manifest == null) {
return DeletionResult.create(DeletionResult.Status.ALREADY_DELETED, 0);
}
// Doing a sanity check on the date. This is the only place we use the
// CommitLogManifest, so maybe removing this test will improve performance.
// However, unless it's proven that the performance boost is significant (and
// we've tested this enough to be sure it never happens)- the safety of "let's
// not delete stuff we need from prod" is more important.
if (manifest.getCommitTime().isAfter(deletionThreshold)) {
return DeletionResult.create(DeletionResult.Status.AFTER_THRESHOLD, 0);
}
Iterable<Key<CommitLogMutation>> commitLogMutationKeys =
auditedOfy()
.load()
.type(CommitLogMutation.class)
.ancestor(manifestKey)
.keys()
.iterable();
ImmutableList<Key<?>> keysToDelete =
ImmutableList.<Key<?>>builder()
.addAll(commitLogMutationKeys)
.add(manifestKey)
.build();
// Normally in a dry run we would log the entities that would be deleted, but
// those can number in the millions so we skip the logging.
if (!isDryRun) {
auditedOfy().deleteWithoutBackup().keys(keysToDelete);
}
return DeletionResult.create(
DeletionResult.Status.SUCCESS, keysToDelete.size());
});
switch (deletionResult.status()) {
case SUCCESS:
@@ -25,7 +25,7 @@ import static google.registry.backup.BackupUtils.GcsMetadataKeys.NUM_TRANSACTION
import static google.registry.backup.BackupUtils.GcsMetadataKeys.UPPER_BOUND_CHECKPOINT;
import static google.registry.backup.BackupUtils.serializeEntity;
import static google.registry.model.ofy.CommitLogBucket.getBucketKey;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static google.registry.util.DateTimeUtils.isAtOrAfter;
import static java.nio.channels.Channels.newOutputStream;
@@ -89,11 +89,14 @@ public final class ExportCommitLogDiffAction implements Runnable {
checkArgument(lowerCheckpointTime.isBefore(upperCheckpointTime));
// Load the boundary checkpoints - lower is exclusive and may not exist (on the first export,
// when lowerCheckpointTime is START_OF_TIME), whereas the upper is inclusive and must exist.
CommitLogCheckpoint lowerCheckpoint = lowerCheckpointTime.isAfter(START_OF_TIME)
? verifyNotNull(ofy().load().key(CommitLogCheckpoint.createKey(lowerCheckpointTime)).now())
: null;
CommitLogCheckpoint lowerCheckpoint =
lowerCheckpointTime.isAfter(START_OF_TIME)
? verifyNotNull(
auditedOfy().load().key(CommitLogCheckpoint.createKey(lowerCheckpointTime)).now())
: null;
CommitLogCheckpoint upperCheckpoint =
verifyNotNull(ofy().load().key(CommitLogCheckpoint.createKey(upperCheckpointTime)).now());
verifyNotNull(
auditedOfy().load().key(CommitLogCheckpoint.createKey(upperCheckpointTime)).now());
// Load the keys of all the manifests to include in this diff.
List<Key<CommitLogManifest>> sortedKeys = loadAllDiffKeys(lowerCheckpoint, upperCheckpoint);
@@ -117,7 +120,7 @@ public final class ExportCommitLogDiffAction implements Runnable {
// asynchronously load the entities for the next one.
List<List<Key<CommitLogManifest>>> keyChunks = partition(sortedKeys, batchSize);
// Objectify's map return type is asynchronous. Calling .values() will block until it loads.
Map<?, CommitLogManifest> nextChunkToExport = ofy().load().keys(keyChunks.get(0));
Map<?, CommitLogManifest> nextChunkToExport = auditedOfy().load().keys(keyChunks.get(0));
for (int i = 0; i < keyChunks.size(); i++) {
// Force the async load to finish.
Collection<CommitLogManifest> chunkValues = nextChunkToExport.values();
@@ -125,10 +128,10 @@ public final class ExportCommitLogDiffAction implements Runnable {
// Since there is no hard bound on how much data this might be, take care not to let the
// Objectify session cache fill up and potentially run out of memory. This is the only safe
// point to do this since at this point there is no async load in progress.
ofy().clearSessionCache();
auditedOfy().clearSessionCache();
// Kick off the next async load, which can happen in parallel to the current GCS export.
if (i + 1 < keyChunks.size()) {
nextChunkToExport = ofy().load().keys(keyChunks.get(i + 1));
nextChunkToExport = auditedOfy().load().keys(keyChunks.get(i + 1));
}
exportChunk(gcsStream, chunkValues);
logger.atInfo().log("Exported %d manifests", chunkValues.size());
@@ -192,7 +195,8 @@ public final class ExportCommitLogDiffAction implements Runnable {
return ImmutableSet.of();
}
Key<CommitLogBucket> bucketKey = getBucketKey(bucketNum);
return ofy().load()
return auditedOfy()
.load()
.type(CommitLogManifest.class)
.ancestor(bucketKey)
.filterKey(">=", CommitLogManifest.createKey(bucketKey, lowerBound))
@@ -208,7 +212,7 @@ public final class ExportCommitLogDiffAction implements Runnable {
new ImmutableList.Builder<>();
for (CommitLogManifest manifest : chunk) {
entities.add(ImmutableList.of(manifest));
entities.add(ofy().load().type(CommitLogMutation.class).ancestor(manifest));
entities.add(auditedOfy().load().type(CommitLogMutation.class).ancestor(manifest));
}
for (ImmutableObject entity : concat(entities.build())) {
serializeEntity(entity, gcsStream);
@@ -16,7 +16,7 @@ package google.registry.backup;
import static google.registry.backup.ExportCommitLogDiffAction.DIFF_FILE_PREFIX;
import static google.registry.model.ofy.EntityWritePriorities.getEntityPriority;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static javax.servlet.http.HttpServletResponse.SC_NO_CONTENT;
import static org.joda.time.Duration.standardHours;
@@ -151,7 +151,7 @@ public class ReplayCommitLogsToSqlAction implements Runnable {
}
private void handleEntityPut(Entity entity) {
Object ofyPojo = ofy().toPojo(entity);
Object ofyPojo = auditedOfy().toPojo(entity);
if (ofyPojo instanceof DatastoreEntity) {
DatastoreEntity datastoreEntity = (DatastoreEntity) ofyPojo;
datastoreEntity
@@ -18,7 +18,7 @@ import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.Iterators.peekingIterator;
import static google.registry.backup.BackupUtils.createDeserializingIterator;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.Entity;
@@ -146,10 +146,10 @@ public class RestoreCommitLogsAction implements Runnable {
private CommitLogManifest restoreOneTransaction(PeekingIterator<ImmutableObject> commitLogs) {
final CommitLogManifest manifest = (CommitLogManifest) commitLogs.next();
Result<?> deleteResult = deleteAsync(manifest.getDeletions());
List<Entity> entitiesToSave = Lists.newArrayList(ofy().save().toEntity(manifest));
List<Entity> entitiesToSave = Lists.newArrayList(auditedOfy().save().toEntity(manifest));
while (commitLogs.hasNext() && commitLogs.peek() instanceof CommitLogMutation) {
CommitLogMutation mutation = (CommitLogMutation) commitLogs.next();
entitiesToSave.add(ofy().save().toEntity(mutation));
entitiesToSave.add(auditedOfy().save().toEntity(mutation));
entitiesToSave.add(EntityTranslator.createFromPbBytes(mutation.getEntityProtoBytes()));
}
saveRaw(entitiesToSave);
@@ -176,7 +176,8 @@ public class RestoreCommitLogsAction implements Runnable {
return;
}
retrier.callWithRetry(
() -> ofy().saveWithoutBackup().entities(objectsToSave).now(), RuntimeException.class);
() -> auditedOfy().saveWithoutBackup().entities(objectsToSave).now(),
RuntimeException.class);
}
private Result<?> deleteAsync(Set<Key<?>> keysToDelete) {
@@ -185,7 +186,7 @@ public class RestoreCommitLogsAction implements Runnable {
}
return dryRun || keysToDelete.isEmpty()
? new ResultNow<Void>(null)
: ofy().deleteWithoutBackup().keys(keysToDelete);
: auditedOfy().deleteWithoutBackup().keys(keysToDelete);
}
}
@@ -47,7 +47,7 @@ import javax.annotation.Nullable;
*
* <ul>
* <li>Convert an Objectify entity to a Datastore {@link Entity}: {@code
* ofy().save().toEntity(..)}
* auditedOfy().save().toEntity(..)}
* <li>Entity is serializable, but the more efficient approach is to convert an Entity to a
* ProtocolBuffer ({@link com.google.storage.onestore.v3.OnestoreEntity.EntityProto}) and then
* to raw bytes.
@@ -34,7 +34,7 @@ import static google.registry.model.ResourceTransferUtils.denyPendingTransfer;
import static google.registry.model.ResourceTransferUtils.handlePendingTransferOnDelete;
import static google.registry.model.ResourceTransferUtils.updateForeignKeyIndexDeletionTime;
import static google.registry.model.eppcommon.StatusValue.PENDING_DELETE;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.model.reporting.HistoryEntry.Type.CONTACT_DELETE;
import static google.registry.model.reporting.HistoryEntry.Type.CONTACT_DELETE_FAILURE;
import static google.registry.model.reporting.HistoryEntry.Type.HOST_DELETE;
@@ -109,6 +109,7 @@ import org.joda.time.Duration;
* A mapreduce that processes batch asynchronous deletions of contact and host resources by mapping
* over all domains and checking for any references to the contacts/hosts in pending deletion.
*/
@Deprecated
@Action(
service = Action.Service.BACKEND,
path = "/_dr/task/deleteContactsAndHosts",
@@ -335,7 +336,7 @@ public class DeleteContactsAndHostsAction implements Runnable {
DeletionRequest deletionRequest, boolean hasNoActiveReferences) {
DateTime now = tm().getTransactionTime();
EppResource resource =
ofy().load().key(deletionRequest.key()).now().cloneProjectedAtTime(now);
auditedOfy().load().key(deletionRequest.key()).now().cloneProjectedAtTime(now);
// Double-check transactionally that the resource is still active and in PENDING_DELETE.
if (!doesResourceStateAllowDeletion(resource, now)) {
return DeletionResult.create(Type.ERRORED, "");
@@ -369,11 +370,10 @@ public class DeleteContactsAndHostsAction implements Runnable {
: "it was transferred prior to deletion");
HistoryEntry historyEntry =
new HistoryEntry.Builder()
HistoryEntry.createBuilderForResource(resource)
.setClientId(deletionRequest.requestingClientId())
.setModificationTime(now)
.setType(getHistoryEntryType(resource, deleteAllowed))
.setParent(deletionRequest.key())
.build();
PollMessage.OneTime pollMessage =
@@ -408,7 +408,9 @@ public class DeleteContactsAndHostsAction implements Runnable {
} else {
resourceToSave = resource.asBuilder().removeStatusValue(PENDING_DELETE).build();
}
ofy().save().<ImmutableObject>entities(resourceToSave, historyEntry, pollMessage);
auditedOfy()
.save()
.<ImmutableObject>entities(resourceToSave, historyEntry.asHistoryEntry(), pollMessage);
return DeletionResult.create(
deleteAllowed ? Type.DELETED : Type.NOT_DELETED, pollMessageText);
}
@@ -525,7 +527,8 @@ public class DeleteContactsAndHostsAction implements Runnable {
Key.create(
checkNotNull(params.get(PARAM_RESOURCE_KEY), "Resource to delete not specified"));
EppResource resource =
checkNotNull(ofy().load().key(resourceKey).now(), "Resource to delete doesn't exist");
checkNotNull(
auditedOfy().load().key(resourceKey).now(), "Resource to delete doesn't exist");
checkState(
resource instanceof ContactResource || resource instanceof HostResource,
"Cannot delete a %s via this action",
@@ -18,7 +18,7 @@ import static com.google.common.base.Preconditions.checkState;
import static google.registry.config.RegistryEnvironment.PRODUCTION;
import static google.registry.mapreduce.MapreduceRunner.PARAM_DRY_RUN;
import static google.registry.mapreduce.inputs.EppResourceInputs.createEntityInput;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
@@ -125,12 +125,11 @@ public class DeleteLoadTestDataAction implements Runnable {
Key.create(EppResourceIndex.create(Key.create(resource)));
final Key<? extends ForeignKeyIndex<?>> fki = ForeignKeyIndex.createKey(resource);
int numEntitiesDeleted =
tm()
.transact(
tm().transact(
() -> {
// This ancestor query selects all descendant entities.
List<Key<Object>> resourceAndDependentKeys =
ofy().load().ancestor(resource).keys().list();
auditedOfy().load().ancestor(resource).keys().list();
ImmutableSet<Key<?>> allKeys =
new ImmutableSet.Builder<Key<?>>()
.add(fki)
@@ -140,7 +139,7 @@ public class DeleteLoadTestDataAction implements Runnable {
if (isDryRun) {
logger.atInfo().log("Would hard-delete the following entities: %s", allKeys);
} else {
ofy().deleteWithoutBackup().keys(allKeys);
auditedOfy().deleteWithoutBackup().keys(allKeys);
}
return allKeys.size();
});
@@ -20,7 +20,7 @@ import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.config.RegistryEnvironment.PRODUCTION;
import static google.registry.mapreduce.MapreduceRunner.PARAM_DRY_RUN;
import static google.registry.model.ResourceTransferUtils.updateForeignKeyIndexDeletionTime;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.model.registry.Registries.getTldsOfType;
import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_DELETE;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
@@ -44,11 +44,11 @@ import google.registry.mapreduce.MapreduceRunner;
import google.registry.mapreduce.inputs.EppResourceInputs;
import google.registry.model.EppResourceUtils;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.index.EppResourceIndex;
import google.registry.model.index.ForeignKeyIndex;
import google.registry.model.registry.Registry;
import google.registry.model.registry.Registry.TldType;
import google.registry.model.reporting.HistoryEntry;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.Response;
@@ -166,7 +166,7 @@ public class DeleteProberDataAction implements Runnable {
}
private void deleteDomain(final Key<DomainBase> domainKey) {
final DomainBase domain = ofy().load().key(domainKey).now();
final DomainBase domain = auditedOfy().load().key(domainKey).now();
DateTime now = DateTime.now(UTC);
@@ -220,14 +220,13 @@ public class DeleteProberDataAction implements Runnable {
final Key<? extends ForeignKeyIndex<?>> fki = ForeignKeyIndex.createKey(domain);
int entitiesDeleted =
tm()
.transact(
tm().transact(
() -> {
// This ancestor query selects all descendant HistoryEntries, BillingEvents,
// PollMessages,
// and TLD-specific entities, as well as the domain itself.
List<Key<Object>> domainAndDependentKeys =
ofy().load().ancestor(domainKey).keys().list();
auditedOfy().load().ancestor(domainKey).keys().list();
ImmutableSet<Key<?>> allKeys =
new ImmutableSet.Builder<Key<?>>()
.add(fki)
@@ -237,7 +236,7 @@ public class DeleteProberDataAction implements Runnable {
if (isDryRun) {
logger.atInfo().log("Would hard-delete the following entities: %s", allKeys);
} else {
ofy().deleteWithoutBackup().keys(allKeys);
auditedOfy().deleteWithoutBackup().keys(allKeys);
}
return allKeys.size();
});
@@ -254,9 +253,9 @@ public class DeleteProberDataAction implements Runnable {
.setDeletionTime(tm().getTransactionTime())
.setStatusValues(null)
.build();
HistoryEntry historyEntry =
new HistoryEntry.Builder()
.setParent(domain)
DomainHistory historyEntry =
new DomainHistory.Builder()
.setDomain(domain)
.setType(DOMAIN_DELETE)
.setModificationTime(tm().getTransactionTime())
.setBySuperuser(true)
@@ -264,11 +263,9 @@ public class DeleteProberDataAction implements Runnable {
.setClientId(registryAdminClientId)
.build();
// Note that we don't bother handling grace periods, billing events, pending
// transfers,
// poll messages, or auto-renews because these will all be hard-deleted the next
// time the
// mapreduce runs anyway.
ofy().save().entities(deletedDomain, historyEntry);
// transfers, poll messages, or auto-renews because these will all be hard-deleted
// the next time the mapreduce runs anyway.
tm().putAll(deletedDomain, historyEntry);
updateForeignKeyIndexDeletionTime(deletedDomain);
dnsQueue.addDomainRefreshTask(deletedDomain.getDomainName());
});
@@ -49,6 +49,7 @@ import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.common.Cursor;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.Period;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.DomainTransactionRecord;
@@ -91,7 +92,8 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
@Override
public void run() {
Cursor cursor = ofy().load().key(Cursor.createGlobalKey(RECURRING_BILLING)).now();
Cursor cursor =
tm().loadByKeyIfPresent(Cursor.createGlobalVKey(RECURRING_BILLING)).orElse(null);
DateTime executeTime = clock.nowUtc();
DateTime persistedCursorTime = (cursor == null ? START_OF_TIME : cursor.getCursorTime());
DateTime cursorTime = cursorTimeParam.orElse(persistedCursorTime);
@@ -187,12 +189,14 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
// an event persisted.
for (DateTime billingTime : difference(billingTimes, existingBillingTimes)) {
// Construct a new HistoryEntry that parents over the OneTime
HistoryEntry historyEntry =
new HistoryEntry.Builder()
DomainHistory historyEntry =
new DomainHistory.Builder()
.setBySuperuser(false)
.setClientId(recurring.getClientId())
.setModificationTime(tm().getTransactionTime())
.setParent(domainKey)
// TODO (jianglai): modify this to use setDomain instead when
// converting this action to be SQL-aware.
.setDomainRepoId(domainKey.getName())
.setPeriod(Period.create(1, YEARS))
.setReason(
"Domain autorenewal by ExpandRecurringBillingEventsAction")
@@ -15,7 +15,7 @@
package google.registry.batch;
import static google.registry.mapreduce.MapreduceRunner.PARAM_FAST;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.appengine.tools.mapreduce.Mapper;
@@ -104,13 +104,13 @@ public class ResaveAllEppResourcesAction implements Runnable {
boolean resaved =
tm().transact(
() -> {
EppResource originalResource = ofy().load().key(resourceKey).now();
EppResource originalResource = auditedOfy().load().key(resourceKey).now();
EppResource projectedResource =
originalResource.cloneProjectedAtTime(tm().getTransactionTime());
if (isFast && originalResource.equals(projectedResource)) {
return false;
} else {
ofy().save().entity(projectedResource).now();
auditedOfy().save().entity(projectedResource).now();
return true;
}
});
@@ -21,21 +21,30 @@ import com.google.auto.value.AutoValue;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Streams;
import google.registry.backup.AppEngineEnvironment;
import google.registry.beam.common.RegistryQuery.QueryComposerFactory;
import google.registry.beam.common.RegistryQuery.RegistryQueryFactory;
import google.registry.model.ofy.ObjectifyService;
import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.persistence.transaction.TransactionManagerFactory;
import java.io.Serializable;
import java.util.Objects;
import java.util.concurrent.ThreadLocalRandom;
import javax.persistence.criteria.CriteriaQuery;
import org.apache.beam.sdk.coders.Coder;
import org.apache.beam.sdk.coders.SerializableCoder;
import org.apache.beam.sdk.metrics.Counter;
import org.apache.beam.sdk.metrics.Metrics;
import org.apache.beam.sdk.transforms.Create;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.GroupIntoBatches;
import org.apache.beam.sdk.transforms.PTransform;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.transforms.Reshuffle;
import org.apache.beam.sdk.transforms.SerializableFunction;
import org.apache.beam.sdk.transforms.WithKeys;
import org.apache.beam.sdk.util.ShardedKey;
import org.apache.beam.sdk.values.KV;
import org.apache.beam.sdk.values.PBegin;
import org.apache.beam.sdk.values.PCollection;
/**
@@ -51,10 +60,127 @@ public final class RegistryJpaIO {
private RegistryJpaIO() {}
public static <R> Read<R, R> read(QueryComposerFactory<R> queryFactory) {
return Read.<R, R>builder().queryFactory(queryFactory).build();
}
public static <R, T> Read<R, T> read(
QueryComposerFactory<R> queryFactory, SerializableFunction<R, T> resultMapper) {
return Read.<R, T>builder().queryFactory(queryFactory).resultMapper(resultMapper).build();
}
/**
* Returns a {@link Read} connector based on the given {@code jpql} query string.
*
* <p>User should take care to prevent sql-injection attacks.
*/
public static <R, T> Read<R, T> read(String jpql, SerializableFunction<R, T> resultMapper) {
return Read.<R, T>builder().jpqlQueryFactory(jpql).resultMapper(resultMapper).build();
}
public static <T> Write<T> write() {
return Write.<T>builder().build();
}
/**
* A {@link PTransform transform} that transactionally executes a JPA {@link CriteriaQuery} and
* adds the results to the BEAM pipeline. Users have the option to transform the results before
* sending them to the next stages.
*/
@AutoValue
public abstract static class Read<R, T> extends PTransform<PBegin, PCollection<T>> {
public static final String DEFAULT_NAME = "RegistryJpaIO.Read";
abstract String name();
abstract RegistryQueryFactory<R> queryFactory();
abstract SerializableFunction<R, T> resultMapper();
abstract Coder<T> coder();
abstract Builder<R, T> toBuilder();
@Override
@SuppressWarnings("deprecation") // Reshuffle still recommended by GCP.
public PCollection<T> expand(PBegin input) {
return input
.apply("Starting " + name(), Create.of((Void) null))
.apply(
"Run query for " + name(),
ParDo.of(new QueryRunner<>(queryFactory(), resultMapper())))
.setCoder(coder())
.apply("Reshuffle", Reshuffle.viaRandomKey());
}
public Read<R, T> withName(String name) {
return toBuilder().name(name).build();
}
public Read<R, T> withResultMapper(SerializableFunction<R, T> mapper) {
return toBuilder().resultMapper(mapper).build();
}
public Read<R, T> withCoder(Coder<T> coder) {
return toBuilder().coder(coder).build();
}
static <R, T> Builder<R, T> builder() {
return new AutoValue_RegistryJpaIO_Read.Builder()
.name(DEFAULT_NAME)
.resultMapper(x -> x)
.coder(SerializableCoder.of(Serializable.class));
}
@AutoValue.Builder
public abstract static class Builder<R, T> {
abstract Builder<R, T> name(String name);
abstract Builder<R, T> queryFactory(RegistryQueryFactory<R> queryFactory);
abstract Builder<R, T> resultMapper(SerializableFunction<R, T> mapper);
abstract Builder<R, T> coder(Coder coder);
abstract Read<R, T> build();
Builder<R, T> queryFactory(QueryComposerFactory<R> queryFactory) {
return queryFactory(RegistryQuery.createQueryFactory(queryFactory));
}
Builder<R, T> jpqlQueryFactory(String jpql) {
return queryFactory(RegistryQuery.createQueryFactory(jpql));
}
}
static class QueryRunner<R, T> extends DoFn<Void, T> {
private final RegistryQueryFactory<R> queryFactory;
private final SerializableFunction<R, T> resultMapper;
QueryRunner(RegistryQueryFactory<R> queryFactory, SerializableFunction<R, T> resultMapper) {
this.queryFactory = queryFactory;
this.resultMapper = resultMapper;
}
@ProcessElement
public void processElement(OutputReceiver<T> outputReceiver) {
// AppEngineEnvironment is need for handling VKeys, which involve Ofy keys. Unlike
// SqlBatchWriter, it is unnecessary to initialize ObjectifyService in this class.
try (AppEngineEnvironment env = new AppEngineEnvironment()) {
// TODO(b/187210388): JpaTransactionManager should support non-transactional query.
jpaTm()
.transactNoRetry(
() ->
queryFactory.apply(jpaTm()).stream()
.map(resultMapper::apply)
.forEach(outputReceiver::output));
}
}
}
}
/**
* A {@link PTransform transform} that writes a PCollection of entities to the SQL database using
* the {@link JpaTransactionManager}.
@@ -182,8 +308,9 @@ public final class RegistryJpaIO {
@Setup
public void setup() {
// Below is needed as long as Objectify keys are still involved in the handling of SQL
// entities (e.g., in VKeys).
// AppEngineEnvironment is needed as long as Objectify keys are still involved in the handling
// of SQL entities (e.g., in VKeys). ObjectifyService needs to be initialized when conversion
// between Ofy entity and Datastore entity is needed.
try (AppEngineEnvironment env = new AppEngineEnvironment()) {
ObjectifyService.initOfy();
}
@@ -0,0 +1,89 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.common;
import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.persistence.transaction.QueryComposer;
import java.util.stream.Stream;
import javax.persistence.EntityManager;
import javax.persistence.Query;
import org.apache.beam.sdk.transforms.SerializableFunction;
/** Interface for query instances used by {@link RegistryJpaIO.Read}. */
public interface RegistryQuery<T> {
Stream<T> stream();
/** Factory for {@link RegistryQuery}. */
interface RegistryQueryFactory<T>
extends SerializableFunction<JpaTransactionManager, RegistryQuery<T>> {}
// TODO(mmuller): Consider detached JpaQueryComposer that works with any JpaTransactionManager
// instance, i.e., change composer.buildQuery() to composer.buildQuery(JpaTransactionManager).
// This way QueryComposer becomes reusable and serializable (at least with Hibernate), and this
// interface would no longer be necessary.
interface QueryComposerFactory<T>
extends SerializableFunction<JpaTransactionManager, QueryComposer<T>> {}
/**
* Returns a {@link RegistryQueryFactory} that creates a JPQL query from constant text.
*
* @param <T> Type of each row in the result set, {@link Object} in single-select queries, and
* {@code Object[]} in multi-select queries.
*/
@SuppressWarnings("unchecked") // query.getResultStream: jpa api uses raw type
static <T> RegistryQueryFactory<T> createQueryFactory(String jpql) {
return (JpaTransactionManager jpa) ->
() -> {
EntityManager entityManager = jpa.getEntityManager();
Query query = entityManager.createQuery(jpql);
return query.getResultStream().map(e -> detach(entityManager, e));
};
}
static <T> RegistryQueryFactory<T> createQueryFactory(
QueryComposerFactory<T> queryComposerFactory) {
return (JpaTransactionManager jpa) ->
() -> queryComposerFactory.apply(jpa).withAutoDetachOnLoad(true).stream();
}
/**
* Removes an object from the JPA session cache if applicable.
*
* @param object An object that represents a row in the result set. It may be a JPA entity, a
* non-entity object, or an array that holds JPA entities and/or non-entities.
*/
static <T> T detach(EntityManager entityManager, T object) {
if (object.getClass().isArray()) {
for (Object arrayElement : (Object[]) object) {
detachObject(entityManager, arrayElement);
}
} else {
detachObject(entityManager, object);
}
return object;
}
static void detachObject(EntityManager entityManager, Object object) {
Class<?> objectClass = object.getClass();
if (objectClass.isPrimitive() || objectClass == String.class) {
return;
}
try {
entityManager.detach(object);
} catch (IllegalArgumentException e) {
// Not an entity. Do nothing.
}
}
}
@@ -87,20 +87,18 @@ public class BulkDeleteDatastorePipeline {
private final BulkDeletePipelineOptions options;
private final Pipeline pipeline;
BulkDeleteDatastorePipeline(BulkDeletePipelineOptions options) {
this.options = options;
pipeline = Pipeline.create(options);
}
public void run() {
setupPipeline();
Pipeline pipeline = Pipeline.create(options);
setupPipeline(pipeline);
pipeline.run();
}
@SuppressWarnings("deprecation") // org.apache.beam.sdk.transforms.Reshuffle
private void setupPipeline() {
private void setupPipeline(Pipeline pipeline) {
checkState(
!FORBIDDEN_PROJECTS.contains(options.getProject()),
"Bulk delete is forbidden in %s",
@@ -120,26 +120,22 @@ public class InitSqlPipeline implements Serializable {
private final InitSqlPipelineOptions options;
private final Pipeline pipeline;
InitSqlPipeline(InitSqlPipelineOptions options) {
this.options = options;
pipeline = Pipeline.create(options);
}
PipelineResult run() {
return run(Pipeline.create(options));
}
@VisibleForTesting
InitSqlPipeline(InitSqlPipelineOptions options, Pipeline pipeline) {
this.options = options;
this.pipeline = pipeline;
}
public PipelineResult run() {
setupPipeline();
PipelineResult run(Pipeline pipeline) {
setupPipeline(pipeline);
return pipeline.run();
}
@VisibleForTesting
void setupPipeline() {
void setupPipeline(Pipeline pipeline) {
options.setIsolationOverride(TransactionIsolationLevel.TRANSACTION_READ_UNCOMMITTED);
PCollectionTuple datastoreSnapshot =
pipeline.apply(
@@ -19,7 +19,7 @@ import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.beam.initsql.BackupPaths.getCommitLogTimestamp;
import static google.registry.beam.initsql.BackupPaths.getExportFilePatterns;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import static java.util.Comparator.comparing;
@@ -32,9 +32,12 @@ import com.google.appengine.api.datastore.EntityTranslator;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Streams;
import google.registry.backup.CommitLogImports;
import google.registry.backup.VersionedEntity;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.domain.DomainBase;
import google.registry.model.reporting.HistoryEntry;
import google.registry.schema.replay.DatastoreAndSqlEntity;
@@ -256,7 +259,58 @@ public final class Transforms {
.iterator()));
}
// Production data repair configs go below. See b/185954992.
// Prober domains in bad state, without associated contacts, hosts, billings, and history.
// They can be safely ignored.
private static final ImmutableSet<String> IGNORED_DOMAINS =
ImmutableSet.of("6AF6D2-IQCANT", "2-IQANYT");
// Prober hosts referencing phantom registrars. They and their associated history entries can be
// safely ignored.
private static final ImmutableSet<String> IGNORED_HOSTS =
ImmutableSet.of(
"4E21_WJ0TEST-GOOGLE",
"4E21_WJ1TEST-GOOGLE",
"4E21_WJ2TEST-GOOGLE",
"4E21_WJ3TEST-GOOGLE");
// Prober contacts referencing phantom registrars. They and their associated history entries can
// be safely ignored.
private static final ImmutableSet IGNORED_CONTACTS =
ImmutableSet.of(
"1_WJ0TEST-GOOGLE", "1_WJ1TEST-GOOGLE", "1_WJ2TEST-GOOGLE", "1_WJ3TEST-GOOGLE");
private static boolean isMigratable(Entity entity) {
// Checks specific to production data. See b/185954992 for details.
// The names of these bad entities in production do not conflict with other environments. For
// simplicities sake we apply them regardless of the source of the data.
if (entity.getKind().equals("DomainBase")
&& IGNORED_DOMAINS.contains(entity.getKey().getName())) {
return false;
}
if (entity.getKind().equals("ContactResource")) {
String roid = entity.getKey().getName();
return !IGNORED_CONTACTS.contains(roid);
}
if (entity.getKind().equals("HostResource")) {
String roid = entity.getKey().getName();
return !IGNORED_HOSTS.contains(roid);
}
if (entity.getKind().equals("HistoryEntry")) {
// Remove production bad data: History of the contacts to be ignored:
com.google.appengine.api.datastore.Key parentKey = entity.getKey().getParent();
if (parentKey.getKind().equals("ContactResource")) {
String contactRoid = parentKey.getName();
return !IGNORED_CONTACTS.contains(contactRoid);
}
if (parentKey.getKind().equals("HostResource")) {
String hostRoid = parentKey.getName();
return !IGNORED_HOSTS.contains(hostRoid);
}
}
// End of production-specific checks.
if (entity.getKind().equals("HistoryEntry")) {
// DOMAIN_APPLICATION_CREATE is deprecated type and should not be migrated.
// The Enum name DOMAIN_APPLICATION_CREATE no longer exists in Java and cannot
@@ -266,6 +320,18 @@ public final class Transforms {
return true;
}
private static Entity repairBadData(Entity entity) {
if (entity.getKind().equals("Cancellation")
&& Objects.equals(entity.getProperty("reason"), "AUTO_RENEW")) {
// AUTO_RENEW has been moved from 'reason' to flags. Change reason to RENEW and add the
// AUTO_RENEW flag. Note: all affected entities have empty flags so we can simply assign
// instead of append. See b/185954992.
entity.setUnindexedProperty("reason", Reason.RENEW.name());
entity.setUnindexedProperty("flags", ImmutableList.of(Flag.AUTO_RENEW.name()));
}
return entity;
}
private static SqlEntity toSqlEntity(Object ofyEntity) {
if (ofyEntity instanceof HistoryEntry) {
HistoryEntry ofyHistory = (HistoryEntry) ofyEntity;
@@ -286,7 +352,8 @@ public final class Transforms {
return dsEntity
.getEntity()
.filter(Transforms::isMigratable)
.map(e -> ofy().toPojo(e))
.map(Transforms::repairBadData)
.map(e -> auditedOfy().toPojo(e))
.map(Transforms::toSqlEntity)
.orElse(null);
}
@@ -17,7 +17,6 @@ package google.registry.beam.invoicing;
import static google.registry.beam.BeamUtils.getQueryFromFile;
import static org.apache.beam.sdk.values.TypeDescriptors.strings;
import com.google.common.annotations.VisibleForTesting;
import google.registry.beam.invoicing.BillingEvent.InvoiceGroupingKey;
import google.registry.beam.invoicing.BillingEvent.InvoiceGroupingKey.InvoiceGroupingKeyCoder;
import google.registry.reporting.billing.BillingModule;
@@ -60,24 +59,18 @@ public class InvoicingPipeline implements Serializable {
DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.SSSSSS");
private final InvoicingPipelineOptions options;
private final Pipeline pipeline;
@VisibleForTesting
InvoicingPipeline(InvoicingPipelineOptions options, Pipeline pipeline) {
this.options = options;
this.pipeline = pipeline;
}
InvoicingPipeline(InvoicingPipelineOptions options) {
this(options, Pipeline.create(options));
this.options = options;
}
PipelineResult run() {
setupPipeline();
Pipeline pipeline = Pipeline.create(options);
setupPipeline(pipeline);
return pipeline.run();
}
void setupPipeline() {
void setupPipeline(Pipeline pipeline) {
PCollection<BillingEvent> billingEvents =
pipeline.apply(
"Read BillingEvents from Bigquery",
@@ -18,7 +18,6 @@ import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.beam.BeamUtils.getQueryFromFile;
import com.google.auto.value.AutoValue;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableSet;
import dagger.Component;
import dagger.Module;
@@ -84,26 +83,19 @@ public class Spec11Pipeline implements Serializable {
private final Spec11PipelineOptions options;
private final EvaluateSafeBrowsingFn safeBrowsingFn;
private final Pipeline pipeline;
@VisibleForTesting
Spec11Pipeline(
Spec11PipelineOptions options, EvaluateSafeBrowsingFn safeBrowsingFn, Pipeline pipeline) {
this.options = options;
this.safeBrowsingFn = safeBrowsingFn;
this.pipeline = pipeline;
}
Spec11Pipeline(Spec11PipelineOptions options, EvaluateSafeBrowsingFn safeBrowsingFn) {
this(options, safeBrowsingFn, Pipeline.create(options));
this.options = options;
this.safeBrowsingFn = safeBrowsingFn;
}
PipelineResult run() {
setupPipeline();
Pipeline pipeline = Pipeline.create(options);
setupPipeline(pipeline);
return pipeline.run();
}
void setupPipeline() {
void setupPipeline(Pipeline pipeline) {
PCollection<Subdomain> domains =
pipeline.apply(
"Read active domains from BigQuery",
@@ -203,7 +203,7 @@ public final class RegistryConfig {
* Configuration for analytics services installed in the web console.
*
* @see google.registry.ui.server.registrar.ConsoleUiAction
* @see google.registry.ui.soy.AnalyticsSoyInfo
* @see google.registry.ui.soy.registrar.AnalyticsSoyInfo
*/
@Provides
@Config("analyticsConfig")
@@ -1,5 +1,5 @@
# Example of a reserved list file. This is simply a CSV file with two
# columns: sub-domain name and price (specified as currency type and value).
# Example of a premium list file. This is simply a CSV file with two
# columns: sub-domain name, and price (specified as currency type and value).
#
# These are manipulated using the "nomulus" tool
# {create,update,delete,list}_premium_list commands.
@@ -391,6 +391,12 @@
<url-pattern>/_dr/task/wipeOutDatastore</url-pattern>
</servlet-mapping>
<!-- Action to create synthetic history entries during async replication to SQL -->
<servlet-mapping>
<servlet-name>backend-servlet</servlet-name>
<url-pattern>/_dr/task/createSyntheticHistoryEntries</url-pattern>
</servlet-mapping>
<!-- Security config -->
<security-constraint>
<web-resource-collection>
@@ -32,12 +32,12 @@ import com.google.common.net.MediaType;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.registry.Registry;
import google.registry.model.registry.label.PremiumList.PremiumListEntry;
import google.registry.model.registry.label.PremiumListDualDao;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.RequestParameters;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.schema.tld.PremiumListDao;
import google.registry.storage.drive.DriveConnection;
import java.io.IOException;
import java.util.Optional;
@@ -113,7 +113,7 @@ public class ExportPremiumTermsAction implements Runnable {
"Skipping premium terms export for TLD %s because Drive folder isn't specified", tld);
return Optional.of("Skipping export because no Drive folder is associated with this TLD");
}
if (registry.getPremiumList() == null) {
if (!registry.getPremiumList().isPresent()) {
logger.atInfo().log("No premium terms to export for TLD %s", tld);
return Optional.of("No premium lists configured");
}
@@ -137,11 +137,13 @@ public class ExportPremiumTermsAction implements Runnable {
}
private String getFormattedPremiumTerms(Registry registry) {
String premiumListName = registry.getPremiumList().getName();
checkState(registry.getPremiumList().isPresent(), "%s does not have a premium list", tld);
String premiumListName = registry.getPremiumList().get().getName();
checkState(
PremiumListDualDao.exists(premiumListName), "Could not load premium list for " + tld);
PremiumListDao.getLatestRevision(premiumListName).isPresent(),
"Could not load premium list for " + tld);
SortedSet<String> premiumTerms =
Streams.stream(PremiumListDualDao.loadAllPremiumListEntries(premiumListName))
Streams.stream(PremiumListDao.loadAllPremiumListEntries(premiumListName))
.map(PremiumListEntry::toString)
.collect(ImmutableSortedSet.toImmutableSortedSet(String::compareTo));
@@ -20,6 +20,7 @@ import com.google.common.base.Strings;
import dagger.Module;
import dagger.Provides;
import google.registry.flows.picker.FlowPicker;
import google.registry.model.contact.ContactHistory;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.AuthInfo;
@@ -31,6 +32,7 @@ import google.registry.model.eppinput.ResourceCommand;
import google.registry.model.eppinput.ResourceCommand.SingleResourceCommand;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.eppoutput.Result;
import google.registry.model.host.HostHistory;
import google.registry.model.reporting.HistoryEntry;
import java.lang.annotation.Documented;
import java.util.Optional;
@@ -211,39 +213,78 @@ public class FlowModule {
return Strings.nullToEmpty(((Poll) eppInput.getCommandWrapper().getCommand()).getMessageId());
}
private static <B extends HistoryEntry.Builder<? extends HistoryEntry, ?>>
B makeHistoryEntryBuilder(
B builder,
Trid trid,
byte[] inputXmlBytes,
boolean isSuperuser,
String clientId,
EppInput eppInput) {
builder
.setTrid(trid)
.setXmlBytes(inputXmlBytes)
.setBySuperuser(isSuperuser)
.setClientId(clientId);
Optional<MetadataExtension> metadataExtension =
eppInput.getSingleExtension(MetadataExtension.class);
metadataExtension.ifPresent(
extension ->
builder
.setReason(extension.getReason())
.setRequestedByRegistrar(extension.getRequestedByRegistrar()));
return builder;
}
/**
* Provides a partially filled in {@link HistoryEntry} builder.
* Provides a partially filled in {@link ContactHistory.Builder}
*
* <p>This is not marked with {@link FlowScope} so that each retry gets a fresh one. Otherwise,
* the fact that the builder is one-use would cause NPEs.
*/
@Provides
static HistoryEntry.Builder provideHistoryEntryBuilder(
static ContactHistory.Builder provideContactHistoryBuilder(
Trid trid,
@InputXml byte[] inputXmlBytes,
@Superuser boolean isSuperuser,
@ClientId String clientId,
EppInput eppInput) {
HistoryEntry.Builder historyBuilder =
new HistoryEntry.Builder()
.setTrid(trid)
.setXmlBytes(inputXmlBytes)
.setBySuperuser(isSuperuser)
.setClientId(clientId);
Optional<MetadataExtension> metadataExtension =
eppInput.getSingleExtension(MetadataExtension.class);
if (metadataExtension.isPresent()) {
historyBuilder
.setReason(metadataExtension.get().getReason())
.setRequestedByRegistrar(metadataExtension.get().getRequestedByRegistrar());
}
return historyBuilder;
return makeHistoryEntryBuilder(
new ContactHistory.Builder(), trid, inputXmlBytes, isSuperuser, clientId, eppInput);
}
/**
* Provides a partially filled in {@link HostHistory.Builder}
*
* <p>This is not marked with {@link FlowScope} so that each retry gets a fresh one. Otherwise,
* the fact that the builder is one-use would cause NPEs.
*/
@Provides
static HostHistory.Builder provideHostHistoryBuilder(
Trid trid,
@InputXml byte[] inputXmlBytes,
@Superuser boolean isSuperuser,
@ClientId String clientId,
EppInput eppInput) {
return makeHistoryEntryBuilder(
new HostHistory.Builder(), trid, inputXmlBytes, isSuperuser, clientId, eppInput);
}
/**
* Provides a partially filled in {@link DomainHistory.Builder}
*
* <p>This is not marked with {@link FlowScope} so that each retry gets a fresh one. Otherwise,
* the fact that the builder is one-use would cause NPEs.
*/
@Provides
static DomainHistory.Builder provideDomainHistoryBuilder(
HistoryEntry.Builder historyEntryBuilder) {
return new DomainHistory.Builder().copyFrom(historyEntryBuilder.build());
Trid trid,
@InputXml byte[] inputXmlBytes,
@Superuser boolean isSuperuser,
@ClientId String clientId,
EppInput eppInput) {
return makeHistoryEntryBuilder(
new DomainHistory.Builder(), trid, inputXmlBytes, isSuperuser, clientId, eppInput);
}
/**
@@ -256,7 +297,7 @@ public class FlowModule {
static EppResponse.Builder provideEppResponseBuilder(Trid trid) {
return new EppResponse.Builder()
.setTrid(trid)
.setResultFromCode(Result.Code.SUCCESS); // Default to success.
.setResultFromCode(Result.Code.SUCCESS); // Default to success.
}
@Provides
@@ -22,15 +22,19 @@ import static java.nio.charset.StandardCharsets.UTF_8;
import com.google.common.base.Throwables;
import com.google.common.flogger.FluentLogger;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException.CommandUseErrorException;
import google.registry.flows.EppException.ParameterValueRangeErrorException;
import google.registry.flows.EppException.SyntaxErrorException;
import google.registry.flows.EppException.UnimplementedProtocolVersionException;
import google.registry.flows.custom.EntityChanges;
import google.registry.model.EppResource;
import google.registry.model.eppcommon.EppXmlTransformer;
import google.registry.model.eppinput.EppInput.WrongProtocolVersionException;
import google.registry.model.eppoutput.EppOutput;
import google.registry.model.host.InetAddressAdapter.IpVersionMismatchException;
import google.registry.model.ofy.ObjectifyService;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.translators.CurrencyUnitAdapter.UnknownCurrencyException;
import google.registry.xml.XmlException;
import java.util.List;
@@ -99,6 +103,11 @@ public final class FlowUtils {
}
}
public static <H extends HistoryEntry> Key<H> createHistoryKey(
EppResource parent, Class<H> clazz) {
return Key.create(Key.create(parent), clazz, ObjectifyService.allocateId());
}
/** Registrar is not logged in. */
public static class NotLoggedInException extends CommandUseErrorException {
public NotLoggedInException() {
@@ -16,6 +16,7 @@ package google.registry.flows;
import static com.google.common.collect.Sets.intersection;
import static google.registry.model.EppResourceUtils.getLinkedDomainKeys;
import static google.registry.model.EppResourceUtils.isLinked;
import static google.registry.model.EppResourceUtils.loadByForeignKey;
import static google.registry.model.index.ForeignKeyIndex.loadAndGetKey;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
@@ -62,7 +63,10 @@ public final class ResourceFlowUtils {
private ResourceFlowUtils() {}
/** In {@link #failfastForAsyncDelete}, check this (arbitrary) number of query results. */
/**
* In {@link #checkLinkedDomains(String, DateTime, Class, Function)}, check this (arbitrary)
* number of query results.
*/
private static final int FAILFAST_CHECK_COUNT = 5;
/** Check that the given clientId corresponds to the owner of given resource. */
@@ -73,36 +77,54 @@ public final class ResourceFlowUtils {
}
}
/** Check whether an asynchronous delete would obviously fail, and throw an exception if so. */
public static <R extends EppResource> void failfastForAsyncDelete(
/**
* Check whether if there are domains linked to the resource to be deleted. Throws an exception if
* so.
*
* <p>Note that in datastore this is a smoke test as the query for linked domains is eventually
* consistent, so we only check a few domains to fail fast.
*/
public static <R extends EppResource> void checkLinkedDomains(
final String targetId,
final DateTime now,
final Class<R> resourceClass,
final Function<DomainBase, ImmutableSet<?>> getPotentialReferences) throws EppException {
// Enter a transactionless context briefly.
final Function<DomainBase, ImmutableSet<?>> getPotentialReferences)
throws EppException {
EppException failfastException =
tm().doTransactionless(
() -> {
final ForeignKeyIndex<R> fki = ForeignKeyIndex.load(resourceClass, targetId, now);
if (fki == null) {
return new ResourceDoesNotExistException(resourceClass, targetId);
}
/* Query for the first few linked domains, and if found, actually load them. The
* query is eventually consistent and so might be very stale, but the direct
* load will not be stale, just non-transactional. If we find at least one
* actual reference then we can reliably fail. If we don't find any, we can't
* trust the query and need to do the full mapreduce.
*/
Iterable<VKey<DomainBase>> keys =
getLinkedDomainKeys(fki.getResourceKey(), now, FAILFAST_CHECK_COUNT);
tm().isOfy()
? tm().doTransactionless(
() -> {
final ForeignKeyIndex<R> fki =
ForeignKeyIndex.load(resourceClass, targetId, now);
if (fki == null) {
return new ResourceDoesNotExistException(resourceClass, targetId);
}
// Query for the first few linked domains, and if found, actually load them.
// The query is eventually consistent and so might be very stale, but the
// direct load will not be stale, just non-transactional. If we find at least
// one actual reference then we can reliably fail. If we don't find any,
// we can't trust the query and need to do the full mapreduce.
Iterable<VKey<DomainBase>> keys =
getLinkedDomainKeys(fki.getResourceKey(), now, FAILFAST_CHECK_COUNT);
VKey<R> resourceVKey = fki.getResourceKey();
Predicate<DomainBase> predicate =
domain -> getPotentialReferences.apply(domain).contains(resourceVKey);
return tm().loadByKeys(keys).values().stream().anyMatch(predicate)
? new ResourceToDeleteIsReferencedException()
: null;
});
VKey<R> resourceVKey = fki.getResourceKey();
Predicate<DomainBase> predicate =
domain -> getPotentialReferences.apply(domain).contains(resourceVKey);
return tm().loadByKeys(keys).values().stream().anyMatch(predicate)
? new ResourceToDeleteIsReferencedException()
: null;
})
: tm().transact(
() -> {
final ForeignKeyIndex<R> fki =
ForeignKeyIndex.load(resourceClass, targetId, now);
if (fki == null) {
return new ResourceDoesNotExistException(resourceClass, targetId);
}
return isLinked(fki.getResourceKey(), now)
? new ResourceToDeleteIsReferencedException()
: null;
});
if (failfastException != null) {
throw failfastException;
}
@@ -123,8 +145,7 @@ public final class ResourceFlowUtils {
}
public static <R extends EppResource & ForeignKeyedEppResource> R loadAndVerifyExistence(
Class<R> clazz, String targetId, DateTime now)
throws ResourceDoesNotExistException {
Class<R> clazz, String targetId, DateTime now) throws ResourceDoesNotExistException {
return verifyExistence(clazz, targetId, loadByForeignKey(clazz, targetId, now));
}
@@ -156,16 +177,16 @@ public final class ResourceFlowUtils {
}
/** Check that the given AuthInfo is either missing or else is valid for the given resource. */
public static void verifyOptionalAuthInfo(
Optional<AuthInfo> authInfo, ContactResource contact) throws EppException {
public static void verifyOptionalAuthInfo(Optional<AuthInfo> authInfo, ContactResource contact)
throws EppException {
if (authInfo.isPresent()) {
verifyAuthInfo(authInfo.get(), contact);
}
}
/** Check that the given AuthInfo is either missing or else is valid for the given resource. */
public static void verifyOptionalAuthInfo(
Optional<AuthInfo> authInfo, DomainBase domain) throws EppException {
public static void verifyOptionalAuthInfo(Optional<AuthInfo> authInfo, DomainBase domain)
throws EppException {
if (authInfo.isPresent()) {
verifyAuthInfo(authInfo.get(), domain);
}
@@ -229,7 +250,7 @@ public final class ResourceFlowUtils {
/** Check that the same values aren't being added and removed in an update command. */
public static void checkSameValuesNotAddedAndRemoved(
ImmutableSet<?> fieldsToAdd, ImmutableSet<?> fieldsToRemove)
throws AddRemoveSameValueException {
throws AddRemoveSameValueException {
if (!intersection(fieldsToAdd, fieldsToRemove).isEmpty()) {
throw new AddRemoveSameValueException();
}
@@ -33,6 +33,7 @@ import google.registry.flows.annotations.ReportingSpec;
import google.registry.flows.exceptions.ResourceAlreadyExistsForThisClientException;
import google.registry.flows.exceptions.ResourceCreateContentionException;
import google.registry.model.contact.ContactCommand.Create;
import google.registry.model.contact.ContactHistory;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppinput.ResourceCommand;
@@ -61,7 +62,7 @@ public final class ContactCreateFlow implements TransactionalFlow {
@Inject ExtensionManager extensionManager;
@Inject @ClientId String clientId;
@Inject @TargetId String targetId;
@Inject HistoryEntry.Builder historyBuilder;
@Inject ContactHistory.Builder historyBuilder;
@Inject EppResponse.Builder responseBuilder;
@Inject @Config("contactAndHostRoidSuffix") String roidSuffix;
@Inject ContactCreateFlow() {}
@@ -93,12 +94,12 @@ public final class ContactCreateFlow implements TransactionalFlow {
historyBuilder
.setType(HistoryEntry.Type.CONTACT_CREATE)
.setModificationTime(now)
.setXmlBytes(null) // We don't want to store contact details in the history entry.
.setParent(Key.create(newContact));
.setXmlBytes(null) // We don't want to store contact details in the history entry.
.setContact(newContact);
tm().insertAll(
ImmutableSet.of(
newContact,
historyBuilder.build().toChildHistoryEntity(),
historyBuilder.build(),
ForeignKeyIndex.create(newContact, newContact.getDeletionTime()),
EppResourceIndex.create(Key.create(newContact))));
return responseBuilder
@@ -15,16 +15,19 @@
package google.registry.flows.contact;
import static google.registry.flows.FlowUtils.validateClientIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.failfastForAsyncDelete;
import static google.registry.flows.ResourceFlowUtils.checkLinkedDomains;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
import static google.registry.flows.ResourceFlowUtils.verifyNoDisallowedStatuses;
import static google.registry.flows.ResourceFlowUtils.verifyOptionalAuthInfo;
import static google.registry.flows.ResourceFlowUtils.verifyResourceOwnership;
import static google.registry.model.ResourceTransferUtils.denyPendingTransfer;
import static google.registry.model.ResourceTransferUtils.handlePendingTransferOnDelete;
import static google.registry.model.eppoutput.Result.Code.SUCCESS;
import static google.registry.model.eppoutput.Result.Code.SUCCESS_WITH_ACTION_PENDING;
import static google.registry.model.transfer.TransferStatus.SERVER_CANCELLED;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.batch.AsyncTaskEnqueuer;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
@@ -33,6 +36,7 @@ import google.registry.flows.FlowModule.Superuser;
import google.registry.flows.FlowModule.TargetId;
import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.model.contact.ContactHistory;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.metadata.MetadataExtension;
@@ -40,7 +44,8 @@ import google.registry.model.eppcommon.AuthInfo;
import google.registry.model.eppcommon.StatusValue;
import google.registry.model.eppcommon.Trid;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.eppoutput.Result.Code;
import google.registry.model.reporting.HistoryEntry.Type;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import java.util.Optional;
import javax.inject.Inject;
@@ -63,10 +68,11 @@ import org.joda.time.DateTime;
@ReportingSpec(ActivityReportField.CONTACT_DELETE)
public final class ContactDeleteFlow implements TransactionalFlow {
private static final ImmutableSet<StatusValue> DISALLOWED_STATUSES = ImmutableSet.of(
StatusValue.CLIENT_DELETE_PROHIBITED,
StatusValue.PENDING_DELETE,
StatusValue.SERVER_DELETE_PROHIBITED);
private static final ImmutableSet<StatusValue> DISALLOWED_STATUSES =
ImmutableSet.of(
StatusValue.CLIENT_DELETE_PROHIBITED,
StatusValue.PENDING_DELETE,
StatusValue.SERVER_DELETE_PROHIBITED);
@Inject ExtensionManager extensionManager;
@Inject @ClientId String clientId;
@@ -74,10 +80,12 @@ public final class ContactDeleteFlow implements TransactionalFlow {
@Inject Trid trid;
@Inject @Superuser boolean isSuperuser;
@Inject Optional<AuthInfo> authInfo;
@Inject HistoryEntry.Builder historyBuilder;
@Inject ContactHistory.Builder historyBuilder;
@Inject AsyncTaskEnqueuer asyncTaskEnqueuer;
@Inject EppResponse.Builder responseBuilder;
@Inject ContactDeleteFlow() {}
@Inject
ContactDeleteFlow() {}
@Override
public final EppResponse run() throws EppException {
@@ -85,23 +93,45 @@ public final class ContactDeleteFlow implements TransactionalFlow {
extensionManager.validate();
validateClientIsLoggedIn(clientId);
DateTime now = tm().getTransactionTime();
failfastForAsyncDelete(targetId, now, ContactResource.class, DomainBase::getReferencedContacts);
checkLinkedDomains(targetId, now, ContactResource.class, DomainBase::getReferencedContacts);
ContactResource existingContact = loadAndVerifyExistence(ContactResource.class, targetId, now);
verifyNoDisallowedStatuses(existingContact, DISALLOWED_STATUSES);
verifyOptionalAuthInfo(authInfo, existingContact);
if (!isSuperuser) {
verifyResourceOwnership(clientId, existingContact);
}
asyncTaskEnqueuer.enqueueAsyncDelete(
existingContact, tm().getTransactionTime(), clientId, trid, isSuperuser);
ContactResource newContact =
existingContact.asBuilder().addStatusValue(StatusValue.PENDING_DELETE).build();
historyBuilder
.setType(HistoryEntry.Type.CONTACT_PENDING_DELETE)
.setModificationTime(now)
.setParent(Key.create(existingContact));
tm().insert(historyBuilder.build().toChildHistoryEntity());
Type historyEntryType;
Code resultCode;
ContactResource newContact;
if (tm().isOfy()) {
asyncTaskEnqueuer.enqueueAsyncDelete(
existingContact, tm().getTransactionTime(), clientId, trid, isSuperuser);
newContact = existingContact.asBuilder().addStatusValue(StatusValue.PENDING_DELETE).build();
historyEntryType = Type.CONTACT_PENDING_DELETE;
resultCode = SUCCESS_WITH_ACTION_PENDING;
} else {
// Handle pending transfers on contact deletion.
newContact =
existingContact.getStatusValues().contains(StatusValue.PENDING_TRANSFER)
? denyPendingTransfer(existingContact, SERVER_CANCELLED, now, clientId)
: existingContact;
// Wipe out PII on contact deletion.
newContact =
newContact.asBuilder().wipeOut().setStatusValues(null).setDeletionTime(now).build();
historyEntryType = Type.CONTACT_DELETE;
resultCode = SUCCESS;
}
ContactHistory contactHistory =
historyBuilder
.setType(historyEntryType)
.setModificationTime(now)
.setContact(newContact)
.build();
if (!tm().isOfy()) {
handlePendingTransferOnDelete(existingContact, newContact, now, contactHistory);
}
tm().insert(contactHistory);
tm().update(newContact);
return responseBuilder.setResultFromCode(SUCCESS_WITH_ACTION_PENDING).build();
return responseBuilder.setResultFromCode(resultCode).build();
}
}
@@ -20,19 +20,21 @@ import com.google.common.base.CharMatcher;
import com.google.common.base.Preconditions;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Sets;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.EppException.ParameterValuePolicyErrorException;
import google.registry.flows.EppException.ParameterValueSyntaxErrorException;
import google.registry.model.contact.ContactAddress;
import google.registry.model.contact.ContactHistory;
import google.registry.model.contact.ContactResource;
import google.registry.model.contact.PostalInfo;
import google.registry.model.poll.PendingActionNotificationResponse.ContactPendingActionNotificationResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.transfer.TransferData;
import google.registry.model.transfer.TransferResponse.ContactTransferResponse;
import java.util.Set;
import javax.annotation.Nullable;
import org.joda.time.DateTime;
/** Static utility functions for contact flows. */
public class ContactFlowUtils {
@@ -66,31 +68,35 @@ public class ContactFlowUtils {
/** Create a poll message for the gaining client in a transfer. */
static PollMessage createGainingTransferPollMessage(
String targetId, TransferData transferData, HistoryEntry historyEntry) {
String targetId,
TransferData transferData,
DateTime now,
Key<ContactHistory> contactHistoryKey) {
return new PollMessage.OneTime.Builder()
.setClientId(transferData.getGainingClientId())
.setEventTime(transferData.getPendingTransferExpirationTime())
.setMsg(transferData.getTransferStatus().getMessage())
.setResponseData(ImmutableList.of(
createTransferResponse(targetId, transferData),
ContactPendingActionNotificationResponse.create(
targetId,
transferData.getTransferStatus().isApproved(),
transferData.getTransferRequestTrid(),
historyEntry.getModificationTime())))
.setParent(historyEntry)
.setResponseData(
ImmutableList.of(
createTransferResponse(targetId, transferData),
ContactPendingActionNotificationResponse.create(
targetId,
transferData.getTransferStatus().isApproved(),
transferData.getTransferRequestTrid(),
now)))
.setParentKey(contactHistoryKey)
.build();
}
/** Create a poll message for the losing client in a transfer. */
static PollMessage createLosingTransferPollMessage(
String targetId, TransferData transferData, HistoryEntry historyEntry) {
String targetId, TransferData transferData, Key<ContactHistory> contactHistoryKey) {
return new PollMessage.OneTime.Builder()
.setClientId(transferData.getLosingClientId())
.setEventTime(transferData.getPendingTransferExpirationTime())
.setMsg(transferData.getTransferStatus().getMessage())
.setResponseData(ImmutableList.of(createTransferResponse(targetId, transferData)))
.setParent(historyEntry)
.setParentKey(contactHistoryKey)
.build();
}
@@ -32,6 +32,7 @@ import google.registry.flows.FlowModule.ClientId;
import google.registry.flows.FlowModule.TargetId;
import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.model.contact.ContactHistory;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.AuthInfo;
@@ -66,7 +67,7 @@ public final class ContactTransferApproveFlow implements TransactionalFlow {
@Inject @ClientId String clientId;
@Inject @TargetId String targetId;
@Inject Optional<AuthInfo> authInfo;
@Inject HistoryEntry.Builder historyBuilder;
@Inject ContactHistory.Builder historyBuilder;
@Inject EppResponse.Builder responseBuilder;
@Inject ContactTransferApproveFlow() {}
@@ -86,15 +87,17 @@ public final class ContactTransferApproveFlow implements TransactionalFlow {
verifyResourceOwnership(clientId, existingContact);
ContactResource newContact =
approvePendingTransfer(existingContact, TransferStatus.CLIENT_APPROVED, now);
HistoryEntry historyEntry = historyBuilder
.setType(HistoryEntry.Type.CONTACT_TRANSFER_APPROVE)
.setModificationTime(now)
.setParent(Key.create(existingContact))
.build();
ContactHistory contactHistory =
historyBuilder
.setType(HistoryEntry.Type.CONTACT_TRANSFER_APPROVE)
.setModificationTime(now)
.setContact(newContact)
.build();
// Create a poll message for the gaining client.
PollMessage gainingPollMessage =
createGainingTransferPollMessage(targetId, newContact.getTransferData(), historyEntry);
tm().insertAll(ImmutableSet.of(historyEntry.toChildHistoryEntity(), gainingPollMessage));
createGainingTransferPollMessage(
targetId, newContact.getTransferData(), now, Key.create(contactHistory));
tm().insertAll(ImmutableSet.of(contactHistory, gainingPollMessage));
tm().update(newContact);
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
@@ -32,6 +32,7 @@ import google.registry.flows.FlowModule.ClientId;
import google.registry.flows.FlowModule.TargetId;
import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.model.contact.ContactHistory;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.AuthInfo;
@@ -66,7 +67,7 @@ public final class ContactTransferCancelFlow implements TransactionalFlow {
@Inject Optional<AuthInfo> authInfo;
@Inject @ClientId String clientId;
@Inject @TargetId String targetId;
@Inject HistoryEntry.Builder historyBuilder;
@Inject ContactHistory.Builder historyBuilder;
@Inject EppResponse.Builder responseBuilder;
@Inject ContactTransferCancelFlow() {}
@@ -82,15 +83,17 @@ public final class ContactTransferCancelFlow implements TransactionalFlow {
verifyTransferInitiator(clientId, existingContact);
ContactResource newContact =
denyPendingTransfer(existingContact, TransferStatus.CLIENT_CANCELLED, now, clientId);
HistoryEntry historyEntry = historyBuilder
.setType(HistoryEntry.Type.CONTACT_TRANSFER_CANCEL)
.setModificationTime(now)
.setParent(Key.create(existingContact))
.build();
ContactHistory contactHistory =
historyBuilder
.setType(HistoryEntry.Type.CONTACT_TRANSFER_CANCEL)
.setModificationTime(now)
.setContact(newContact)
.build();
// Create a poll message for the losing client.
PollMessage losingPollMessage =
createLosingTransferPollMessage(targetId, newContact.getTransferData(), historyEntry);
tm().insertAll(ImmutableSet.of(historyEntry.toChildHistoryEntity(), losingPollMessage));
createLosingTransferPollMessage(
targetId, newContact.getTransferData(), Key.create(contactHistory));
tm().insertAll(ImmutableSet.of(contactHistory, losingPollMessage));
tm().update(newContact);
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
@@ -32,6 +32,7 @@ import google.registry.flows.FlowModule.ClientId;
import google.registry.flows.FlowModule.TargetId;
import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.model.contact.ContactHistory;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.AuthInfo;
@@ -64,7 +65,7 @@ public final class ContactTransferRejectFlow implements TransactionalFlow {
@Inject Optional<AuthInfo> authInfo;
@Inject @ClientId String clientId;
@Inject @TargetId String targetId;
@Inject HistoryEntry.Builder historyBuilder;
@Inject ContactHistory.Builder historyBuilder;
@Inject EppResponse.Builder responseBuilder;
@Inject ContactTransferRejectFlow() {}
@@ -80,14 +81,16 @@ public final class ContactTransferRejectFlow implements TransactionalFlow {
verifyResourceOwnership(clientId, existingContact);
ContactResource newContact =
denyPendingTransfer(existingContact, TransferStatus.CLIENT_REJECTED, now, clientId);
HistoryEntry historyEntry = historyBuilder
.setType(HistoryEntry.Type.CONTACT_TRANSFER_REJECT)
.setModificationTime(now)
.setParent(Key.create(existingContact))
.build();
ContactHistory contactHistory =
historyBuilder
.setType(HistoryEntry.Type.CONTACT_TRANSFER_REJECT)
.setModificationTime(now)
.setContact(newContact)
.build();
PollMessage gainingPollMessage =
createGainingTransferPollMessage(targetId, newContact.getTransferData(), historyEntry);
tm().insertAll(ImmutableSet.of(historyEntry.toChildHistoryEntity(), gainingPollMessage));
createGainingTransferPollMessage(
targetId, newContact.getTransferData(), now, Key.create(contactHistory));
tm().insertAll(ImmutableSet.of(contactHistory, gainingPollMessage));
tm().update(newContact);
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
@@ -14,6 +14,7 @@
package google.registry.flows.contact;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.validateClientIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
import static google.registry.flows.ResourceFlowUtils.verifyAuthInfo;
@@ -36,6 +37,7 @@ import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.flows.exceptions.AlreadyPendingTransferException;
import google.registry.flows.exceptions.ObjectAlreadySponsoredException;
import google.registry.model.contact.ContactHistory;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.AuthInfo;
@@ -81,7 +83,8 @@ public final class ContactTransferRequestFlow implements TransactionalFlow {
@Inject @ClientId String gainingClientId;
@Inject @TargetId String targetId;
@Inject @Config("contactAutomaticTransferLength") Duration automaticTransferLength;
@Inject HistoryEntry.Builder historyBuilder;
@Inject ContactHistory.Builder historyBuilder;
@Inject Trid trid;
@Inject EppResponse.Builder responseBuilder;
@Inject ContactTransferRequestFlow() {}
@@ -105,11 +108,7 @@ public final class ContactTransferRequestFlow implements TransactionalFlow {
throw new ObjectAlreadySponsoredException();
}
verifyNoDisallowedStatuses(existingContact, DISALLOWED_STATUSES);
HistoryEntry historyEntry = historyBuilder
.setType(HistoryEntry.Type.CONTACT_TRANSFER_REQUEST)
.setModificationTime(now)
.setParent(Key.create(existingContact))
.build();
DateTime transferExpirationTime = now.plus(automaticTransferLength);
ContactTransferData serverApproveTransferData =
new ContactTransferData.Builder()
@@ -120,12 +119,18 @@ public final class ContactTransferRequestFlow implements TransactionalFlow {
.setPendingTransferExpirationTime(transferExpirationTime)
.setTransferStatus(TransferStatus.SERVER_APPROVED)
.build();
Key<ContactHistory> contactHistoryKey = createHistoryKey(existingContact, ContactHistory.class);
historyBuilder
.setId(contactHistoryKey.getId())
.setType(HistoryEntry.Type.CONTACT_TRANSFER_REQUEST)
.setModificationTime(now);
// If the transfer is server approved, this message will be sent to the losing registrar. */
PollMessage serverApproveLosingPollMessage =
createLosingTransferPollMessage(targetId, serverApproveTransferData, historyEntry);
createLosingTransferPollMessage(targetId, serverApproveTransferData, contactHistoryKey);
// If the transfer is server approved, this message will be sent to the gaining registrar. */
PollMessage serverApproveGainingPollMessage =
createGainingTransferPollMessage(targetId, serverApproveTransferData, historyEntry);
createGainingTransferPollMessage(
targetId, serverApproveTransferData, now, contactHistoryKey);
ContactTransferData pendingTransferData =
serverApproveTransferData
.asBuilder()
@@ -137,8 +142,9 @@ public final class ContactTransferRequestFlow implements TransactionalFlow {
.build();
// When a transfer is requested, a poll message is created to notify the losing registrar.
PollMessage requestPollMessage =
createLosingTransferPollMessage(targetId, pendingTransferData, historyEntry).asBuilder()
.setEventTime(now) // Unlike the serverApprove messages, this applies immediately.
createLosingTransferPollMessage(targetId, pendingTransferData, contactHistoryKey)
.asBuilder()
.setEventTime(now) // Unlike the serverApprove messages, this applies immediately.
.build();
ContactResource newContact = existingContact.asBuilder()
.setTransferData(pendingTransferData)
@@ -147,7 +153,7 @@ public final class ContactTransferRequestFlow implements TransactionalFlow {
tm().update(newContact);
tm().insertAll(
ImmutableSet.of(
historyEntry.toChildHistoryEntity(),
historyBuilder.setContact(newContact).build(),
requestPollMessage,
serverApproveGainingPollMessage,
serverApproveLosingPollMessage));
@@ -27,7 +27,6 @@ import static google.registry.flows.contact.ContactFlowUtils.validateContactAgai
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.ClientId;
@@ -38,6 +37,7 @@ import google.registry.flows.annotations.ReportingSpec;
import google.registry.flows.exceptions.ResourceHasClientUpdateProhibitedException;
import google.registry.model.contact.ContactCommand.Update;
import google.registry.model.contact.ContactCommand.Update.Change;
import google.registry.model.contact.ContactHistory;
import google.registry.model.contact.ContactResource;
import google.registry.model.contact.PostalInfo;
import google.registry.model.domain.metadata.MetadataExtension;
@@ -82,7 +82,7 @@ public final class ContactUpdateFlow implements TransactionalFlow {
@Inject @ClientId String clientId;
@Inject @TargetId String targetId;
@Inject @Superuser boolean isSuperuser;
@Inject HistoryEntry.Builder historyBuilder;
@Inject ContactHistory.Builder historyBuilder;
@Inject EppResponse.Builder responseBuilder;
@Inject ContactUpdateFlow() {}
@@ -102,11 +102,6 @@ public final class ContactUpdateFlow implements TransactionalFlow {
verifyAllStatusesAreClientSettable(union(statusesToAdd, statusToRemove));
}
verifyNoDisallowedStatuses(existingContact, DISALLOWED_STATUSES);
historyBuilder
.setType(HistoryEntry.Type.CONTACT_UPDATE)
.setModificationTime(now)
.setXmlBytes(null) // We don't want to store contact details in the history entry.
.setParent(Key.create(existingContact));
checkSameValuesNotAddedAndRemoved(statusesToAdd, statusToRemove);
ContactResource.Builder builder = existingContact.asBuilder();
Change change = command.getInnerChange();
@@ -150,7 +145,12 @@ public final class ContactUpdateFlow implements TransactionalFlow {
}
validateAsciiPostalInfo(newContact.getInternationalizedPostalInfo());
validateContactAgainstPolicy(newContact);
tm().insert(historyBuilder.build().toChildHistoryEntity());
historyBuilder
.setType(HistoryEntry.Type.CONTACT_UPDATE)
.setModificationTime(now)
.setXmlBytes(null) // We don't want to store contact details in the history entry.
.setContact(newContact);
tm().insert(historyBuilder.build());
tm().update(newContact);
return responseBuilder.build();
}
@@ -45,7 +45,7 @@ import google.registry.model.eppinput.ResourceCommand;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tmch.ClaimsListDualDatabaseDao;
import google.registry.model.tmch.ClaimsListDao;
import google.registry.util.Clock;
import java.util.HashSet;
import java.util.Optional;
@@ -104,8 +104,7 @@ public final class DomainClaimsCheckFlow implements Flow {
verifyClaimsPeriodNotEnded(registry, now);
}
}
Optional<String> claimKey =
ClaimsListDualDatabaseDao.get().getClaimKey(parsedDomain.parts().get(0));
Optional<String> claimKey = ClaimsListDao.get().getClaimKey(parsedDomain.parts().get(0));
launchChecksBuilder.add(
LaunchCheck.create(
LaunchCheckName.create(claimKey.isPresent(), domainName), claimKey.orElse(null)));
@@ -97,7 +97,6 @@ import google.registry.model.eppinput.EppInput;
import google.registry.model.eppinput.ResourceCommand;
import google.registry.model.eppoutput.CreateData.DomainCreateData;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.host.HostResource;
import google.registry.model.index.EppResourceIndex;
import google.registry.model.index.ForeignKeyIndex;
import google.registry.model.ofy.ObjectifyService;
@@ -107,11 +106,11 @@ import google.registry.model.poll.PollMessage.Autorenew;
import google.registry.model.registry.Registry;
import google.registry.model.registry.Registry.TldState;
import google.registry.model.registry.Registry.TldType;
import google.registry.model.registry.label.ReservationType;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.persistence.VKey;
import google.registry.tmch.LordnTaskUtils;
import java.util.Optional;
import javax.inject.Inject;
@@ -302,10 +301,14 @@ public class DomainCreateFlow implements TransactionalFlow {
validateFeeChallenge(targetId, now, feeCreate, feesAndCredits);
Optional<SecDnsCreateExtension> secDnsCreate =
validateSecDnsExtension(eppInput.getSingleExtension(SecDnsCreateExtension.class));
String repoId = createDomainRepoId(ObjectifyService.allocateId(), registry.getTldStr());
DateTime registrationExpirationTime = leapSafeAddYears(now, years);
DomainHistory domainHistory =
buildHistoryEntry(repoId, registry, now, period, registry.getAddGracePeriodLength());
String repoId = createDomainRepoId(ObjectifyService.allocateId(), registry.getTldStr());
Key<DomainHistory> domainHistoryKey =
Key.create(
Key.create(DomainBase.class, repoId),
DomainHistory.class,
ObjectifyService.allocateId());
historyBuilder.setId(domainHistoryKey.getId());
// Bill for the create.
BillingEvent.OneTime createBillingEvent =
createOneTimeBillingEvent(
@@ -315,14 +318,14 @@ public class DomainCreateFlow implements TransactionalFlow {
isReserved(domainName, isSunriseCreate),
years,
feesAndCredits,
domainHistory,
domainHistoryKey,
allocationToken,
now);
// Create a new autorenew billing event and poll message starting at the expiration time.
BillingEvent.Recurring autorenewBillingEvent =
createAutorenewBillingEvent(domainHistory, registrationExpirationTime);
createAutorenewBillingEvent(domainHistoryKey, registrationExpirationTime);
PollMessage.Autorenew autorenewPollMessage =
createAutorenewPollMessage(domainHistory, registrationExpirationTime);
createAutorenewPollMessage(domainHistoryKey, registrationExpirationTime);
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
entitiesToSave.add(createBillingEvent, autorenewBillingEvent, autorenewPollMessage);
// Bill for EAP cost, if any.
@@ -330,14 +333,12 @@ public class DomainCreateFlow implements TransactionalFlow {
entitiesToSave.add(createEapBillingEvent(feesAndCredits, createBillingEvent));
}
ImmutableSet.Builder<StatusValue> statuses = new ImmutableSet.Builder<>();
if (getReservationTypes(domainName).contains(NAME_COLLISION)) {
statuses.add(SERVER_HOLD);
entitiesToSave.add(
createNameCollisionOneTimePollMessage(targetId, domainHistory, clientId, now));
}
DomainBase newDomain =
ImmutableSet<ReservationType> reservationTypes = getReservationTypes(domainName);
ImmutableSet<StatusValue> statuses =
reservationTypes.contains(NAME_COLLISION)
? ImmutableSet.of(SERVER_HOLD)
: ImmutableSet.of();
DomainBase domain =
new DomainBase.Builder()
.setCreationClientId(clientId)
.setPersistedCurrentSponsorClientId(clientId)
@@ -348,34 +349,38 @@ public class DomainCreateFlow implements TransactionalFlow {
.setAutorenewPollMessage(autorenewPollMessage.createVKey())
.setLaunchNotice(hasClaimsNotice ? launchCreate.get().getNotice() : null)
.setSmdId(signedMarkId)
.setDsData(secDnsCreate.isPresent() ? secDnsCreate.get().getDsData() : null)
.setDsData(secDnsCreate.map(SecDnsCreateExtension::getDsData).orElse(null))
.setRegistrant(command.getRegistrant())
.setAuthInfo(command.getAuthInfo())
.setDomainName(targetId)
.setNameservers(
(ImmutableSet<VKey<HostResource>>)
command.getNameservers().stream().collect(toImmutableSet()))
.setStatusValues(statuses.build())
.setNameservers(command.getNameservers().stream().collect(toImmutableSet()))
.setStatusValues(statuses)
.setContacts(command.getContacts())
.addGracePeriod(
GracePeriod.forBillingEvent(GracePeriodStatus.ADD, repoId, createBillingEvent))
.build();
DomainHistory domainHistory =
buildDomainHistory(domain, registry, now, period, registry.getAddGracePeriodLength());
if (reservationTypes.contains(NAME_COLLISION)) {
entitiesToSave.add(
createNameCollisionOneTimePollMessage(targetId, domainHistory, clientId, now));
}
entitiesToSave.add(
newDomain,
domainHistory.asBuilder().setDomainContent(newDomain).build(),
ForeignKeyIndex.create(newDomain, newDomain.getDeletionTime()),
EppResourceIndex.create(Key.create(newDomain)));
domain,
domainHistory,
ForeignKeyIndex.create(domain, domain.getDeletionTime()),
EppResourceIndex.create(Key.create(domain)));
if (allocationToken.isPresent()
&& TokenType.SINGLE_USE.equals(allocationToken.get().getTokenType())) {
entitiesToSave.add(
allocationTokenFlowUtils.redeemToken(allocationToken.get(), domainHistory.createVKey()));
}
enqueueTasks(newDomain, hasSignedMarks, hasClaimsNotice);
enqueueTasks(domain, hasSignedMarks, hasClaimsNotice);
EntityChanges entityChanges =
flowCustomLogic.beforeSave(
DomainCreateFlowCustomLogic.BeforeSaveParameters.newBuilder()
.setNewDomain(newDomain)
.setNewDomain(domain)
.setHistoryEntry(domainHistory)
.setEntityChanges(
EntityChanges.newBuilder().setSaves(entitiesToSave.build()).build())
@@ -480,8 +485,8 @@ public class DomainCreateFlow implements TransactionalFlow {
: null);
}
private DomainHistory buildHistoryEntry(
String repoId, Registry registry, DateTime now, Period period, Duration addGracePeriod) {
private DomainHistory buildDomainHistory(
DomainBase domain, Registry registry, DateTime now, Period period, Duration addGracePeriod) {
// We ignore prober transactions
if (registry.getTldType() == TldType.REAL) {
historyBuilder
@@ -497,7 +502,7 @@ public class DomainCreateFlow implements TransactionalFlow {
.setType(HistoryEntry.Type.DOMAIN_CREATE)
.setPeriod(period)
.setModificationTime(now)
.setParent(Key.create(DomainBase.class, repoId))
.setDomain(domain)
.build();
}
@@ -508,7 +513,7 @@ public class DomainCreateFlow implements TransactionalFlow {
boolean isReserved,
int years,
FeesAndCredits feesAndCredits,
DomainHistory domainHistory,
Key<DomainHistory> domainHistoryKey,
Optional<AllocationToken> allocationToken,
DateTime now) {
ImmutableSet.Builder<Flag> flagsBuilder = new ImmutableSet.Builder<>();
@@ -537,12 +542,12 @@ public class DomainCreateFlow implements TransactionalFlow {
? registry.getAnchorTenantAddGracePeriodLength()
: registry.getAddGracePeriodLength()))
.setFlags(flagsBuilder.build())
.setParent(domainHistory)
.setParent(domainHistoryKey)
.build();
}
private Recurring createAutorenewBillingEvent(
DomainHistory domainHistory, DateTime registrationExpirationTime) {
Key<DomainHistory> domainHistoryKey, DateTime registrationExpirationTime) {
return new BillingEvent.Recurring.Builder()
.setReason(Reason.RENEW)
.setFlags(ImmutableSet.of(Flag.AUTO_RENEW))
@@ -550,18 +555,18 @@ public class DomainCreateFlow implements TransactionalFlow {
.setClientId(clientId)
.setEventTime(registrationExpirationTime)
.setRecurrenceEndTime(END_OF_TIME)
.setParent(domainHistory)
.setParent(domainHistoryKey)
.build();
}
private Autorenew createAutorenewPollMessage(
HistoryEntry historyEntry, DateTime registrationExpirationTime) {
Key<DomainHistory> domainHistoryKey, DateTime registrationExpirationTime) {
return new PollMessage.Autorenew.Builder()
.setTargetId(targetId)
.setClientId(clientId)
.setEventTime(registrationExpirationTime)
.setMsg("Domain was auto-renewed.")
.setParent(historyEntry)
.setParentKey(domainHistoryKey)
.build();
}
@@ -16,6 +16,7 @@ package google.registry.flows.domain;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Strings.isNullOrEmpty;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.persistEntityChanges;
import static google.registry.flows.FlowUtils.validateClientIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -63,6 +64,7 @@ import google.registry.flows.custom.EntityChanges;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingEvent;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.fee.BaseFee.FeeType;
import google.registry.model.domain.fee.Credit;
@@ -125,7 +127,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
@Inject @ClientId String clientId;
@Inject @TargetId String targetId;
@Inject @Superuser boolean isSuperuser;
@Inject HistoryEntry.Builder historyBuilder;
@Inject DomainHistory.Builder historyBuilder;
@Inject DnsQueue dnsQueue;
@Inject Trid trid;
@Inject AsyncTaskEnqueuer asyncTaskEnqueuer;
@@ -177,8 +179,8 @@ public final class DomainDeleteFlow implements TransactionalFlow {
? Duration.ZERO
// By default, this should be 30 days of grace, and 5 days of pending delete.
: redemptionGracePeriodLength.plus(pendingDeleteLength);
HistoryEntry historyEntry =
buildHistoryEntry(existingDomain, registry, now, durationUntilDelete, inAddGracePeriod);
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
historyBuilder.setId(domainHistoryKey.getId());
DateTime deletionTime = now.plus(durationUntilDelete);
if (durationUntilDelete.equals(Duration.ZERO)) {
builder.setDeletionTime(now).setStatusValues(null);
@@ -210,7 +212,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
// it is synchronous).
if (durationUntilDelete.isLongerThan(Duration.ZERO) || isSuperuser) {
PollMessage.OneTime deletePollMessage =
createDeletePollMessage(existingDomain, historyEntry, deletionTime);
createDeletePollMessage(existingDomain, domainHistoryKey, deletionTime);
entitiesToSave.add(deletePollMessage);
builder.setDeletePollMessage(deletePollMessage.createVKey());
}
@@ -220,7 +222,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
if (durationUntilDelete.isLongerThan(Duration.ZERO)
&& !clientId.equals(existingDomain.getPersistedCurrentSponsorClientId())) {
entitiesToSave.add(
createImmediateDeletePollMessage(existingDomain, historyEntry, now, deletionTime));
createImmediateDeletePollMessage(existingDomain, domainHistoryKey, now, deletionTime));
}
// Cancel any grace periods that were still active, and set the expiration time accordingly.
@@ -229,7 +231,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
// No cancellation is written if the grace period was not for a billable event.
if (gracePeriod.hasBillingEvent()) {
entitiesToSave.add(
BillingEvent.Cancellation.forGracePeriod(gracePeriod, historyEntry, targetId));
BillingEvent.Cancellation.forGracePeriod(gracePeriod, now, domainHistoryKey, targetId));
if (gracePeriod.getOneTimeBillingEvent() != null) {
// Take the amount of amount of registration time being refunded off the expiration time.
// This can be either add grace periods or renew grace periods.
@@ -245,8 +247,10 @@ public final class DomainDeleteFlow implements TransactionalFlow {
builder.setRegistrationExpirationTime(newExpirationTime);
DomainBase newDomain = builder.build();
DomainHistory domainHistory =
buildDomainHistory(newDomain, registry, now, durationUntilDelete, inAddGracePeriod);
updateForeignKeyIndexDeletionTime(newDomain);
handlePendingTransferOnDelete(existingDomain, newDomain, now, historyEntry);
handlePendingTransferOnDelete(existingDomain, newDomain, now, domainHistory);
// Close the autorenew billing event and poll message. This may delete the poll message.
updateAutorenewRecurrenceEndTime(existingDomain, now);
// If there's a pending transfer, the gaining client's autorenew billing
@@ -254,14 +258,16 @@ public final class DomainDeleteFlow implements TransactionalFlow {
// ResourceDeleteFlow since it's listed in serverApproveEntities.
dnsQueue.addDomainRefreshTask(existingDomain.getDomainName());
entitiesToSave.add(newDomain, historyEntry);
EntityChanges entityChanges = flowCustomLogic.beforeSave(
BeforeSaveParameters.newBuilder()
.setExistingDomain(existingDomain)
.setNewDomain(newDomain)
.setHistoryEntry(historyEntry)
.setEntityChanges(EntityChanges.newBuilder().setSaves(entitiesToSave.build()).build())
.build());
entitiesToSave.add(newDomain, domainHistory);
EntityChanges entityChanges =
flowCustomLogic.beforeSave(
BeforeSaveParameters.newBuilder()
.setExistingDomain(existingDomain)
.setNewDomain(newDomain)
.setHistoryEntry(domainHistory)
.setEntityChanges(
EntityChanges.newBuilder().setSaves(entitiesToSave.build()).build())
.build());
BeforeResponseReturnData responseData =
flowCustomLogic.beforeResponse(
BeforeResponseParameters.newBuilder()
@@ -292,8 +298,8 @@ public final class DomainDeleteFlow implements TransactionalFlow {
}
}
private HistoryEntry buildHistoryEntry(
DomainBase existingResource,
private DomainHistory buildDomainHistory(
DomainBase domain,
Registry registry,
DateTime now,
Duration durationUntilDelete,
@@ -307,31 +313,30 @@ public final class DomainDeleteFlow implements TransactionalFlow {
registry.getRenewGracePeriodLength()));
ImmutableSet<DomainTransactionRecord> cancelledRecords =
createCancelingRecords(
existingResource,
domain,
now,
maxGracePeriod,
Sets.immutableEnumSet(Sets.union(ADD_FIELDS, RENEW_FIELDS)));
historyBuilder
.setDomainTransactionRecords(
union(
cancelledRecords,
DomainTransactionRecord.create(
existingResource.getTld(),
now.plus(durationUntilDelete),
inAddGracePeriod
? TransactionReportField.DELETED_DOMAINS_GRACE
: TransactionReportField.DELETED_DOMAINS_NOGRACE,
1)));
historyBuilder.setDomainTransactionRecords(
union(
cancelledRecords,
DomainTransactionRecord.create(
domain.getTld(),
now.plus(durationUntilDelete),
inAddGracePeriod
? TransactionReportField.DELETED_DOMAINS_GRACE
: TransactionReportField.DELETED_DOMAINS_NOGRACE,
1)));
}
return historyBuilder
.setType(HistoryEntry.Type.DOMAIN_DELETE)
.setModificationTime(now)
.setParent(Key.create(existingResource))
.setDomain(domain)
.build();
}
private PollMessage.OneTime createDeletePollMessage(
DomainBase existingDomain, HistoryEntry historyEntry, DateTime deletionTime) {
DomainBase existingDomain, Key<DomainHistory> domainHistoryKey, DateTime deletionTime) {
Optional<MetadataExtension> metadataExtension =
eppInput.getSingleExtension(MetadataExtension.class);
boolean hasMetadataMessage =
@@ -350,16 +355,19 @@ public final class DomainDeleteFlow implements TransactionalFlow {
ImmutableList.of(
DomainPendingActionNotificationResponse.create(
existingDomain.getDomainName(), true, trid, deletionTime)))
.setParent(historyEntry)
.setParentKey(domainHistoryKey)
.build();
}
private PollMessage.OneTime createImmediateDeletePollMessage(
DomainBase existingDomain, HistoryEntry historyEntry, DateTime now, DateTime deletionTime) {
DomainBase existingDomain,
Key<DomainHistory> domainHistoryKey,
DateTime now,
DateTime deletionTime) {
return new PollMessage.OneTime.Builder()
.setClientId(existingDomain.getPersistedCurrentSponsorClientId())
.setEventTime(now)
.setParent(historyEntry)
.setParentKey(domainHistoryKey)
.setMsg(
String.format(
"Domain %s was deleted by registry administrator with final deletion effective: %s",
@@ -25,11 +25,8 @@ import static com.google.common.collect.Iterables.any;
import static com.google.common.collect.Sets.difference;
import static com.google.common.collect.Sets.intersection;
import static com.google.common.collect.Sets.union;
import static google.registry.model.DatabaseMigrationUtils.getPrimaryDatabase;
import static google.registry.model.common.DatabaseTransitionSchedule.PrimaryDatabase.DATASTORE;
import static google.registry.model.common.DatabaseTransitionSchedule.TransitionId.REPLAYED_ENTITIES;
import static google.registry.model.domain.DomainBase.MAX_REGISTRATION_YEARS;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.model.registry.Registries.findTldForName;
import static google.registry.model.registry.Registries.getTlds;
import static google.registry.model.registry.Registry.TldState.GENERAL_AVAILABILITY;
@@ -129,7 +126,7 @@ import google.registry.model.registry.label.ReservedList;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.tmch.ClaimsListDualDatabaseDao;
import google.registry.model.tmch.ClaimsListDao;
import google.registry.persistence.VKey;
import google.registry.tldconfig.idn.IdnLabelValidator;
import google.registry.util.Idn;
@@ -993,8 +990,7 @@ public class DomainFlowUtils {
static void verifyClaimsNoticeIfAndOnlyIfNeeded(
InternetDomainName domainName, boolean hasSignedMarks, boolean hasClaimsNotice)
throws EppException {
boolean isInClaimsList =
ClaimsListDualDatabaseDao.get().getClaimKey(domainName.parts().get(0)).isPresent();
boolean isInClaimsList = ClaimsListDao.get().getClaimKey(domainName.parts().get(0)).isPresent();
if (hasClaimsNotice && !isInClaimsList) {
throw new UnexpectedClaimsNoticeException(domainName.toString());
}
@@ -1083,8 +1079,8 @@ public class DomainFlowUtils {
private static List<? extends HistoryEntry> findRecentHistoryEntries(
DomainBase domainBase, DateTime now, Duration maxSearchPeriod) {
if (getPrimaryDatabase(REPLAYED_ENTITIES).equals(DATASTORE)) {
return ofy()
if (tm().isOfy()) {
return auditedOfy()
.load()
.type(HistoryEntry.class)
.ancestor(domainBase)
@@ -14,6 +14,7 @@
package google.registry.flows.domain;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.persistEntityChanges;
import static google.registry.flows.FlowUtils.validateClientIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -33,6 +34,7 @@ import static google.registry.util.DateTimeUtils.leapSafeAddYears;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.EppException.ParameterValueRangeErrorException;
import google.registry.flows.ExtensionManager;
@@ -52,6 +54,7 @@ import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainCommand.Renew;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainRenewData;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.Period;
@@ -123,7 +126,7 @@ public final class DomainRenewFlow implements TransactionalFlow {
@Inject @ClientId String clientId;
@Inject @TargetId String targetId;
@Inject @Superuser boolean isSuperuser;
@Inject HistoryEntry.Builder historyBuilder;
@Inject DomainHistory.Builder historyBuilder;
@Inject EppResponse.Builder responseBuilder;
@Inject DomainRenewFlowCustomLogic flowCustomLogic;
@Inject DomainPricingLogic pricingLogic;
@@ -156,22 +159,23 @@ public final class DomainRenewFlow implements TransactionalFlow {
.setNow(now)
.setYears(years)
.build());
Registry registry = Registry.get(existingDomain.getTld());
HistoryEntry historyEntry = buildHistoryEntry(
existingDomain, now, command.getPeriod(), registry.getRenewGracePeriodLength());
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
historyBuilder.setId(domainHistoryKey.getId());
String tld = existingDomain.getTld();
// Bill for this explicit renew itself.
BillingEvent.OneTime explicitRenewEvent =
createRenewBillingEvent(tld, feesAndCredits.getTotalCost(), years, historyEntry, now);
createRenewBillingEvent(tld, feesAndCredits.getTotalCost(), years, domainHistoryKey, now);
// Create a new autorenew billing event and poll message starting at the new expiration time.
BillingEvent.Recurring newAutorenewEvent = newAutorenewBillingEvent(existingDomain)
.setEventTime(newExpirationTime)
.setParent(historyEntry)
.build();
PollMessage.Autorenew newAutorenewPollMessage = newAutorenewPollMessage(existingDomain)
.setEventTime(newExpirationTime)
.setParent(historyEntry)
.build();
BillingEvent.Recurring newAutorenewEvent =
newAutorenewBillingEvent(existingDomain)
.setEventTime(newExpirationTime)
.setParent(domainHistoryKey)
.build();
PollMessage.Autorenew newAutorenewPollMessage =
newAutorenewPollMessage(existingDomain)
.setEventTime(newExpirationTime)
.setParentKey(domainHistoryKey)
.build();
// End the old autorenew billing event and poll message now. This may delete the poll message.
updateAutorenewRecurrenceEndTime(existingDomain, now);
DomainBase newDomain =
@@ -186,6 +190,10 @@ public final class DomainRenewFlow implements TransactionalFlow {
GracePeriod.forBillingEvent(
GracePeriodStatus.RENEW, existingDomain.getRepoId(), explicitRenewEvent))
.build();
Registry registry = Registry.get(existingDomain.getTld());
DomainHistory domainHistory =
buildDomainHistory(
newDomain, now, command.getPeriod(), registry.getRenewGracePeriodLength());
EntityChanges entityChanges =
flowCustomLogic.beforeSave(
BeforeSaveParameters.newBuilder()
@@ -193,19 +201,18 @@ public final class DomainRenewFlow implements TransactionalFlow {
.setNewDomain(newDomain)
.setNow(now)
.setYears(years)
.setHistoryEntry(historyEntry)
.setHistoryEntry(domainHistory)
.setEntityChanges(
EntityChanges.newBuilder()
.setSaves(
ImmutableSet.of(
newDomain,
historyEntry,
domainHistory,
explicitRenewEvent,
newAutorenewEvent,
newAutorenewPollMessage))
.build())
.build());
persistEntityChanges(entityChanges);
BeforeResponseReturnData responseData =
flowCustomLogic.beforeResponse(
BeforeResponseParameters.newBuilder()
@@ -213,23 +220,24 @@ public final class DomainRenewFlow implements TransactionalFlow {
.setResData(DomainRenewData.create(targetId, newExpirationTime))
.setResponseExtensions(createResponseExtensions(feesAndCredits, feeRenew))
.build());
persistEntityChanges(entityChanges);
return responseBuilder
.setResData(responseData.resData())
.setExtensions(responseData.responseExtensions())
.build();
}
private HistoryEntry buildHistoryEntry(
DomainBase existingDomain, DateTime now, Period period, Duration renewGracePeriod) {
private DomainHistory buildDomainHistory(
DomainBase newDomain, DateTime now, Period period, Duration renewGracePeriod) {
return historyBuilder
.setType(HistoryEntry.Type.DOMAIN_RENEW)
.setPeriod(period)
.setModificationTime(now)
.setParent(existingDomain)
.setDomain(newDomain)
.setDomainTransactionRecords(
ImmutableSet.of(
DomainTransactionRecord.create(
existingDomain.getTld(),
newDomain.getTld(),
now.plus(renewGracePeriod),
TransactionReportField.netRenewsFieldFromYears(period.getValue()),
1)))
@@ -255,7 +263,7 @@ public final class DomainRenewFlow implements TransactionalFlow {
}
private OneTime createRenewBillingEvent(
String tld, Money renewCost, int years, HistoryEntry historyEntry, DateTime now) {
String tld, Money renewCost, int years, Key<DomainHistory> domainHistoryKey, DateTime now) {
return new BillingEvent.OneTime.Builder()
.setReason(Reason.RENEW)
.setTargetId(targetId)
@@ -264,7 +272,7 @@ public final class DomainRenewFlow implements TransactionalFlow {
.setCost(renewCost)
.setEventTime(now)
.setBillingTime(now.plus(Registry.get(tld).getRenewGracePeriodLength()))
.setParent(historyEntry)
.setParent(domainHistoryKey)
.build();
}
@@ -14,6 +14,7 @@
package google.registry.flows.domain;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.validateClientIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
import static google.registry.flows.ResourceFlowUtils.verifyOptionalAuthInfo;
@@ -49,6 +50,7 @@ import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainCommand.Update;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.fee.BaseFee.FeeType;
import google.registry.model.domain.fee.Fee;
import google.registry.model.domain.fee.FeeTransformResponseExtension;
@@ -117,7 +119,7 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
@Inject @ClientId String clientId;
@Inject @TargetId String targetId;
@Inject @Superuser boolean isSuperuser;
@Inject HistoryEntry.Builder historyBuilder;
@Inject DomainHistory.Builder historyBuilder;
@Inject DnsQueue dnsQueue;
@Inject EppResponse.Builder responseBuilder;
@Inject DomainPricingLogic pricingLogic;
@@ -142,7 +144,8 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
Optional<FeeUpdateCommandExtension> feeUpdate =
eppInput.getSingleExtension(FeeUpdateCommandExtension.class);
verifyRestoreAllowed(command, existingDomain, feeUpdate, feesAndCredits, now);
HistoryEntry historyEntry = buildHistoryEntry(existingDomain, now);
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
historyBuilder.setId(domainHistoryKey.getId());
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
DateTime newExpirationTime =
@@ -150,29 +153,31 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
// Restore the expiration time on the deleted domain, except if that's already passed, then add
// a year and bill for it immediately, with no grace period.
if (isExpired) {
entitiesToSave.add(createRenewBillingEvent(historyEntry, feesAndCredits.getRenewCost(), now));
entitiesToSave.add(
createRenewBillingEvent(domainHistoryKey, feesAndCredits.getRenewCost(), now));
}
// Always bill for the restore itself.
entitiesToSave.add(
createRestoreBillingEvent(historyEntry, feesAndCredits.getRestoreCost(), now));
createRestoreBillingEvent(domainHistoryKey, feesAndCredits.getRestoreCost(), now));
BillingEvent.Recurring autorenewEvent =
newAutorenewBillingEvent(existingDomain)
.setEventTime(newExpirationTime)
.setRecurrenceEndTime(END_OF_TIME)
.setParent(historyEntry)
.setParent(domainHistoryKey)
.build();
PollMessage.Autorenew autorenewPollMessage =
newAutorenewPollMessage(existingDomain)
.setEventTime(newExpirationTime)
.setAutorenewEndTime(END_OF_TIME)
.setParent(historyEntry)
.setParentKey(domainHistoryKey)
.build();
DomainBase newDomain =
performRestore(
existingDomain, newExpirationTime, autorenewEvent, autorenewPollMessage, now, clientId);
updateForeignKeyIndexDeletionTime(newDomain);
entitiesToSave.add(newDomain, historyEntry, autorenewEvent, autorenewPollMessage);
DomainHistory domainHistory = buildDomainHistory(newDomain, now);
entitiesToSave.add(newDomain, domainHistory, autorenewEvent, autorenewPollMessage);
tm().putAll(entitiesToSave.build());
tm().delete(existingDomain.getDeletePollMessage());
dnsQueue.addDomainRefreshTask(existingDomain.getDomainName());
@@ -181,15 +186,15 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
.build();
}
private HistoryEntry buildHistoryEntry(DomainBase existingDomain, DateTime now) {
private DomainHistory buildDomainHistory(DomainBase newDomain, DateTime now) {
return historyBuilder
.setType(HistoryEntry.Type.DOMAIN_RESTORE)
.setModificationTime(now)
.setParent(Key.create(existingDomain))
.setDomain(newDomain)
.setDomainTransactionRecords(
ImmutableSet.of(
DomainTransactionRecord.create(
existingDomain.getTld(), now, TransactionReportField.RESTORED_DOMAINS, 1)))
newDomain.getTld(), now, TransactionReportField.RESTORED_DOMAINS, 1)))
.build();
}
@@ -242,20 +247,19 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
}
private OneTime createRenewBillingEvent(
HistoryEntry historyEntry, Money renewCost, DateTime now) {
return prepareBillingEvent(historyEntry, renewCost, now)
.setReason(Reason.RENEW)
.build();
Key<DomainHistory> domainHistoryKey, Money renewCost, DateTime now) {
return prepareBillingEvent(domainHistoryKey, renewCost, now).setReason(Reason.RENEW).build();
}
private BillingEvent.OneTime createRestoreBillingEvent(
HistoryEntry historyEntry, Money restoreCost, DateTime now) {
return prepareBillingEvent(historyEntry, restoreCost, now)
Key<DomainHistory> domainHistoryKey, Money restoreCost, DateTime now) {
return prepareBillingEvent(domainHistoryKey, restoreCost, now)
.setReason(Reason.RESTORE)
.build();
}
private OneTime.Builder prepareBillingEvent(HistoryEntry historyEntry, Money cost, DateTime now) {
private OneTime.Builder prepareBillingEvent(
Key<DomainHistory> domainHistoryKey, Money cost, DateTime now) {
return new BillingEvent.OneTime.Builder()
.setTargetId(targetId)
.setClientId(clientId)
@@ -263,7 +267,7 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
.setBillingTime(now)
.setPeriodYears(1)
.setCost(cost)
.setParent(historyEntry);
.setParent(domainHistoryKey);
}
private static ImmutableList<FeeTransformResponseExtension> createResponseExtensions(
@@ -15,6 +15,7 @@
package google.registry.flows.domain;
import static com.google.common.collect.Iterables.getOnlyElement;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.validateClientIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.computeExDateForApprovalTime;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -48,6 +49,7 @@ import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.domain.rgp.GracePeriodStatus;
@@ -90,7 +92,7 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
@Inject @ClientId String clientId;
@Inject @TargetId String targetId;
@Inject @Superuser boolean isSuperuser;
@Inject HistoryEntry.Builder historyBuilder;
@Inject DomainHistory.Builder historyBuilder;
@Inject EppResponse.Builder responseBuilder;
@Inject DomainTransferApproveFlow() {}
@@ -114,10 +116,10 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
}
DomainTransferData transferData = existingDomain.getTransferData();
String gainingClientId = transferData.getGainingClientId();
Registry registry = Registry.get(existingDomain.getTld());
HistoryEntry historyEntry = buildHistoryEntry(existingDomain, registry, now, gainingClientId);
// Create a transfer billing event for 1 year, unless the superuser extension was used to set
// the transfer period to zero. There is not a transfer cost if the transfer period is zero.
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
historyBuilder.setId(domainHistoryKey.getId());
Optional<BillingEvent.OneTime> billingEvent =
(transferData.getTransferPeriod().getValue() == 0)
? Optional.empty()
@@ -130,10 +132,9 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
.setCost(getDomainRenewCost(targetId, transferData.getTransferRequestTime(), 1))
.setEventTime(now)
.setBillingTime(now.plus(Registry.get(tld).getTransferGracePeriodLength()))
.setParent(historyEntry)
.setParent(domainHistoryKey)
.build());
ImmutableList.Builder<ImmutableObject> entitiesToSave = new ImmutableList.Builder<>();
entitiesToSave.add(historyEntry);
// If we are within an autorenew grace period, cancel the autorenew billing event and don't
// increase the registration time, since the transfer subsumes the autorenew's extra year.
GracePeriod autorenewGrace =
@@ -146,7 +147,8 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
// still needs to be charged for the auto-renew.
if (billingEvent.isPresent()) {
entitiesToSave.add(
BillingEvent.Cancellation.forGracePeriod(autorenewGrace, historyEntry, targetId));
BillingEvent.Cancellation.forGracePeriod(
autorenewGrace, now, domainHistoryKey, targetId));
}
}
// Close the old autorenew event and poll message at the transfer time (aka now). This may end
@@ -155,24 +157,26 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
DateTime newExpirationTime =
computeExDateForApprovalTime(existingDomain, now, transferData.getTransferPeriod());
// Create a new autorenew event starting at the expiration time.
BillingEvent.Recurring autorenewEvent = new BillingEvent.Recurring.Builder()
.setReason(Reason.RENEW)
.setFlags(ImmutableSet.of(Flag.AUTO_RENEW))
.setTargetId(targetId)
.setClientId(gainingClientId)
.setEventTime(newExpirationTime)
.setRecurrenceEndTime(END_OF_TIME)
.setParent(historyEntry)
.build();
BillingEvent.Recurring autorenewEvent =
new BillingEvent.Recurring.Builder()
.setReason(Reason.RENEW)
.setFlags(ImmutableSet.of(Flag.AUTO_RENEW))
.setTargetId(targetId)
.setClientId(gainingClientId)
.setEventTime(newExpirationTime)
.setRecurrenceEndTime(END_OF_TIME)
.setParent(domainHistoryKey)
.build();
// Create a new autorenew poll message.
PollMessage.Autorenew gainingClientAutorenewPollMessage = new PollMessage.Autorenew.Builder()
.setTargetId(targetId)
.setClientId(gainingClientId)
.setEventTime(newExpirationTime)
.setAutorenewEndTime(END_OF_TIME)
.setMsg("Domain was auto-renewed.")
.setParent(historyEntry)
.build();
PollMessage.Autorenew gainingClientAutorenewPollMessage =
new PollMessage.Autorenew.Builder()
.setTargetId(targetId)
.setClientId(gainingClientId)
.setEventTime(newExpirationTime)
.setAutorenewEndTime(END_OF_TIME)
.setMsg("Domain was auto-renewed.")
.setParentKey(domainHistoryKey)
.build();
// Construct the post-transfer domain.
DomainBase partiallyApprovedDomain =
approvePendingTransfer(existingDomain, TransferStatus.CLIENT_APPROVED, now);
@@ -204,13 +208,19 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
.setLastEppUpdateTime(now)
.setLastEppUpdateClientId(clientId)
.build();
Registry registry = Registry.get(existingDomain.getTld());
DomainHistory domainHistory = buildDomainHistory(newDomain, registry, now, gainingClientId);
// Create a poll message for the gaining client.
PollMessage gainingClientPollMessage =
createGainingTransferPollMessage(
targetId, newDomain.getTransferData(), newExpirationTime, historyEntry);
targetId, newDomain.getTransferData(), newExpirationTime, now, domainHistoryKey);
billingEvent.ifPresent(entitiesToSave::add);
entitiesToSave.add(
autorenewEvent, gainingClientPollMessage, gainingClientAutorenewPollMessage, newDomain);
autorenewEvent,
gainingClientPollMessage,
gainingClientAutorenewPollMessage,
newDomain,
domainHistory);
tm().putAll(entitiesToSave.build());
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
@@ -221,11 +231,11 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
.build();
}
private HistoryEntry buildHistoryEntry(
DomainBase existingDomain, Registry registry, DateTime now, String gainingClientId) {
private DomainHistory buildDomainHistory(
DomainBase newDomain, Registry registry, DateTime now, String gainingClientId) {
ImmutableSet<DomainTransactionRecord> cancelingRecords =
createCancelingRecords(
existingDomain,
newDomain,
now,
registry.getAutomaticTransferLength().plus(registry.getTransferGracePeriodLength()),
ImmutableSet.of(TRANSFER_SUCCESSFUL));
@@ -233,15 +243,15 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
.setType(HistoryEntry.Type.DOMAIN_TRANSFER_APPROVE)
.setModificationTime(now)
.setOtherClientId(gainingClientId)
.setParent(Key.create(existingDomain))
.setDomain(newDomain)
.setDomainTransactionRecords(
union(
cancelingRecords,
DomainTransactionRecord.create(
existingDomain.getTld(),
now.plus(registry.getTransferGracePeriodLength()),
TRANSFER_SUCCESSFUL,
1)))
newDomain.getTld(),
now.plus(registry.getTransferGracePeriodLength()),
TRANSFER_SUCCESSFUL,
1)))
.build();
}
}
@@ -14,6 +14,7 @@
package google.registry.flows.domain;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.validateClientIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
import static google.registry.flows.ResourceFlowUtils.verifyHasPendingTransfer;
@@ -39,6 +40,7 @@ import google.registry.flows.FlowModule.TargetId;
import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.AuthInfo;
import google.registry.model.eppoutput.EppResponse;
@@ -77,7 +79,7 @@ public final class DomainTransferCancelFlow implements TransactionalFlow {
@Inject @ClientId String clientId;
@Inject @TargetId String targetId;
@Inject @Superuser boolean isSuperuser;
@Inject HistoryEntry.Builder historyBuilder;
@Inject DomainHistory.Builder historyBuilder;
@Inject EppResponse.Builder responseBuilder;
@Inject DomainTransferCancelFlow() {}
@@ -95,14 +97,20 @@ public final class DomainTransferCancelFlow implements TransactionalFlow {
checkAllowedAccessToTld(clientId, existingDomain.getTld());
}
Registry registry = Registry.get(existingDomain.getTld());
HistoryEntry historyEntry = buildHistoryEntry(existingDomain, registry, now);
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
historyBuilder
.setId(domainHistoryKey.getId())
.setOtherClientId(existingDomain.getTransferData().getLosingClientId());
DomainBase newDomain =
denyPendingTransfer(existingDomain, TransferStatus.CLIENT_CANCELLED, now, clientId);
DomainHistory domainHistory = buildDomainHistory(newDomain, registry, now);
tm().putAll(
newDomain,
historyEntry,
domainHistory,
createLosingTransferPollMessage(
targetId, newDomain.getTransferData(), null, historyEntry));
targetId, newDomain.getTransferData(), null, domainHistoryKey));
// Reopen the autorenew event and poll message that we closed for the implicit transfer. This
// may recreate the autorenew poll message if it was deleted when the transfer request was made.
updateAutorenewRecurrenceEndTime(existingDomain, END_OF_TIME);
@@ -114,19 +122,17 @@ public final class DomainTransferCancelFlow implements TransactionalFlow {
.build();
}
private HistoryEntry buildHistoryEntry(
DomainBase existingDomain, Registry registry, DateTime now) {
private DomainHistory buildDomainHistory(DomainBase newDomain, Registry registry, DateTime now) {
ImmutableSet<DomainTransactionRecord> cancelingRecords =
createCancelingRecords(
existingDomain,
newDomain,
now,
registry.getAutomaticTransferLength().plus(registry.getTransferGracePeriodLength()),
ImmutableSet.of(TRANSFER_SUCCESSFUL));
return historyBuilder
.setType(HistoryEntry.Type.DOMAIN_TRANSFER_CANCEL)
.setOtherClientId(existingDomain.getTransferData().getLosingClientId())
.setModificationTime(now)
.setParent(Key.create(existingDomain))
.setDomain(newDomain)
.setDomainTransactionRecords(cancelingRecords)
.build();
}
@@ -14,6 +14,7 @@
package google.registry.flows.domain;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.validateClientIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
import static google.registry.flows.ResourceFlowUtils.verifyHasPendingTransfer;
@@ -41,6 +42,7 @@ import google.registry.flows.FlowModule.TargetId;
import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.AuthInfo;
import google.registry.model.eppoutput.EppResponse;
@@ -79,7 +81,7 @@ public final class DomainTransferRejectFlow implements TransactionalFlow {
@Inject @ClientId String clientId;
@Inject @TargetId String targetId;
@Inject @Superuser boolean isSuperuser;
@Inject HistoryEntry.Builder historyBuilder;
@Inject DomainHistory.Builder historyBuilder;
@Inject EppResponse.Builder responseBuilder;
@Inject DomainTransferRejectFlow() {}
@@ -91,7 +93,11 @@ public final class DomainTransferRejectFlow implements TransactionalFlow {
DateTime now = tm().getTransactionTime();
DomainBase existingDomain = loadAndVerifyExistence(DomainBase.class, targetId, now);
Registry registry = Registry.get(existingDomain.getTld());
HistoryEntry historyEntry = buildHistoryEntry(existingDomain, registry, now);
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
historyBuilder
.setId(domainHistoryKey.getId())
.setOtherClientId(existingDomain.getTransferData().getGainingClientId());
verifyOptionalAuthInfo(authInfo, existingDomain);
verifyHasPendingTransfer(existingDomain);
verifyResourceOwnership(clientId, existingDomain);
@@ -100,11 +106,12 @@ public final class DomainTransferRejectFlow implements TransactionalFlow {
}
DomainBase newDomain =
denyPendingTransfer(existingDomain, TransferStatus.CLIENT_REJECTED, now, clientId);
DomainHistory domainHistory = buildDomainHistory(newDomain, registry, now);
tm().putAll(
newDomain,
historyEntry,
domainHistory,
createGainingTransferPollMessage(
targetId, newDomain.getTransferData(), null, historyEntry));
targetId, newDomain.getTransferData(), null, now, domainHistoryKey));
// Reopen the autorenew event and poll message that we closed for the implicit transfer. This
// may end up recreating the poll message if it was deleted upon the transfer request.
updateAutorenewRecurrenceEndTime(existingDomain, END_OF_TIME);
@@ -116,24 +123,21 @@ public final class DomainTransferRejectFlow implements TransactionalFlow {
.build();
}
private HistoryEntry buildHistoryEntry(
DomainBase existingDomain, Registry registry, DateTime now) {
private DomainHistory buildDomainHistory(DomainBase newDomain, Registry registry, DateTime now) {
ImmutableSet<DomainTransactionRecord> cancelingRecords =
createCancelingRecords(
existingDomain,
newDomain,
now,
registry.getAutomaticTransferLength().plus(registry.getTransferGracePeriodLength()),
ImmutableSet.of(TRANSFER_SUCCESSFUL));
return historyBuilder
.setType(HistoryEntry.Type.DOMAIN_TRANSFER_REJECT)
.setModificationTime(now)
.setOtherClientId(existingDomain.getTransferData().getGainingClientId())
.setParent(Key.create(existingDomain))
.setDomainTransactionRecords(
union(
cancelingRecords,
DomainTransactionRecord.create(
existingDomain.getTld(), now, TRANSFER_NACKED, 1)))
DomainTransactionRecord.create(newDomain.getTld(), now, TRANSFER_NACKED, 1)))
.setDomain(newDomain)
.build();
}
}
@@ -14,6 +14,7 @@
package google.registry.flows.domain;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.validateClientIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.computeExDateForApprovalTime;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -51,6 +52,7 @@ import google.registry.flows.exceptions.TransferPeriodMustBeOneYearException;
import google.registry.flows.exceptions.TransferPeriodZeroAndFeeTransferExtensionException;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainCommand.Transfer;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.Period;
import google.registry.model.domain.fee.FeeTransferCommandExtension;
import google.registry.model.domain.fee.FeeTransformResponseExtension;
@@ -126,7 +128,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
@Inject @ClientId String gainingClientId;
@Inject @TargetId String targetId;
@Inject @Superuser boolean isSuperuser;
@Inject HistoryEntry.Builder historyBuilder;
@Inject DomainHistory.Builder historyBuilder;
@Inject Trid trid;
@Inject AsyncTaskEnqueuer asyncTaskEnqueuer;
@Inject EppResponse.Builder responseBuilder;
@@ -169,7 +171,10 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
if (feesAndCredits.isPresent()) {
validateFeeChallenge(targetId, now, feeTransfer, feesAndCredits.get());
}
HistoryEntry historyEntry = buildHistoryEntry(existingDomain, registry, now, period);
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
historyBuilder
.setId(domainHistoryKey.getId())
.setOtherClientId(existingDomain.getCurrentSponsorClientId());
DateTime automaticTransferTime =
superuserExtension.isPresent()
? now.plusDays(superuserExtension.get().getAutomaticTransferLength())
@@ -190,7 +195,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
createTransferServerApproveEntities(
automaticTransferTime,
serverApproveNewExpirationTime,
historyEntry,
domainHistoryKey,
existingDomain,
trid,
gainingClientId,
@@ -209,9 +214,12 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
serverApproveEntities,
period);
// Create a poll message to notify the losing registrar that a transfer was requested.
PollMessage requestPollMessage = createLosingTransferPollMessage(
targetId, pendingTransferData, serverApproveNewExpirationTime, historyEntry)
.asBuilder().setEventTime(now).build();
PollMessage requestPollMessage =
createLosingTransferPollMessage(
targetId, pendingTransferData, serverApproveNewExpirationTime, domainHistoryKey)
.asBuilder()
.setEventTime(now)
.build();
// End the old autorenew event and poll message at the implicit transfer time. This may delete
// the poll message if it has no events left. Note that if the automatic transfer succeeds, then
// cloneProjectedAtTime() will replace these old autorenew entities with the server approve ones
@@ -225,10 +233,12 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
.setLastEppUpdateTime(now)
.setLastEppUpdateClientId(gainingClientId)
.build();
DomainHistory domainHistory = buildDomainHistory(newDomain, registry, now, period);
asyncTaskEnqueuer.enqueueAsyncResave(newDomain, now, automaticTransferTime);
tm().putAll(
new ImmutableSet.Builder<>()
.add(newDomain, historyEntry, requestPollMessage)
.add(newDomain, domainHistory, requestPollMessage)
.addAll(serverApproveEntities)
.build());
return responseBuilder
@@ -302,14 +312,13 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
}
}
private HistoryEntry buildHistoryEntry(
DomainBase existingDomain, Registry registry, DateTime now, Period period) {
private DomainHistory buildDomainHistory(
DomainBase newDomain, Registry registry, DateTime now, Period period) {
return historyBuilder
.setType(HistoryEntry.Type.DOMAIN_TRANSFER_REQUEST)
.setOtherClientId(existingDomain.getCurrentSponsorClientId())
.setPeriod(period)
.setModificationTime(now)
.setParent(Key.create(existingDomain))
.setDomain(newDomain)
.setDomainTransactionRecords(
ImmutableSet.of(
DomainTransactionRecord.create(
@@ -20,10 +20,12 @@ import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.Period;
import google.registry.model.domain.rgp.GracePeriodStatus;
@@ -31,7 +33,6 @@ import google.registry.model.eppcommon.Trid;
import google.registry.model.poll.PendingActionNotificationResponse.DomainPendingActionNotificationResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.transfer.DomainTransferData;
import google.registry.model.transfer.TransferData;
import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
@@ -90,20 +91,21 @@ public final class DomainTransferUtils {
* Returns a set of entities created speculatively in anticipation of a server approval.
*
* <p>This set consists of:
*
* <ul>
* <li>The one-time billing event charging the gaining registrar for the transfer
* <li>A cancellation of an autorenew charge for the losing registrar, if the autorenew grace
* period will apply at transfer time
* <li>A new post-transfer autorenew billing event for the domain (and gaining registrar)
* <li>A new post-transfer autorenew poll message for the domain (and gaining registrar)
* <li>A poll message for the gaining registrar
* <li>A poll message for the losing registrar
* <li>The one-time billing event charging the gaining registrar for the transfer
* <li>A cancellation of an autorenew charge for the losing registrar, if the autorenew grace
* period will apply at transfer time
* <li>A new post-transfer autorenew billing event for the domain (and gaining registrar)
* <li>A new post-transfer autorenew poll message for the domain (and gaining registrar)
* <li>A poll message for the gaining registrar
* <li>A poll message for the losing registrar
* </ul>
*/
public static ImmutableSet<TransferServerApproveEntity> createTransferServerApproveEntities(
DateTime automaticTransferTime,
DateTime serverApproveNewExpirationTime,
HistoryEntry historyEntry,
Key<DomainHistory> domainHistoryKey,
DomainBase existingDomain,
Trid trid,
String gainingClientId,
@@ -123,32 +125,39 @@ public final class DomainTransferUtils {
.build();
Registry registry = Registry.get(existingDomain.getTld());
ImmutableSet.Builder<TransferServerApproveEntity> builder = new ImmutableSet.Builder<>();
if (transferCost.isPresent()) {
builder.add(
createTransferBillingEvent(
automaticTransferTime,
historyEntry,
targetId,
gainingClientId,
registry,
transferCost.get()));
}
transferCost.ifPresent(
cost ->
builder.add(
createTransferBillingEvent(
automaticTransferTime,
domainHistoryKey,
targetId,
gainingClientId,
registry,
cost)));
createOptionalAutorenewCancellation(
automaticTransferTime, historyEntry, targetId, existingDomain, transferCost)
automaticTransferTime, now, domainHistoryKey, targetId, existingDomain, transferCost)
.ifPresent(builder::add);
return builder
.add(
createGainingClientAutorenewEvent(
serverApproveNewExpirationTime, historyEntry, targetId, gainingClientId))
serverApproveNewExpirationTime, domainHistoryKey, targetId, gainingClientId))
.add(
createGainingClientAutorenewPollMessage(
serverApproveNewExpirationTime, historyEntry, targetId, gainingClientId))
serverApproveNewExpirationTime, domainHistoryKey, targetId, gainingClientId))
.add(
createGainingTransferPollMessage(
targetId, serverApproveTransferData, serverApproveNewExpirationTime, historyEntry))
targetId,
serverApproveTransferData,
serverApproveNewExpirationTime,
now,
domainHistoryKey))
.add(
createLosingTransferPollMessage(
targetId, serverApproveTransferData, serverApproveNewExpirationTime, historyEntry))
targetId,
serverApproveTransferData,
serverApproveNewExpirationTime,
domainHistoryKey))
.build();
}
@@ -157,19 +166,21 @@ public final class DomainTransferUtils {
String targetId,
TransferData transferData,
@Nullable DateTime extendedRegistrationExpirationTime,
HistoryEntry historyEntry) {
DateTime now,
Key<DomainHistory> domainHistoryKey) {
return new PollMessage.OneTime.Builder()
.setClientId(transferData.getGainingClientId())
.setEventTime(transferData.getPendingTransferExpirationTime())
.setMsg(transferData.getTransferStatus().getMessage())
.setResponseData(ImmutableList.of(
createTransferResponse(targetId, transferData, extendedRegistrationExpirationTime),
DomainPendingActionNotificationResponse.create(
targetId,
transferData.getTransferStatus().isApproved(),
transferData.getTransferRequestTrid(),
historyEntry.getModificationTime())))
.setParent(historyEntry)
.setResponseData(
ImmutableList.of(
createTransferResponse(targetId, transferData, extendedRegistrationExpirationTime),
DomainPendingActionNotificationResponse.create(
targetId,
transferData.getTransferStatus().isApproved(),
transferData.getTransferRequestTrid(),
now)))
.setParentKey(domainHistoryKey)
.build();
}
@@ -178,14 +189,15 @@ public final class DomainTransferUtils {
String targetId,
TransferData transferData,
@Nullable DateTime extendedRegistrationExpirationTime,
HistoryEntry historyEntry) {
Key<DomainHistory> domainHistoryKey) {
return new PollMessage.OneTime.Builder()
.setClientId(transferData.getLosingClientId())
.setEventTime(transferData.getPendingTransferExpirationTime())
.setMsg(transferData.getTransferStatus().getMessage())
.setResponseData(ImmutableList.of(
createTransferResponse(targetId, transferData, extendedRegistrationExpirationTime)))
.setParent(historyEntry)
.setResponseData(
ImmutableList.of(
createTransferResponse(targetId, transferData, extendedRegistrationExpirationTime)))
.setParentKey(domainHistoryKey)
.build();
}
@@ -207,7 +219,7 @@ public final class DomainTransferUtils {
private static PollMessage.Autorenew createGainingClientAutorenewPollMessage(
DateTime serverApproveNewExpirationTime,
HistoryEntry historyEntry,
Key<DomainHistory> domainHistoryKey,
String targetId,
String gainingClientId) {
return new PollMessage.Autorenew.Builder()
@@ -216,13 +228,13 @@ public final class DomainTransferUtils {
.setEventTime(serverApproveNewExpirationTime)
.setAutorenewEndTime(END_OF_TIME)
.setMsg("Domain was auto-renewed.")
.setParent(historyEntry)
.setParentKey(domainHistoryKey)
.build();
}
private static BillingEvent.Recurring createGainingClientAutorenewEvent(
DateTime serverApproveNewExpirationTime,
HistoryEntry historyEntry,
Key<DomainHistory> domainHistoryKey,
String targetId,
String gainingClientId) {
return new BillingEvent.Recurring.Builder()
@@ -232,7 +244,7 @@ public final class DomainTransferUtils {
.setClientId(gainingClientId)
.setEventTime(serverApproveNewExpirationTime)
.setRecurrenceEndTime(END_OF_TIME)
.setParent(historyEntry)
.setParent(domainHistoryKey)
.build();
}
@@ -254,7 +266,8 @@ public final class DomainTransferUtils {
*/
private static Optional<BillingEvent.Cancellation> createOptionalAutorenewCancellation(
DateTime automaticTransferTime,
HistoryEntry historyEntry,
DateTime now,
Key<DomainHistory> domainHistoryKey,
String targetId,
DomainBase existingDomain,
Optional<Money> transferCost) {
@@ -265,7 +278,8 @@ public final class DomainTransferUtils {
domainAtTransferTime.getGracePeriodsOfType(GracePeriodStatus.AUTO_RENEW), null);
if (autorenewGracePeriod != null && transferCost.isPresent()) {
return Optional.of(
BillingEvent.Cancellation.forGracePeriod(autorenewGracePeriod, historyEntry, targetId)
BillingEvent.Cancellation.forGracePeriod(
autorenewGracePeriod, now, domainHistoryKey, targetId)
.asBuilder()
.setEventTime(automaticTransferTime)
.build());
@@ -275,7 +289,7 @@ public final class DomainTransferUtils {
private static BillingEvent.OneTime createTransferBillingEvent(
DateTime automaticTransferTime,
HistoryEntry historyEntry,
Key<DomainHistory> domainHistoryKey,
String targetId,
String gainingClientId,
Registry registry,
@@ -288,7 +302,7 @@ public final class DomainTransferUtils {
.setPeriodYears(1)
.setEventTime(automaticTransferTime)
.setBillingTime(automaticTransferTime.plus(registry.getTransferGracePeriodLength()))
.setParent(historyEntry)
.setParent(domainHistoryKey)
.build();
}
@@ -43,7 +43,6 @@ import static google.registry.persistence.transaction.TransactionManagerFactory.
import com.google.common.collect.ImmutableSet;
import com.google.common.net.InternetDomainName;
import com.googlecode.objectify.Key;
import google.registry.dns.DnsQueue;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
@@ -65,6 +64,7 @@ import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainCommand.Update;
import google.registry.model.domain.DomainCommand.Update.AddRemove;
import google.registry.model.domain.DomainCommand.Update.Change;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.fee.FeeUpdateCommandExtension;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.domain.secdns.DelegationSignerData;
@@ -142,7 +142,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
@Inject @ClientId String clientId;
@Inject @TargetId String targetId;
@Inject @Superuser boolean isSuperuser;
@Inject HistoryEntry.Builder historyBuilder;
@Inject DomainHistory.Builder historyBuilder;
@Inject DnsQueue dnsQueue;
@Inject EppResponse.Builder responseBuilder;
@Inject DomainUpdateFlowCustomLogic flowCustomLogic;
@@ -165,19 +165,19 @@ public final class DomainUpdateFlow implements TransactionalFlow {
verifyUpdateAllowed(command, existingDomain, now);
flowCustomLogic.afterValidation(
AfterValidationParameters.newBuilder().setExistingDomain(existingDomain).build());
HistoryEntry historyEntry = buildHistoryEntry(existingDomain, now);
DomainBase newDomain = performUpdate(command, existingDomain, now);
DomainHistory domainHistory = buildDomainHistory(newDomain, now);
validateNewState(newDomain);
dnsQueue.addDomainRefreshTask(targetId);
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
entitiesToSave.add(newDomain, historyEntry);
entitiesToSave.add(newDomain, domainHistory);
Optional<BillingEvent.OneTime> statusUpdateBillingEvent =
createBillingEventForStatusUpdates(existingDomain, newDomain, historyEntry, now);
createBillingEventForStatusUpdates(existingDomain, newDomain, domainHistory, now);
statusUpdateBillingEvent.ifPresent(entitiesToSave::add);
EntityChanges entityChanges =
flowCustomLogic.beforeSave(
BeforeSaveParameters.newBuilder()
.setHistoryEntry(historyEntry)
.setHistoryEntry(domainHistory)
.setNewDomain(newDomain)
.setExistingDomain(existingDomain)
.setEntityChanges(
@@ -217,11 +217,11 @@ public final class DomainUpdateFlow implements TransactionalFlow {
tld, add.getNameserverFullyQualifiedHostNames());
}
private HistoryEntry buildHistoryEntry(DomainBase existingDomain, DateTime now) {
private DomainHistory buildDomainHistory(DomainBase newDomain, DateTime now) {
return historyBuilder
.setType(HistoryEntry.Type.DOMAIN_UPDATE)
.setModificationTime(now)
.setParent(Key.create(existingDomain))
.setDomain(newDomain)
.build();
}
@@ -45,6 +45,7 @@ import google.registry.model.eppinput.ResourceCommand;
import google.registry.model.eppoutput.CreateData.HostCreateData;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.host.HostCommand.Create;
import google.registry.model.host.HostHistory;
import google.registry.model.host.HostResource;
import google.registry.model.index.EppResourceIndex;
import google.registry.model.index.ForeignKeyIndex;
@@ -85,7 +86,7 @@ public final class HostCreateFlow implements TransactionalFlow {
@Inject ExtensionManager extensionManager;
@Inject @ClientId String clientId;
@Inject @TargetId String targetId;
@Inject HistoryEntry.Builder historyBuilder;
@Inject HostHistory.Builder historyBuilder;
@Inject DnsQueue dnsQueue;
@Inject EppResponse.Builder responseBuilder;
@@ -128,14 +129,11 @@ public final class HostCreateFlow implements TransactionalFlow {
.setRepoId(createRepoId(ObjectifyService.allocateId(), roidSuffix))
.setSuperordinateDomain(superordinateDomain.map(DomainBase::createVKey).orElse(null))
.build();
historyBuilder
.setType(HistoryEntry.Type.HOST_CREATE)
.setModificationTime(now)
.setParent(Key.create(newHost));
historyBuilder.setType(HistoryEntry.Type.HOST_CREATE).setModificationTime(now).setHost(newHost);
ImmutableSet<ImmutableObject> entitiesToSave =
ImmutableSet.of(
newHost,
historyBuilder.build().toChildHistoryEntity(),
historyBuilder.build(),
ForeignKeyIndex.create(newHost, newHost.getDeletionTime()),
EppResourceIndex.create(Key.create(newHost)));
if (superordinateDomain.isPresent()) {
@@ -15,17 +15,18 @@
package google.registry.flows.host;
import static google.registry.flows.FlowUtils.validateClientIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.failfastForAsyncDelete;
import static google.registry.flows.ResourceFlowUtils.checkLinkedDomains;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
import static google.registry.flows.ResourceFlowUtils.verifyNoDisallowedStatuses;
import static google.registry.flows.ResourceFlowUtils.verifyResourceOwnership;
import static google.registry.flows.host.HostFlowUtils.validateHostName;
import static google.registry.model.eppoutput.Result.Code.SUCCESS;
import static google.registry.model.eppoutput.Result.Code.SUCCESS_WITH_ACTION_PENDING;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.batch.AsyncTaskEnqueuer;
import google.registry.dns.DnsQueue;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.ClientId;
@@ -39,8 +40,11 @@ import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.StatusValue;
import google.registry.model.eppcommon.Trid;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.eppoutput.Result;
import google.registry.model.host.HostHistory;
import google.registry.model.host.HostResource;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.HistoryEntry.Type;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import javax.inject.Inject;
import org.joda.time.DateTime;
@@ -65,20 +69,25 @@ import org.joda.time.DateTime;
@ReportingSpec(ActivityReportField.HOST_DELETE)
public final class HostDeleteFlow implements TransactionalFlow {
private static final ImmutableSet<StatusValue> DISALLOWED_STATUSES = ImmutableSet.of(
StatusValue.CLIENT_DELETE_PROHIBITED,
StatusValue.PENDING_DELETE,
StatusValue.SERVER_DELETE_PROHIBITED);
private static final ImmutableSet<StatusValue> DISALLOWED_STATUSES =
ImmutableSet.of(
StatusValue.CLIENT_DELETE_PROHIBITED,
StatusValue.PENDING_DELETE,
StatusValue.SERVER_DELETE_PROHIBITED);
private static final DnsQueue dnsQueue = DnsQueue.create();
@Inject ExtensionManager extensionManager;
@Inject @ClientId String clientId;
@Inject @TargetId String targetId;
@Inject Trid trid;
@Inject @Superuser boolean isSuperuser;
@Inject HistoryEntry.Builder historyBuilder;
@Inject HostHistory.Builder historyBuilder;
@Inject AsyncTaskEnqueuer asyncTaskEnqueuer;
@Inject EppResponse.Builder responseBuilder;
@Inject HostDeleteFlow() {}
@Inject
HostDeleteFlow() {}
@Override
public final EppResponse run() throws EppException {
@@ -87,7 +96,7 @@ public final class HostDeleteFlow implements TransactionalFlow {
validateClientIsLoggedIn(clientId);
DateTime now = tm().getTransactionTime();
validateHostName(targetId);
failfastForAsyncDelete(targetId, now, HostResource.class, DomainBase::getNameservers);
checkLinkedDomains(targetId, now, HostResource.class, DomainBase::getNameservers);
HostResource existingHost = loadAndVerifyExistence(HostResource.class, targetId, now);
verifyNoDisallowedStatuses(existingHost, DISALLOWED_STATUSES);
if (!isSuperuser) {
@@ -99,16 +108,31 @@ public final class HostDeleteFlow implements TransactionalFlow {
: existingHost;
verifyResourceOwnership(clientId, owningResource);
}
asyncTaskEnqueuer.enqueueAsyncDelete(
existingHost, tm().getTransactionTime(), clientId, trid, isSuperuser);
HostResource newHost =
existingHost.asBuilder().addStatusValue(StatusValue.PENDING_DELETE).build();
historyBuilder
.setType(HistoryEntry.Type.HOST_PENDING_DELETE)
.setModificationTime(now)
.setParent(Key.create(existingHost));
tm().insert(historyBuilder.build().toChildHistoryEntity());
HistoryEntry.Type historyEntryType;
Result.Code resultCode;
HostResource newHost;
if (tm().isOfy()) {
asyncTaskEnqueuer.enqueueAsyncDelete(
existingHost, tm().getTransactionTime(), clientId, trid, isSuperuser);
newHost = existingHost.asBuilder().addStatusValue(StatusValue.PENDING_DELETE).build();
historyEntryType = Type.HOST_PENDING_DELETE;
resultCode = SUCCESS_WITH_ACTION_PENDING;
} else {
newHost = existingHost.asBuilder().setStatusValues(null).setDeletionTime(now).build();
if (existingHost.isSubordinate()) {
dnsQueue.addHostRefreshTask(existingHost.getHostName());
tm().update(
tm().loadByKey(existingHost.getSuperordinateDomain())
.asBuilder()
.removeSubordinateHost(existingHost.getHostName())
.build());
}
historyEntryType = Type.HOST_DELETE;
resultCode = SUCCESS;
}
historyBuilder.setType(historyEntryType).setModificationTime(now).setHost(newHost);
tm().insert(historyBuilder.build());
tm().update(newHost);
return responseBuilder.setResultFromCode(SUCCESS_WITH_ACTION_PENDING).build();
return responseBuilder.setResultFromCode(resultCode).build();
}
}
@@ -31,7 +31,6 @@ import static google.registry.persistence.transaction.TransactionManagerFactory.
import static google.registry.util.CollectionUtils.isNullOrEmpty;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.batch.AsyncTaskEnqueuer;
import google.registry.dns.DnsQueue;
import google.registry.flows.EppException;
@@ -55,6 +54,7 @@ import google.registry.model.eppoutput.EppResponse;
import google.registry.model.host.HostCommand.Update;
import google.registry.model.host.HostCommand.Update.AddRemove;
import google.registry.model.host.HostCommand.Update.Change;
import google.registry.model.host.HostHistory;
import google.registry.model.host.HostResource;
import google.registry.model.index.ForeignKeyIndex;
import google.registry.model.reporting.HistoryEntry;
@@ -116,7 +116,7 @@ public final class HostUpdateFlow implements TransactionalFlow {
@Inject @ClientId String clientId;
@Inject @TargetId String targetId;
@Inject @Superuser boolean isSuperuser;
@Inject HistoryEntry.Builder historyBuilder;
@Inject HostHistory.Builder historyBuilder;
@Inject AsyncTaskEnqueuer asyncTaskEnqueuer;
@Inject DnsQueue dnsQueue;
@Inject EppResponse.Builder responseBuilder;
@@ -205,9 +205,8 @@ public final class HostUpdateFlow implements TransactionalFlow {
historyBuilder
.setType(HistoryEntry.Type.HOST_UPDATE)
.setModificationTime(now)
.setParent(Key.create(existingHost))
.build()
.toChildHistoryEntity());
.setHost(newHost)
.build());
tm().updateAll(entitiesToUpdate.build());
tm().insertAll(entitiesToInsert.build());
return responseBuilder.build();
@@ -15,57 +15,29 @@
package google.registry.flows.poll;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.QueryComposer.Comparator.EQ;
import static google.registry.persistence.transaction.QueryComposer.Comparator.LTE;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.persistence.transaction.TransactionManagerUtil.transactIfJpaTm;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import com.googlecode.objectify.cmd.Query;
import google.registry.model.poll.PollMessage;
import google.registry.persistence.transaction.QueryComposer;
import java.util.Optional;
import org.joda.time.DateTime;
/** Static utility functions for poll flows. */
public final class PollFlowUtils {
public static final String SQL_POLL_MESSAGE_QUERY =
"FROM PollMessage WHERE clientId = :registrarId AND eventTime <= :now ORDER BY eventTime ASC";
private static final String SQL_POLL_MESSAGE_COUNT_QUERY =
"SELECT COUNT(*) FROM PollMessage WHERE clientId = :registrarId AND eventTime <= :now";
/** Returns the number of poll messages for the given registrar that are not in the future. */
public static int getPollMessageCount(String registrarId, DateTime now) {
if (tm().isOfy()) {
return datastorePollMessageQuery(registrarId, now).count();
} else {
return jpaTm()
.transact(
() ->
jpaTm()
.query(SQL_POLL_MESSAGE_COUNT_QUERY, Long.class)
.setParameter("registrarId", registrarId)
.setParameter("now", now)
.getSingleResult()
.intValue());
}
return transactIfJpaTm(() -> createPollMessageQuery(registrarId, now).count()).intValue();
}
/** Returns the first (by event time) poll message not in the future for this registrar. */
public static Optional<PollMessage> getFirstPollMessage(String registrarId, DateTime now) {
if (tm().isOfy()) {
return Optional.ofNullable(datastorePollMessageQuery(registrarId, now).first().now());
} else {
return jpaTm()
.transact(
() ->
jpaTm()
.query(SQL_POLL_MESSAGE_QUERY, PollMessage.class)
.setParameter("registrarId", registrarId)
.setParameter("now", now)
.setMaxResults(1)
.getResultStream()
.findFirst());
}
return transactIfJpaTm(
() -> createPollMessageQuery(registrarId, now).orderBy("eventTime").first());
}
/**
@@ -106,14 +78,15 @@ public final class PollFlowUtils {
return includeAckedMessageInCount;
}
/** A Datastore query for poll messages from the given registrar that are not in the future. */
public static Query<PollMessage> datastorePollMessageQuery(String registrarId, DateTime now) {
return ofy()
.load()
.type(PollMessage.class)
.filter("clientId", registrarId)
.filter("eventTime <=", now.toDate())
.order("eventTime");
/**
* Returns the QueryComposer for poll messages from the given registrar that are not in the
* future.
*/
public static QueryComposer<PollMessage> createPollMessageQuery(
String registrarId, DateTime now) {
return tm().createQueryComposer(PollMessage.class)
.where("clientId", EQ, registrarId)
.where("eventTime", LTE, now);
}
private PollFlowUtils() {}
@@ -18,12 +18,9 @@ import static com.google.common.base.CaseFormat.LOWER_HYPHEN;
import static com.google.common.base.CaseFormat.UPPER_UNDERSCORE;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.model.common.EntityGroupRoot.getCrossTldKey;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import com.googlecode.objectify.Key;
import google.registry.config.RegistryConfig.Config;
@@ -37,7 +34,6 @@ import google.registry.privileges.secretmanager.KeyringSecretStore;
import java.io.IOException;
import java.util.Arrays;
import java.util.Optional;
import java.util.stream.Stream;
import javax.inject.Inject;
import org.bouncycastle.openpgp.PGPException;
import org.bouncycastle.openpgp.PGPKeyPair;
@@ -51,6 +47,7 @@ import org.bouncycastle.openpgp.PGPPublicKey;
* @see <a href="https://cloud.google.com/kms/docs/">Google Cloud Key Management Service
* Documentation</a>
*/
// TODO(2021-06-01): rename this class to SecretManagerKeyring and delete KmsSecretRevision
public class KmsKeyring implements Keyring {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
@@ -208,9 +205,9 @@ public class KmsKeyring implements Keyring {
String encryptedData;
if (tm().isOfy()) {
KmsSecret secret =
ofy().load().key(Key.create(getCrossTldKey(), KmsSecret.class, keyName)).now();
auditedOfy().load().key(Key.create(getCrossTldKey(), KmsSecret.class, keyName)).now();
checkState(secret != null, "Requested secret '%s' does not exist.", keyName);
encryptedData = ofy().load().key(secret.getLatestRevision()).now().getEncryptedValue();
encryptedData = auditedOfy().load().key(secret.getLatestRevision()).now().getEncryptedValue();
} else {
Optional<KmsSecretRevision> revision =
tm().transact(() -> KmsSecretRevisionSqlDao.getLatestRevision(keyName));
@@ -221,8 +218,7 @@ public class KmsKeyring implements Keyring {
try {
return kmsConnection.decrypt(keyName, encryptedData);
} catch (Exception e) {
throw new KeyringException(
String.format("CloudKMS decrypt operation failed for secret %s", keyName), e);
return new byte[0];
}
}
@@ -230,7 +226,7 @@ public class KmsKeyring implements Keyring {
try {
return secretStore.getSecret(keyName);
} catch (Exception e) {
return new byte[0];
throw new KeyringException("Failed to retrieve secret for " + keyName, e);
}
}
@@ -243,44 +239,6 @@ public class KmsKeyring implements Keyring {
return secretStoreData;
}
logger.atWarning().log("Values for %s in Datastore and Secret Manager do not match.", keyName);
return dsData;
}
/**
* Generates the tasks to migrate secrets from Datastore to Secret Manager.
*
* <p>The keys in the returned {@link ImmutableMap} are the names of the secrets that need
* migration. The values in the map are {@link Runnable Runnables} that copy secret data from
* Datastore to Secret Manager for their corresponding keys. Only secrets that are absent in
* Secret Manager or have inconsistent values are included in the returned map.
*/
public ImmutableMap<String, Runnable> migrationPlan() {
ImmutableMap.Builder<String, Runnable> tasks = new ImmutableMap.Builder<>();
ImmutableList<String> labels =
Streams.concat(
Stream.of(PrivateKeyLabel.values()).map(PrivateKeyLabel::getLabel),
Stream.of(PublicKeyLabel.values()).map(PublicKeyLabel::getLabel),
Stream.of(StringKeyLabel.values()).map(StringKeyLabel::getLabel))
.collect(ImmutableList.toImmutableList());
for (String keyName : labels) {
byte[] dsData;
try {
dsData = getDecryptedDataFromDatastore(keyName);
} catch (IllegalStateException e) {
logger.atWarning().log("Cannot load %s from Datastore. Skipping...", keyName);
continue;
}
byte[] secretStoreData = getDataFromSecretStore(keyName);
if (Arrays.equals(dsData, secretStoreData)) {
logger.atInfo().log("%s is already up to date.\n", keyName);
continue;
}
logger.atInfo().log("%s needs to be migrated.\n", keyName);
tasks.put(keyName, () -> secretStore.createOrUpdateSecret(keyName, dsData));
}
return tasks.build();
return secretStoreData;
}
}
@@ -36,6 +36,7 @@ import static google.registry.persistence.transaction.TransactionManagerFactory.
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.google.common.collect.ImmutableMap;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryConfig.Config;
import google.registry.keyring.api.KeySerializer;
import google.registry.keyring.kms.KmsKeyring.PrivateKeyLabel;
@@ -57,7 +58,9 @@ import org.bouncycastle.openpgp.PGPPublicKey;
* The {@link KmsUpdater} accumulates updates to a {@link KmsKeyring} and persists them to KMS and
* Datastore when closed.
*/
// TODO(2021-06-01): rename this class to SecretManagerKeyringUpdater
public final class KmsUpdater {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final KmsConnection kmsConnection;
private final KeyringSecretStore secretStore;
@@ -126,29 +129,30 @@ public final class KmsUpdater {
}
/**
* Generates new encryption keys in KMS, encrypts the updated secrets with them, and persists the
* encrypted secrets to Datastore.
* Persists the secrets in the Secret Manager (primary) and the Datastore (secondary).
*
* <p>The operations in this method are organized so that existing {@link KmsSecretRevision}
* entities remain primary and decryptable if a failure occurs.
* <p>Updates to the Secret Manager are not transactional. If an error happens, the successful
* updates are not reverted; unwritten updates are aborted. This is not a problem right now, since
* this class is only used by the {@code UpdateKmsKeyringCommand}, which is invoked manually and
* only updates one secret at a time.
*/
public void update() {
checkState(!secretValues.isEmpty(), "At least one Keyring value must be persisted");
persistEncryptedValues(encryptValues(secretValues));
// Errors when writing to secret store can be thrown to the top, since writes are always
// executed by a human user using the UpdateKmsKeyringCommand.
try {
secretValues
.entrySet()
.forEach(e -> secretStore.createOrUpdateSecret(e.getKey(), e.getValue()));
for (Map.Entry<String, byte[]> e : secretValues.entrySet()) {
secretStore.createOrUpdateSecret(e.getKey(), e.getValue());
logger.atInfo().log("Secret %s updated.", e.getKey());
}
} catch (RuntimeException e) {
throw new RuntimeException(
"Failed to persist secrets to Secret Manager. "
+ "Please check the status of Secret Manager and re-run the command.",
e);
}
// TODO(2021-06-01): remove the writes to Datastore
persistEncryptedValues(encryptValues(secretValues));
}
/**
@@ -15,7 +15,7 @@
package google.registry.mapreduce.inputs;
import static google.registry.model.EntityClasses.ALL_CLASSES;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import com.google.appengine.api.datastore.Cursor;
import com.google.appengine.api.datastore.QueryResultIterator;
@@ -232,7 +232,7 @@ class ChildEntityReader<R extends EppResource, I extends ImmutableObject> extend
/** Query for children of the current resource and of the current child class. */
@Override
public QueryResultIterator<I> getQueryIterator(Cursor cursor) {
return startQueryAt(ofy().load().type(type).ancestor(ancestor), cursor).iterator();
return startQueryAt(auditedOfy().load().type(type).ancestor(ancestor), cursor).iterator();
}
@Override
@@ -14,7 +14,7 @@
package google.registry.mapreduce.inputs;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import com.google.appengine.api.datastore.Cursor;
import com.google.appengine.api.datastore.QueryResultIterator;
@@ -68,7 +68,8 @@ class CommitLogManifestReader
/** Query for children of this bucket. */
Query<CommitLogManifest> createBucketQuery() {
Query<CommitLogManifest> query = ofy().load().type(CommitLogManifest.class).ancestor(bucketKey);
Query<CommitLogManifest> query =
auditedOfy().load().type(CommitLogManifest.class).ancestor(bucketKey);
if (olderThan != null) {
query = query.filterKey(
"<",
@@ -15,7 +15,7 @@
package google.registry.mapreduce.inputs;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import com.google.appengine.api.datastore.Cursor;
import com.google.appengine.api.datastore.QueryResultIterator;
@@ -68,7 +68,8 @@ abstract class EppResourceBaseReader<T> extends RetryingInputReader<EppResourceI
/** Query for children of this bucket. */
Query<EppResourceIndex> query() {
Query<EppResourceIndex> query = ofy().load().type(EppResourceIndex.class).ancestor(bucketKey);
Query<EppResourceIndex> query =
auditedOfy().load().type(EppResourceIndex.class).ancestor(bucketKey);
return filterKinds.isEmpty() ? query : query.filter("kind in", filterKinds);
}
@@ -14,7 +14,7 @@
package google.registry.mapreduce.inputs;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import com.google.appengine.tools.mapreduce.InputReader;
import com.google.common.collect.ImmutableSet;
@@ -62,7 +62,7 @@ class EppResourceEntityReader<R extends EppResource> extends EppResourceBaseRead
// Loop until we find a value, or nextQueryResult() throws a NoSuchElementException.
while (true) {
Key<? extends EppResource> key = nextQueryResult().getKey();
EppResource resource = ofy().load().key(key).now();
EppResource resource = auditedOfy().load().key(key).now();
if (resource == null) {
logger.atSevere().log("EppResourceIndex key %s points at a missing resource", key);
continue;
@@ -15,7 +15,7 @@
package google.registry.mapreduce.inputs;
import static com.google.common.base.Preconditions.checkNotNull;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import com.google.appengine.api.datastore.Cursor;
import com.google.appengine.api.datastore.DatastoreTimeoutException;
@@ -150,7 +150,7 @@ abstract class RetryingInputReader<I, T> extends InputReader<T> {
String.format("Got an unrecoverable failure while reading item %d/%d.", loaded, total),
e);
} finally {
ofy().clearSessionCache();
auditedOfy().clearSessionCache();
}
}
@@ -47,9 +47,9 @@ import google.registry.model.server.KmsSecret;
import google.registry.model.server.KmsSecretRevision;
import google.registry.model.server.Lock;
import google.registry.model.server.ServerSecret;
import google.registry.model.tmch.ClaimsListShard;
import google.registry.model.tmch.ClaimsListShard.ClaimsListRevision;
import google.registry.model.tmch.ClaimsListShard.ClaimsListSingleton;
import google.registry.model.tmch.ClaimsList;
import google.registry.model.tmch.ClaimsList.ClaimsListRevision;
import google.registry.model.tmch.ClaimsList.ClaimsListSingleton;
import google.registry.model.tmch.TmchCrl;
import google.registry.schema.replay.LastSqlTransaction;
@@ -64,7 +64,7 @@ public final class EntityClasses {
BillingEvent.Modification.class,
BillingEvent.OneTime.class,
BillingEvent.Recurring.class,
ClaimsListShard.class,
ClaimsList.class,
ClaimsListRevision.class,
ClaimsListSingleton.class,
CommitLogBucket.class,
@@ -137,7 +137,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
/** Status values associated with this resource. */
@Column(name = "statuses")
// TODO(mmuller): rename to "statuses" once we're off datastore.
// TODO(b/177567432): rename to "statuses" once we're off datastore.
Set<StatusValue> status;
/**
@@ -405,15 +405,18 @@ public final class EppResourceUtils {
.setParameter("fkRepoId", key.getSqlKey())
.setParameter("now", now.toDate());
}
return (ImmutableSet<VKey<DomainBase>>)
query
.setMaxResults(limit)
.getResultStream()
.map(
repoId ->
DomainBase.createVKey(
Key.create(DomainBase.class, (String) repoId)))
.collect(toImmutableSet());
@SuppressWarnings("unchecked")
ImmutableSet<VKey<DomainBase>> domainBaseKeySet =
(ImmutableSet<VKey<DomainBase>>)
query
.setMaxResults(limit)
.getResultStream()
.map(
repoId ->
DomainBase.createVKey(
Key.create(DomainBase.class, (String) repoId)))
.collect(toImmutableSet());
return domainBaseKeySet;
});
}
}
@@ -16,7 +16,7 @@ package google.registry.model;
import static com.google.common.collect.Iterables.transform;
import static com.google.common.collect.Maps.transformValues;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static java.lang.annotation.ElementType.FIELD;
import static java.lang.annotation.RetentionPolicy.RUNTIME;
@@ -189,7 +189,7 @@ public abstract class ImmutableObject implements Cloneable {
private static Object hydrate(Object value) {
if (value instanceof Key) {
if (tm().isOfy()) {
return hydrate(ofy().load().key((Key<?>) value).now());
return hydrate(auditedOfy().load().key((Key<?>) value).now());
}
return value;
} else if (value instanceof Map) {
@@ -39,8 +39,8 @@ import google.registry.model.registrar.RegistrarContact;
import google.registry.model.registry.Registry;
import google.registry.model.registry.Registry.TldState;
import google.registry.model.registry.label.PremiumList;
import google.registry.model.registry.label.PremiumListDualDao;
import google.registry.persistence.VKey;
import google.registry.schema.tld.PremiumListDao;
import google.registry.util.CidrAddressBlock;
import java.util.Collection;
import java.util.Optional;
@@ -295,7 +295,7 @@ public final class OteAccountBuilder {
boolean isEarlyAccess,
int roidSuffix) {
String tldNameAlphaNumerical = tldName.replaceAll("[^a-z0-9]", "");
Optional<PremiumList> premiumList = PremiumListDualDao.getLatestRevision(DEFAULT_PREMIUM_LIST);
Optional<PremiumList> premiumList = PremiumListDao.getLatestRevision(DEFAULT_PREMIUM_LIST);
checkState(premiumList.isPresent(), "Couldn't find premium list %s.", DEFAULT_PREMIUM_LIST);
Registry.Builder builder =
new Registry.Builder()
@@ -42,6 +42,7 @@ import google.registry.model.ImmutableObject;
import google.registry.model.annotations.ReportedOn;
import google.registry.model.common.TimeOfYear;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.domain.token.AllocationToken;
@@ -114,7 +115,7 @@ public abstract class BillingEvent extends ImmutableObject
/** Entity id. */
@Id @javax.persistence.Id Long id;
@Parent @DoNotHydrate @Transient Key<HistoryEntry> parent;
@Parent @DoNotHydrate @Transient Key<? extends HistoryEntry> parent;
/** The registrar to bill. */
@Index
@@ -191,7 +192,7 @@ public abstract class BillingEvent extends ImmutableObject
return targetId;
}
public Key<HistoryEntry> getParentKey() {
public Key<? extends HistoryEntry> getParentKey() {
return parent;
}
@@ -258,7 +259,7 @@ public abstract class BillingEvent extends ImmutableObject
return thisCastToDerived();
}
public B setParent(Key<HistoryEntry> parentKey) {
public B setParent(Key<? extends HistoryEntry> parentKey) {
getInstance().parent = parentKey;
return thisCastToDerived();
}
@@ -602,23 +603,27 @@ public abstract class BillingEvent extends ImmutableObject
GracePeriodStatus.TRANSFER, Reason.TRANSFER);
/**
* Creates a cancellation billing event (parented on the provided history entry, and with the
* history entry's event time) that will cancel out the provided grace period's billing event,
* Creates a cancellation billing event (parented on the provided history key, and with the
* corresponding event time) that will cancel out the provided grace period's billing event,
* using the supplied targetId and deriving other metadata (clientId, billing time, and the
* cancellation reason) from the grace period.
*/
public static BillingEvent.Cancellation forGracePeriod(
GracePeriod gracePeriod, HistoryEntry historyEntry, String targetId) {
GracePeriod gracePeriod,
DateTime eventTime,
Key<DomainHistory> domainHistoryKey,
String targetId) {
checkArgument(gracePeriod.hasBillingEvent(),
"Cannot create cancellation for grace period without billing event");
BillingEvent.Cancellation.Builder builder = new BillingEvent.Cancellation.Builder()
.setReason(checkNotNull(GRACE_PERIOD_TO_REASON.get(gracePeriod.getType())))
.setTargetId(targetId)
.setClientId(gracePeriod.getClientId())
.setEventTime(historyEntry.getModificationTime())
// The charge being cancelled will take place at the grace period's expiration time.
.setBillingTime(gracePeriod.getExpirationTime())
.setParent(historyEntry);
BillingEvent.Cancellation.Builder builder =
new BillingEvent.Cancellation.Builder()
.setReason(checkNotNull(GRACE_PERIOD_TO_REASON.get(gracePeriod.getType())))
.setTargetId(targetId)
.setClientId(gracePeriod.getClientId())
.setEventTime(eventTime)
// The charge being cancelled will take place at the grace period's expiration time.
.setBillingTime(gracePeriod.getExpirationTime())
.setParent(domainHistoryKey);
// Set the grace period's billing event using the appropriate Cancellation builder method.
if (gracePeriod.getOneTimeBillingEvent() != null) {
builder.setOneTimeEventKey(gracePeriod.getOneTimeBillingEvent());
@@ -38,7 +38,7 @@ import google.registry.model.common.TimedTransitionProperty.TimedTransition;
import google.registry.model.registry.label.PremiumList;
import google.registry.model.registry.label.ReservedList;
import google.registry.model.smd.SignedMarkRevocationList;
import google.registry.model.tmch.ClaimsListShard;
import google.registry.model.tmch.ClaimsList;
import google.registry.persistence.VKey;
import google.registry.schema.replay.DatastoreOnlyEntity;
import java.util.Optional;
@@ -61,7 +61,7 @@ public class DatabaseTransitionSchedule extends ImmutableObject implements Datas
/** The id of the transition schedule. */
public enum TransitionId {
/** The schedule for migration of {@link ClaimsListShard} entities. */
/** The schedule for migration of {@link ClaimsList} entities. */
CLAIMS_LIST,
/** The schedule for the migration of {@link PremiumList} and {@link ReservedList}. */
DOMAIN_LABEL_LISTS,
@@ -287,7 +287,7 @@ public class ContactBase extends EppResource implements ResourceWithTransferData
}
@Override
public Builder asBuilder() {
public Builder<? extends ContactBase, ?> asBuilder() {
return new Builder<>(clone(this));
}
@@ -41,6 +41,11 @@ import javax.persistence.PostLoad;
* <p>In addition to the general history fields (e.g. action time, registrar ID) we also persist a
* copy of the contact entity at this point in time. We persist a raw {@link ContactBase} so that
* the foreign-keyed fields in that class can refer to this object.
*
* <p>This class is only marked as a Datastore entity subclass and registered with Objectify so that
* when building it its ID can be auto-populated by Objectify. It is converted to its superclass
* {@link HistoryEntry} when persisted to Datastore using {@link
* google.registry.persistence.transaction.TransactionManager}.
*/
@Entity
@javax.persistence.Table(
@@ -109,6 +114,9 @@ public class ContactHistory extends HistoryEntry implements SqlEntity {
if (contactBase != null && contactBase.getContactId() == null) {
contactBase = null;
}
if (contactBase != null && contactBase.getRepoId() == null) {
contactBase = contactBase.asBuilder().setRepoId(parent.getName()).build();
}
}
// In Datastore, save as a HistoryEntry object regardless of this object's type
@@ -194,7 +202,7 @@ public class ContactHistory extends HistoryEntry implements SqlEntity {
super(instance);
}
public Builder setContactBase(@Nullable ContactBase contactBase) {
public Builder setContact(@Nullable ContactBase contactBase) {
// Nullable for the sake of pre-Registry-3.0 history objects
if (contactBase == null) {
return this;

Some files were not shown because too many files have changed in this diff Show More