1
0
mirror of https://github.com/google/nomulus synced 2026-06-09 16:33:02 +00:00

Compare commits

...

22 Commits

Author SHA1 Message Date
Lai Jiang 01464e8e05 Remove async-delete-pull queue (#1841)
It is not being used anymore.
2022-11-07 22:16:49 -05:00
Weimin Yu 52b0574c73 Use GoogleCredentials for tools Cloud SQL access (#1844) 2022-11-04 17:20:21 -04:00
Lai Jiang a0f177b71f The only remaining Ofy entity is GaeUserConverter after this PR. (#1838)
Therefore this PR also removed several classes and related tests that
support the setup and verification of Ofy entities.

In addition, support for creating a VKey from a string is limited to
VKey<? extends EppResource> only because it is the only use case (to
pass a key to an EPP resource in a web safe way to facilitate resave),
and we do not want to keep an extra simple name to class mapping, in
addition to what persistence.xml contains. I looked into using
PersistenceXmlUtility to obtain the mapping, but the xml file contains
classes with the same simple name (namely OneTime from both PollMessage
and BillingEvent). It doesn't seem like a worthwhile investment to write
more code to deal with that, when the fact is that we only need to
consider EppResource.
2022-11-04 12:47:11 -04:00
Weimin Yu e01448b52e Fix list_cursors command for globals (#1840) 2022-11-04 11:18:20 -04:00
Weimin Yu 4da474e094 Remove Cloud KMS from Nomulus Server (#1839)
* Remove Cloud KMS from Nomulus Server

Removed Cloud KMS from the Nomulus (:core) since it is no longer used.

Renamed remaining classes to reflect their use of the SecretManager.

Updated the config instructions to use a new codename for the keyring:
KMS to CSM. This PR works with both codenames. Will drop 'KMS' after
the internal repo is updated.
2022-11-04 11:17:15 -04:00
Weimin Yu e273a18b4a Implement Keyless Delegated credential (#1836)
Add a implementation of Delegated credential without using downloaded private key.

This is a stop-gap implementation while waiting for a solution from the Java auth library.

Also added a verifier action to test the new credential in production. Testing is helpful because:

Configuration is per-environment, therefore, success in alpha does not fully validate prod.
The relevant use case is triggered by low-frequency activities. Problem may not pop out for hours or longer.
2022-11-03 11:45:23 -04:00
Weimin Yu 8275bc45b9 Switch MetricReporter to App default credential (#1837) 2022-11-02 20:02:52 -04:00
Pavlo Tkach 0b6805531a Add flyway files for allocateId sequence alternative (#1830) 2022-11-02 12:06:40 -04:00
Lai Jiang 592454d97d Remove ofy support from HistoryEntry (#1823)
This PR removes all Ofy related cruft around `HistoryEntry` and its three subclasses in order to support dual-write to datastore and SQL. The class structure was refactored to take advantage of inheritance to reduce code duplication and improve clarity.

Note that for the embedded EPP resources, either their columns are all empty (for pre-3.0 entities imported into SQL), including their unique foreign key (domain name, host name, contact id) and the update timestamp; or they are filled as expected (for entities that were written since dual writing was implemented).

Therefore the check for foreign key column nullness in the various `@PostLoad` methods in the original code is an no-op as the EPP resource would have been loaded as null. In another word, there is no case where the update timestamp is null but other columns are not.

See the following query for the most recent entries in each table where the foreign key column or the update timestamp are null -- they are the same.

```
[I]postgres=> select MAX(history_modification_time) from "DomainHistory" where update_timestamp is null;
            max
----------------------------
 2021-09-27 15:56:52.502+00
(1 row)

[I]postgres=> select MAX(history_modification_time) from "DomainHistory" where domain_name is null;
            max
----------------------------
 2021-09-27 15:56:52.502+00
(1 row)

[I]postgres=> select MAX(history_modification_time) from "ContactHistory" where update_timestamp is null;
            max
----------------------------
 2021-09-27 15:56:04.311+00
(1 row)

[I]postgres=> select MAX(history_modification_time) from "ContactHistory" where contact_id is null;
            max
----------------------------
 2021-09-27 15:56:04.311+00
(1 row)

[I]postgres=> select MAX(history_modification_time) from "HostHistory" where update_timestamp is null;
            max
----------------------------
 2021-09-27 15:52:16.517+00
(1 row)

[I]postgres=> select MAX(history_modification_time) from "HostHistory" where host_name is null;
            max
----------------------------
 2021-09-27 15:52:16.517+00
(1 row)
```
2022-11-01 21:17:20 -04:00
Weimin Yu 671e42474c Document alternative method to deploy schema 2022-11-01 12:58:11 -04:00
Lai Jiang 1c90a6648e Remove bulk query entities (#1834)
These alternative ORMs are introduced as a way to make querying large number of
domains and domain histories more efficient through bulk loading from several
to-be-joined tables separately, then in-memory re-assembly of the final entity,
bypassing the need to query multiple tables each time an entity is queried.

Their primary use case is loading these entities for comparison between
datastore and SQL during the migration, which has been completed. The
code remain unused as of now and their existence makes refactoring and
general maintenance more complicated than necessary due to the need to keep
them up to date.

Therefore we remove the related code.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1834)
<!-- Reviewable:end -->
2022-10-28 12:25:57 -04:00
Lai Jiang 3f68ad5ea3 Rename BackupGroupRoot (#1829)
Also removed the ability to disable update timestamp auto update as it
was only needed during the migration.

Lastly, rectified the use of raw Coder in RegistryJpaIO.
2022-10-28 12:22:53 -04:00
gbrodman 9c6c210e21 Check for entity nonexistence in SqlBatchWriter (#1824)
Passing in an already-existing instance is an antipattern because it can
lead to race conditions where something else modified the object in
between when the pipeline loaded it and when you're saving it. The Write
action should only be writing new entities.

We cannot check IDs for the objects (some IDs are not autogenerated so
they might exist already). We also cannot call `insert` on the objects
because the underlying JPA `persist` call adds the input object to the
persistence context, meaning that any modifications (e.g.
updateTimestamp) are reflected in the input object. Beam doesn't allow
modification of input objects.
2022-10-27 14:46:26 -04:00
sarahcaseybot ca60ca159f Add DEFAULT_PROMO token type (#1832)
* Add DEFAULT_PROMO token type

* Fix test error message check
2022-10-27 13:08:15 -04:00
Lai Jiang 82092b3516 Remove ofy-only functions in TransactionManager (#1826)
Also remove the use of auditedOfy in places other than the
GaeUserIdConverter.
2022-10-25 15:52:00 -04:00
sarahcaseybot 0746d28e0c Check token type of currentPackageToken (#1825)
* Check currentPackageToken TokenType

* Check TokenType of currentPackageToken

* Check that token already exists
2022-10-25 12:39:33 -04:00
Lai Jiang aaa311ec40 Remove the mechanism to compare objects across database (#1822)
The migration is done.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1822)
<!-- Reviewable:end -->
2022-10-20 13:19:48 -04:00
Lai Jiang addef17904 Does not self allocate IDs in Beam by default. (#1809)
* Does not self allocate IDs in Beam by default.

Per b/250948425, it is dangerous to implicitly allow all Beam pipelines
to create buildables by self allocating the IDs. This change makes it so
that one has to explicitly request self allocation in Beam.

A boolean is added to the pipeline option so that it can be passed to
the beam worker initializer that controls the behavior of the JVM on
each worker. Note that we did not add the option in the metadata.json file
because we did not want people to use the override at run time when launching
a pipeline, due to the risk. As shown in RdePipeline.java, we instead
explicitly hard-code the option in the pipeline. There is nothing that
stops one to supply that option when launching the pipeline, but it's
not advised.

Tested=deployed the pipeline alpha and ran it.
2022-10-19 20:44:06 -04:00
Weimin Yu 8fe3c08069 Properly create and use default credential (#1818)
* Properly create and use default credential

This PR consists of the following changes:

- Stopped adding scopes to the default credential when using it to access other
  non-workspace GCP APIs. Scopes are not needed here.

- Started applying scopes to the default credential when using to access
  Drive and Sheets APIs.
  - Upgraded Drive access from the deprecated credential lib to the
    up-to-date one
  - Switched Sheet access from the exported json credential to the
    scoped default credential.

This PR requires that the affected files be writable to the default
service account (project-name@appspot.gserviceaccount.com) of the
project.

- This is already the case for exported files (premium terms, reserved
  terms, and domain list).

- The registrar sync sheets in alpha, sandbox, and production have been
  updated with the new permissions.

All impacted operations have been tested in alpha.

* Properly create and use default credential

This PR consists of the following changes:

- Added a new method to generate scope-less default credential when using it to
  access other non-workspace GCP APIs. Scopes are not needed here.

  - Started to use the new credential in the SecreteManager.
  - Will migrate other usages to this new credential gradually.
  - Marked the old DefaultCredential as deprecated.

- Started applying scopes to the default credential when using to access Drive
  and Sheets APIs.

  - Upgraded Drive access from the deprecated credentials lib
  - Switched Sheet access from the exported json credential to the scoped
    default credential.

This PR requires that the affected files be writable to the default service
account (project-name@appspot.gserviceaccount.com) of the project.

- This is already the case for exported files (premium terms, reserved terms,
  and domain list).

- The registrar sync sheets in alpha, sandbox, and production have been
  updated with the new permissions.

All impacted operations have been tested in alpha.
2022-10-18 20:20:36 -04:00
sarahcaseybot 5dc796b1f7 Add monitoring for package max create limit (#1798)
* Add action for checking package domain create limit compliance

* Add create limit monitoring

* Change variable name

* Add more logging
2022-10-18 12:39:53 -04:00
Ben McIlwain 8bddf35d0d Revert "Upgrade App Engine Standard to Java 17 w/ bundled APIs (#1816)" (#1817)
This reverts commit 1ab077d267.

Apparently the new version of Spinnaker that is compatible with this doesn't
work for our release, so we need to roll this back for now. (Again!)
2022-10-13 10:05:47 -04:00
Pavlo Tkach 7b9c16ca3e Update conditions when domain update flow triggers dns publish task (#1811)
Addressing b/246375161
2022-10-12 10:25:33 -04:00
321 changed files with 3400 additions and 8593 deletions
-1
View File
@@ -186,7 +186,6 @@ dependencies {
implementation deps['com.google.apis:google-api-services-admin-directory']
implementation deps['com.google.apis:google-api-services-appengine']
implementation deps['com.google.apis:google-api-services-bigquery']
implementation deps['com.google.apis:google-api-services-cloudkms']
implementation deps['com.google.apis:google-api-services-dataflow']
implementation deps['com.google.apis:google-api-services-dns']
implementation deps['com.google.apis:google-api-services-drive']
+8 -12
View File
@@ -31,10 +31,10 @@ com.github.jnr:jnr-unixsocket:0.38.17=compileClasspath,default,deploy_jar,nonpro
com.github.jnr:jnr-x86asm:1.0.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.android:annotations:4.1.1.4=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
com.google.api-client:google-api-client-appengine:1.32.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api-client:google-api-client-appengine:1.35.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api-client:google-api-client-jackson2:1.32.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api-client:google-api-client-java6:1.35.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api-client:google-api-client-servlet:1.32.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api-client:google-api-client-servlet:1.35.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api-client:google-api-client:1.35.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
@@ -77,7 +77,6 @@ com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0=c
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-cloudkms:v1-rev20220701-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
@@ -90,15 +89,12 @@ com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1=compileClasspat
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220623-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-storage:v1-rev20220705-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine.tools:appengine-gcs-client:0.8.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine.tools:appengine-gcs-client:0.8.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine.tools:appengine-pipeline:0.2.13=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-api-1.0-sdk:2.0.5=compileClasspath,nonprodCompileClasspath
com.google.appengine:appengine-api-1.0-sdk:2.0.9=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-api-stubs:2.0.9=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-remote-api:2.0.5=compileClasspath,nonprodCompileClasspath
com.google.appengine:appengine-remote-api:2.0.9=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-testing:2.0.9=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-tools-sdk:2.0.9=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-api-1.0-sdk:2.0.5=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-api-stubs:2.0.5=testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-remote-api:2.0.5=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-testing:1.9.86=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auth:google-auth-library-credentials:1.8.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auth:google-auth-library-oauth2-http:1.8.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
@@ -302,7 +298,7 @@ javax.validation:validation-api:1.0.0.GA=compileClasspath,default,deploy_jar,non
javax.xml.bind:jaxb-api:2.3.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
javax.xml.bind:jaxb-api:2.4.0-b180830.0359=jaxb
jline:jline:1.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
joda-time:joda-time:2.10.13=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
joda-time:joda-time:2.10.10=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
junit:junit:4.13.2=default,nonprodCompileClasspath,nonprodRuntimeClasspath,testCompileClasspath,testRuntimeClasspath
net.arnx:nashorn-promise:0.1.1=nonprodRuntime,runtime,testRuntimeClasspath
net.bytebuddy:byte-buddy-agent:1.12.10=testCompileClasspath,testRuntimeClasspath
@@ -17,25 +17,16 @@ package google.registry.batch;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import com.google.appengine.api.taskqueue.Queue;
import com.google.appengine.api.taskqueue.TaskOptions;
import com.google.appengine.api.taskqueue.TaskOptions.Method;
import com.google.appengine.api.taskqueue.TransientFailureException;
import com.google.common.base.Joiner;
import com.google.common.collect.ArrayListMultimap;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Multimap;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.EppResource;
import google.registry.model.eppcommon.Trid;
import google.registry.model.host.Host;
import google.registry.persistence.VKey;
import google.registry.request.Action.Service;
import google.registry.util.CloudTasksUtils;
import google.registry.util.Retrier;
import javax.inject.Inject;
import javax.inject.Named;
import org.joda.time.DateTime;
import org.joda.time.Duration;
@@ -44,45 +35,28 @@ public final class AsyncTaskEnqueuer {
/** The HTTP parameter names used by async flows. */
public static final String PARAM_RESOURCE_KEY = "resourceKey";
public static final String PARAM_REQUESTING_CLIENT_ID = "requestingClientId";
public static final String PARAM_CLIENT_TRANSACTION_ID = "clientTransactionId";
public static final String PARAM_SERVER_TRANSACTION_ID = "serverTransactionId";
public static final String PARAM_IS_SUPERUSER = "isSuperuser";
public static final String PARAM_HOST_KEY = "hostKey";
public static final String PARAM_REQUESTED_TIME = "requestedTime";
public static final String PARAM_RESAVE_TIMES = "resaveTimes";
/** The task queue names used by async flows. */
public static final String QUEUE_ASYNC_ACTIONS = "async-actions";
public static final String QUEUE_ASYNC_DELETE = "async-delete-pull";
public static final String QUEUE_ASYNC_HOST_RENAME = "async-host-rename-pull";
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final Duration MAX_ASYNC_ETA = Duration.standardDays(30);
private final Duration asyncDeleteDelay;
private final Queue asyncDeletePullQueue;
private final Queue asyncDnsRefreshPullQueue;
private final Retrier retrier;
private CloudTasksUtils cloudTasksUtils;
private final CloudTasksUtils cloudTasksUtils;
@Inject
public AsyncTaskEnqueuer(
@Named(QUEUE_ASYNC_DELETE) Queue asyncDeletePullQueue,
@Named(QUEUE_ASYNC_HOST_RENAME) Queue asyncDnsRefreshPullQueue,
@Config("asyncDeleteDelay") Duration asyncDeleteDelay,
CloudTasksUtils cloudTasksUtils,
Retrier retrier) {
this.asyncDeletePullQueue = asyncDeletePullQueue;
this.asyncDnsRefreshPullQueue = asyncDnsRefreshPullQueue;
this.asyncDeleteDelay = asyncDeleteDelay;
public AsyncTaskEnqueuer(CloudTasksUtils cloudTasksUtils) {
this.cloudTasksUtils = cloudTasksUtils;
this.retrier = retrier;
}
/** Enqueues a task to asynchronously re-save an entity at some point in the future. */
public void enqueueAsyncResave(VKey<?> entityToResave, DateTime now, DateTime whenToResave) {
public void enqueueAsyncResave(
VKey<? extends EppResource> entityToResave, DateTime now, DateTime whenToResave) {
enqueueAsyncResave(entityToResave, now, ImmutableSortedSet.of(whenToResave));
}
@@ -93,7 +67,9 @@ public final class AsyncTaskEnqueuer {
* itself to run at the next time if there are remaining re-saves scheduled.
*/
public void enqueueAsyncResave(
VKey<?> entityKey, DateTime now, ImmutableSortedSet<DateTime> whenToResave) {
VKey<? extends EppResource> entityKey,
DateTime now,
ImmutableSortedSet<DateTime> whenToResave) {
DateTime firstResave = whenToResave.first();
checkArgument(isBeforeOrAt(now, firstResave), "Can't enqueue a resave to run in the past");
Duration etaDuration = new Duration(now, firstResave);
@@ -115,46 +91,4 @@ public final class AsyncTaskEnqueuer {
cloudTasksUtils.createPostTaskWithDelay(
ResaveEntityAction.PATH, Service.BACKEND.toString(), params, etaDuration));
}
/** Enqueues a task to asynchronously delete a contact or host, by key. */
public void enqueueAsyncDelete(
EppResource resourceToDelete,
DateTime now,
String requestingRegistrarId,
Trid trid,
boolean isSuperuser) {
logger.atInfo().log(
"Enqueuing async deletion of %s on behalf of registrar %s.",
resourceToDelete.getRepoId(), requestingRegistrarId);
TaskOptions task =
TaskOptions.Builder.withMethod(Method.PULL)
.countdownMillis(asyncDeleteDelay.getMillis())
.param(PARAM_RESOURCE_KEY, resourceToDelete.createVKey().stringify())
.param(PARAM_REQUESTING_CLIENT_ID, requestingRegistrarId)
.param(PARAM_SERVER_TRANSACTION_ID, trid.getServerTransactionId())
.param(PARAM_IS_SUPERUSER, Boolean.toString(isSuperuser))
.param(PARAM_REQUESTED_TIME, now.toString());
trid.getClientTransactionId()
.ifPresent(clTrid -> task.param(PARAM_CLIENT_TRANSACTION_ID, clTrid));
addTaskToQueueWithRetry(asyncDeletePullQueue, task);
}
/** Enqueues a task to asynchronously refresh DNS for a renamed host. */
public void enqueueAsyncDnsRefresh(Host host, DateTime now) {
VKey<Host> hostKey = host.createVKey();
logger.atInfo().log("Enqueuing async DNS refresh for renamed host %s.", hostKey);
addTaskToQueueWithRetry(
asyncDnsRefreshPullQueue,
TaskOptions.Builder.withMethod(Method.PULL)
.param(PARAM_HOST_KEY, hostKey.stringify())
.param(PARAM_REQUESTED_TIME, now.toString()));
}
/**
* Adds a task to a queue with retrying, to avoid aborting the entire flow over a transient issue
* enqueuing a task.
*/
private void addTaskToQueueWithRetry(final Queue queue, final TaskOptions task) {
retrier.callWithRetry(() -> queue.add(task), TransientFailureException.class);
}
}
@@ -14,13 +14,10 @@
package google.registry.batch;
import static com.google.appengine.api.taskqueue.QueueFactory.getQueue;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_REQUESTED_TIME;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_RESAVE_TIMES;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_RESOURCE_KEY;
import static google.registry.batch.AsyncTaskEnqueuer.QUEUE_ASYNC_ACTIONS;
import static google.registry.batch.AsyncTaskEnqueuer.QUEUE_ASYNC_DELETE;
import static google.registry.batch.AsyncTaskEnqueuer.QUEUE_ASYNC_HOST_RENAME;
import static google.registry.batch.CannedScriptExecutionAction.SCRIPT_PARAM;
import static google.registry.request.RequestParameters.extractBooleanParameter;
import static google.registry.request.RequestParameters.extractIntParameter;
import static google.registry.request.RequestParameters.extractLongParameter;
@@ -32,13 +29,11 @@ import static google.registry.request.RequestParameters.extractRequiredDatetimeP
import static google.registry.request.RequestParameters.extractRequiredParameter;
import static google.registry.request.RequestParameters.extractSetOfDatetimeParameters;
import com.google.appengine.api.taskqueue.Queue;
import com.google.common.collect.ImmutableSet;
import dagger.Module;
import dagger.Provides;
import google.registry.request.Parameter;
import java.util.Optional;
import javax.inject.Named;
import javax.servlet.http.HttpServletRequest;
import org.joda.time.DateTime;
@@ -128,21 +123,10 @@ public class BatchModule {
return extractBooleanParameter(req, PARAM_DRY_RUN);
}
// TODO(b/234424397): remove method after credential changes are rolled out.
@Provides
@Named(QUEUE_ASYNC_ACTIONS)
static Queue provideAsyncActionsPushQueue() {
return getQueue(QUEUE_ASYNC_ACTIONS);
}
@Provides
@Named(QUEUE_ASYNC_DELETE)
static Queue provideAsyncDeletePullQueue() {
return getQueue(QUEUE_ASYNC_DELETE);
}
@Provides
@Named(QUEUE_ASYNC_HOST_RENAME)
static Queue provideAsyncHostRenamePullQueue() {
return getQueue(QUEUE_ASYNC_HOST_RENAME);
@Parameter(SCRIPT_PARAM)
static String provideScriptName(HttpServletRequest req) {
return extractRequiredParameter(req, SCRIPT_PARAM);
}
}
@@ -0,0 +1,74 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch;
import static google.registry.request.Action.Method.POST;
import com.google.common.collect.ImmutableMap;
import com.google.common.flogger.FluentLogger;
import google.registry.batch.cannedscript.GroupsApiChecker;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
import javax.inject.Inject;
/**
* Action that executes a canned script specified by the caller.
*
* <p>This class is introduced to help the safe rollout of credential changes. The delegated
* credentials in particular, benefit from this: they require manual configuration of the peer
* system in each environment, and may wait hours or even days after deployment until triggered by
* user activities.
*
* <p>This action can be invoked using the Nomulus CLI command: {@code nomulus -e ${env} curl
* --service BACKEND -X POST -u '/_dr/task/executeCannedScript?script=${script_name}'}
*/
// TODO(b/234424397): remove class after credential changes are rolled out.
@Action(
service = Action.Service.BACKEND,
path = "/_dr/task/executeCannedScript",
method = POST,
automaticallyPrintOk = true,
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
public class CannedScriptExecutionAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
static final String SCRIPT_PARAM = "script";
static final ImmutableMap<String, Runnable> SCRIPTS =
ImmutableMap.of("runGroupsApiChecks", GroupsApiChecker::runGroupsApiChecks);
private final String scriptName;
@Inject
CannedScriptExecutionAction(@Parameter(SCRIPT_PARAM) String scriptName) {
logger.atInfo().log("Received request to run script %s", scriptName);
this.scriptName = scriptName;
}
@Override
public void run() {
if (!SCRIPTS.containsKey(scriptName)) {
throw new IllegalArgumentException("Script not found:" + scriptName);
}
try {
SCRIPTS.get(scriptName).run();
logger.atInfo().log("Finished running %s.", scriptName);
} catch (Throwable t) {
logger.atWarning().withCause(t).log("Error executing %s", scriptName);
throw new RuntimeException("Execution failed.");
}
}
}
@@ -0,0 +1,80 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableList;
import com.google.common.flogger.FluentLogger;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.token.PackagePromotion;
import google.registry.request.Action;
import google.registry.request.Action.Service;
import google.registry.request.auth.Auth;
import java.util.List;
/**
* An action that checks all {@link PackagePromotion} objects for compliance with their max create
* limit.
*/
@Action(
service = Service.BACKEND,
path = CheckPackagesComplianceAction.PATH,
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
public class CheckPackagesComplianceAction implements Runnable {
public static final String PATH = "/_dr/task/checkPackagesCompliance";
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
@Override
public void run() {
tm().transact(
() -> {
ImmutableList<PackagePromotion> packages = tm().loadAllOf(PackagePromotion.class);
ImmutableList.Builder<PackagePromotion> packagesOverCreateLimit =
new ImmutableList.Builder<>();
for (PackagePromotion packagePromo : packages) {
List<DomainHistory> creates =
jpaTm()
.query(
"FROM DomainHistory WHERE current_package_token = :token AND"
+ " modificationTime >= :lastBilling AND type = 'DOMAIN_CREATE'",
DomainHistory.class)
.setParameter("token", packagePromo.getToken().getKey().toString())
.setParameter(
"lastBilling", packagePromo.getNextBillingDate().minusYears(1))
.getResultList();
if (creates.size() > packagePromo.getMaxCreates()) {
int overage = creates.size() - packagePromo.getMaxCreates();
logger.atInfo().log(
"Package with package token %s has exceeded their max domain creation limit"
+ " by %d name(s).",
packagePromo.getToken().getKey(), overage);
packagesOverCreateLimit.add(packagePromo);
}
}
if (packagesOverCreateLimit.build().isEmpty()) {
logger.atInfo().log("Found no packages over their create limit.");
} else {
logger.atInfo().log(
"Found %d packages over their create limit.",
packagesOverCreateLimit.build().size());
// TODO(sarahbot@) Send email to registrar and registry informing of creation
// overage once email template is finalized.
}
});
}
}
@@ -101,15 +101,20 @@ public class DeleteProberDataAction implements Runnable {
@Inject DnsQueue dnsQueue;
@Inject @Parameter(PARAM_DRY_RUN) boolean isDryRun;
@Inject
@Parameter(PARAM_DRY_RUN)
boolean isDryRun;
/** List of TLDs to work on. If empty - will work on all TLDs that end with .test. */
@Inject @Parameter(PARAM_TLDS) ImmutableSet<String> tlds;
@Inject
@Parameter(PARAM_TLDS)
ImmutableSet<String> tlds;
@Inject
@Config("registryAdminClientId")
String registryAdminRegistrarId;
@Inject DeleteProberDataAction() {}
@Inject
DeleteProberDataAction() {}
@Override
public void run() {
@@ -151,7 +156,7 @@ public class DeleteProberDataAction implements Runnable {
DateTime now = tm().getTransactionTime();
// Scroll through domains, soft-deleting as necessary (very few will be soft-deleted) and
// keeping track of which domains to hard-delete (there can be many, so we batch them up)
ScrollableResults scrollableResult =
try (ScrollableResults scrollableResult =
jpaTm()
.query(DOMAIN_QUERY_STRING, Domain.class)
.setParameter("tlds", deletableTlds)
@@ -161,28 +166,30 @@ public class DeleteProberDataAction implements Runnable {
.setParameter("nowAutoTimestamp", CreateAutoTimestamp.create(now))
.unwrap(Query.class)
.setCacheMode(CacheMode.IGNORE)
.scroll(ScrollMode.FORWARD_ONLY);
ImmutableList.Builder<String> domainRepoIdsToHardDelete = new ImmutableList.Builder<>();
ImmutableList.Builder<String> hostNamesToHardDelete = new ImmutableList.Builder<>();
for (int i = 1; scrollableResult.next(); i = (i + 1) % BATCH_SIZE) {
Domain domain = (Domain) scrollableResult.get(0);
processDomain(
domain,
domainRepoIdsToHardDelete,
hostNamesToHardDelete,
softDeletedDomains,
hardDeletedDomains);
// Batch the deletion and DB flush + session clearing so we don't OOM
if (i == 0) {
hardDeleteDomainsAndHosts(domainRepoIdsToHardDelete.build(), hostNamesToHardDelete.build());
domainRepoIdsToHardDelete = new ImmutableList.Builder<>();
hostNamesToHardDelete = new ImmutableList.Builder<>();
jpaTm().getEntityManager().flush();
jpaTm().getEntityManager().clear();
.scroll(ScrollMode.FORWARD_ONLY)) {
ImmutableList.Builder<String> domainRepoIdsToHardDelete = new ImmutableList.Builder<>();
ImmutableList.Builder<String> hostNamesToHardDelete = new ImmutableList.Builder<>();
for (int i = 1; scrollableResult.next(); i = (i + 1) % BATCH_SIZE) {
Domain domain = (Domain) scrollableResult.get(0);
processDomain(
domain,
domainRepoIdsToHardDelete,
hostNamesToHardDelete,
softDeletedDomains,
hardDeletedDomains);
// Batch the deletion and DB flush + session clearing, so we don't OOM
if (i == 0) {
hardDeleteDomainsAndHosts(
domainRepoIdsToHardDelete.build(), hostNamesToHardDelete.build());
domainRepoIdsToHardDelete = new ImmutableList.Builder<>();
hostNamesToHardDelete = new ImmutableList.Builder<>();
jpaTm().getEntityManager().flush();
jpaTm().getEntityManager().clear();
}
}
// process the remainder
hardDeleteDomainsAndHosts(domainRepoIdsToHardDelete.build(), hostNamesToHardDelete.build());
}
// process the remainder
hardDeleteDomainsAndHosts(domainRepoIdsToHardDelete.build(), hostNamesToHardDelete.build());
}
private void processDomain(
@@ -236,7 +243,7 @@ public class DeleteProberDataAction implements Runnable {
.setParameter("repoIds", domainRepoIds)
.executeUpdate();
jpaTm()
.query("DELETE FROM DomainHistory WHERE domainRepoId IN :repoIds")
.query("DELETE FROM DomainHistory WHERE repoId IN :repoIds")
.setParameter("repoIds", domainRepoIds)
.executeUpdate();
jpaTm()
@@ -253,7 +253,7 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
final ImmutableSet<DateTime> billingTimes =
getBillingTimesInScope(eventTimes, cursorTime, executeTime, tld);
VKey<Domain> domainKey = VKey.createSql(Domain.class, recurring.getDomainRepoId());
VKey<Domain> domainKey = VKey.create(Domain.class, recurring.getDomainRepoId());
Iterable<OneTime> oneTimesForDomain;
oneTimesForDomain =
tm().createQueryComposer(OneTime.class)
@@ -23,7 +23,6 @@ import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.flogger.FluentLogger;
import google.registry.model.EppResource;
import google.registry.model.ImmutableObject;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Action.Method;
@@ -75,14 +74,11 @@ public class ResaveEntityAction implements Runnable {
"Re-saving entity %s which was enqueued at %s.", resourceKey, requestedTime);
tm().transact(
() -> {
ImmutableObject entity = tm().loadByKey(VKey.create(resourceKey));
tm().put(
(entity instanceof EppResource)
? ((EppResource) entity).cloneProjectedAtTime(tm().getTransactionTime())
: entity);
EppResource entity = tm().loadByKey(VKey.createEppVKeyFromString(resourceKey));
tm().put(entity.cloneProjectedAtTime(tm().getTransactionTime()));
if (!resaveTimes.isEmpty()) {
asyncTaskEnqueuer.enqueueAsyncResave(
VKey.create(resourceKey), requestedTime, resaveTimes);
VKey.createEppVKeyFromString(resourceKey), requestedTime, resaveTimes);
}
});
response.setPayload("Entity re-saved.");
@@ -0,0 +1,122 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch.cannedscript;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.util.RegistrarUtils.normalizeRegistrarId;
import com.google.api.services.admin.directory.Directory;
import com.google.api.services.groupssettings.Groupssettings;
import com.google.common.base.Supplier;
import com.google.common.base.Suppliers;
import com.google.common.base.Throwables;
import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import dagger.Component;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule;
import google.registry.config.CredentialModule.AdcDelegatedCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.groups.DirectoryGroupsConnection;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.util.GoogleCredentialsBundle;
import google.registry.util.UtilsModule;
import java.util.List;
import java.util.Set;
import javax.inject.Singleton;
/**
* Verifies that the credential with the {@link AdcDelegatedCredential} annotation can be used to
* access the Google Workspace Groups API.
*/
// TODO(b/234424397): remove class after credential changes are rolled out.
public class GroupsApiChecker {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final Supplier<GroupsConnectionComponent> COMPONENT_SUPPLIER =
Suppliers.memoize(DaggerGroupsApiChecker_GroupsConnectionComponent::create);
public static void runGroupsApiChecks() {
GroupsConnectionComponent component = COMPONENT_SUPPLIER.get();
DirectoryGroupsConnection groupsConnection = component.groupsConnection();
List<Registrar> registrars =
Streams.stream(Registrar.loadAllCached())
.filter(registrar -> registrar.isLive() && registrar.getType() == Registrar.Type.REAL)
.collect(toImmutableList());
for (Registrar registrar : registrars) {
for (final RegistrarPoc.Type type : RegistrarPoc.Type.values()) {
String groupKey =
String.format(
"%s-%s-contacts@%s",
normalizeRegistrarId(registrar.getRegistrarId()),
type.getDisplayName(),
component.gSuiteDomainName());
try {
Set<String> currentMembers = groupsConnection.getMembersOfGroup(groupKey);
logger.atInfo().log("Found %s members for %s.", currentMembers.size(), groupKey);
} catch (Exception e) {
Throwables.throwIfUnchecked(e);
throw new RuntimeException(e);
}
}
}
}
@Singleton
@Component(
modules = {
ConfigModule.class,
CredentialModule.class,
GroupsApiModule.class,
UtilsModule.class
})
interface GroupsConnectionComponent {
DirectoryGroupsConnection groupsConnection();
@Config("gSuiteDomainName")
String gSuiteDomainName();
}
@Module
static class GroupsApiModule {
@Provides
static Directory provideDirectory(
@AdcDelegatedCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Directory.Builder(
credentialsBundle.getHttpTransport(),
credentialsBundle.getJsonFactory(),
credentialsBundle.getHttpRequestInitializer())
.setApplicationName(projectId)
.build();
}
@Provides
static Groupssettings provideGroupsSettings(
@AdcDelegatedCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Groupssettings.Builder(
credentialsBundle.getHttpTransport(),
credentialsBundle.getJsonFactory(),
credentialsBundle.getHttpRequestInitializer())
.setApplicationName(projectId)
.build();
}
}
}
@@ -14,6 +14,7 @@
package google.registry.beam.common;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static org.apache.beam.sdk.values.TypeDescriptors.integers;
@@ -21,18 +22,14 @@ import com.google.auto.value.AutoValue;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Streams;
import google.registry.beam.common.RegistryQuery.CriteriaQuerySupplier;
import google.registry.model.UpdateAutoTimestamp;
import google.registry.model.UpdateAutoTimestamp.DisableAutoUpdateResource;
import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.persistence.transaction.TransactionManagerFactory;
import java.io.Serializable;
import java.util.Map;
import java.util.Objects;
import java.util.concurrent.ThreadLocalRandom;
import javax.annotation.Nullable;
import javax.persistence.criteria.CriteriaQuery;
import org.apache.beam.sdk.coders.Coder;
import org.apache.beam.sdk.coders.SerializableCoder;
import org.apache.beam.sdk.metrics.Counter;
import org.apache.beam.sdk.metrics.Metrics;
import org.apache.beam.sdk.transforms.Create;
@@ -135,6 +132,7 @@ public final class RegistryJpaIO {
abstract SerializableFunction<R, T> resultMapper();
@Nullable
abstract Coder<T> coder();
@Nullable
@@ -145,13 +143,16 @@ public final class RegistryJpaIO {
@Override
@SuppressWarnings("deprecation") // Reshuffle still recommended by GCP.
public PCollection<T> expand(PBegin input) {
return input
.apply("Starting " + name(), Create.of((Void) null))
.apply(
"Run query for " + name(),
ParDo.of(new QueryRunner<>(query(), resultMapper(), snapshotId())))
.setCoder(coder())
.apply("Reshuffle", Reshuffle.viaRandomKey());
PCollection<T> output =
input
.apply("Starting " + name(), Create.of((Void) null))
.apply(
"Run query for " + name(),
ParDo.of(new QueryRunner<>(query(), resultMapper(), snapshotId())));
if (coder() != null) {
output = output.setCoder(coder());
}
return output.apply("Reshuffle", Reshuffle.viaRandomKey());
}
public Read<R, T> withName(String name) {
@@ -179,9 +180,7 @@ public final class RegistryJpaIO {
}
static <R, T> Builder<R, T> builder() {
return new AutoValue_RegistryJpaIO_Read.Builder<R, T>()
.name(DEFAULT_NAME)
.coder(SerializableCoder.of(Serializable.class));
return new AutoValue_RegistryJpaIO_Read.Builder<R, T>().name(DEFAULT_NAME);
}
@AutoValue.Builder
@@ -193,7 +192,7 @@ public final class RegistryJpaIO {
abstract Builder<R, T> resultMapper(SerializableFunction<R, T> mapper);
abstract Builder<R, T> coder(Coder coder);
abstract Builder<R, T> coder(Coder<T> coder);
abstract Builder<R, T> snapshotId(@Nullable String sharedSnapshotId);
@@ -298,12 +297,6 @@ public final class RegistryJpaIO {
public abstract SerializableFunction<T, Object> jpaConverter();
/**
* Signal to the writer that the {@link UpdateAutoTimestamp} property should be allowed to
* manipulate its value before persistence. The default value is {@code true}.
*/
abstract boolean withUpdateAutoTimestamp();
public Write<T> withName(String name) {
return toBuilder().name(name).build();
}
@@ -324,10 +317,6 @@ public final class RegistryJpaIO {
return toBuilder().jpaConverter(jpaConverter).build();
}
public Write<T> disableUpdateAutoTimestamp() {
return toBuilder().withUpdateAutoTimestamp(false).build();
}
abstract Builder<T> toBuilder();
@Override
@@ -344,7 +333,7 @@ public final class RegistryJpaIO {
GroupIntoBatches.<Integer, T>ofSize(batchSize()).withShardedKey())
.apply(
"Write in batch for " + name(),
ParDo.of(new SqlBatchWriter<>(name(), jpaConverter(), withUpdateAutoTimestamp())));
ParDo.of(new SqlBatchWriter<>(name(), jpaConverter())));
}
static <T> Builder<T> builder() {
@@ -352,8 +341,7 @@ public final class RegistryJpaIO {
.name(DEFAULT_NAME)
.batchSize(DEFAULT_BATCH_SIZE)
.shards(DEFAULT_SHARDS)
.jpaConverter(x -> x)
.withUpdateAutoTimestamp(true);
.jpaConverter(x -> x);
}
@AutoValue.Builder
@@ -367,8 +355,6 @@ public final class RegistryJpaIO {
abstract Builder<T> jpaConverter(SerializableFunction<T, Object> jpaConverter);
abstract Builder<T> withUpdateAutoTimestamp(boolean withUpdateAutoTimestamp);
abstract Write<T> build();
}
}
@@ -377,24 +363,15 @@ public final class RegistryJpaIO {
private static class SqlBatchWriter<T> extends DoFn<KV<ShardedKey<Integer>, Iterable<T>>, Void> {
private final Counter counter;
private final SerializableFunction<T, Object> jpaConverter;
private final boolean withAutoTimestamp;
SqlBatchWriter(
String type, SerializableFunction<T, Object> jpaConverter, boolean withAutoTimestamp) {
SqlBatchWriter(String type, SerializableFunction<T, Object> jpaConverter) {
counter = Metrics.counter("SQL_WRITE", type);
this.jpaConverter = jpaConverter;
this.withAutoTimestamp = withAutoTimestamp;
}
@ProcessElement
public void processElement(@Element KV<ShardedKey<Integer>, Iterable<T>> kv) {
if (withAutoTimestamp) {
actuallyProcessElement(kv);
return;
}
try (DisableAutoUpdateResource disable = UpdateAutoTimestamp.disableAutoUpdate()) {
actuallyProcessElement(kv);
}
actuallyProcessElement(kv);
}
private void actuallyProcessElement(@Element KV<ShardedKey<Integer>, Iterable<T>> kv) {
@@ -405,7 +382,13 @@ public final class RegistryJpaIO {
.filter(Objects::nonNull)
.collect(ImmutableList.toImmutableList());
try {
jpaTm().transact(() -> jpaTm().putAll(entities));
jpaTm()
.transact(
() -> {
// Don't modify existing objects as it could lead to race conditions
entities.forEach(this::verifyObjectNonexistence);
jpaTm().putAll(entities);
});
counter.inc(entities.size());
} catch (RuntimeException e) {
processSingly(entities);
@@ -419,7 +402,13 @@ public final class RegistryJpaIO {
private void processSingly(ImmutableList<Object> entities) {
for (Object entity : entities) {
try {
jpaTm().transact(() -> jpaTm().put(entity));
jpaTm()
.transact(
() -> {
// Don't modify existing objects as it could lead to race conditions
verifyObjectNonexistence(entity);
jpaTm().put(entity);
});
counter.inc();
} catch (RuntimeException e) {
throw new RuntimeException(toEntityKeyString(entity), e);
@@ -445,5 +434,16 @@ public final class RegistryJpaIO {
return "Non-SqlEntity: " + entity;
}
}
/** SqlBatchWriter should not re-write existing entities due to potential race conditions. */
private void verifyObjectNonexistence(Object obj) {
// We cannot rely on calling "insert" on the objects because the underlying JPA persist call
// adds the input object to the persistence context, meaning that any modifications (e.g.
// updateTimestamp) are reflected in the input object. Beam doesn't allow modification of
// input objects, so this throws an exception.
// TODO(go/non-datastore-allocateid): also check that all the objects have IDs
checkArgument(
!jpaTm().exists(obj), "Entities created in SqlBatchWriter must not already exist");
}
}
}
@@ -20,9 +20,7 @@ import dagger.Lazy;
import google.registry.config.CredentialModule;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.model.domain.Domain;
import google.registry.persistence.PersistenceModule;
import google.registry.persistence.PersistenceModule.BeamBulkQueryJpaTm;
import google.registry.persistence.PersistenceModule.BeamJpaTm;
import google.registry.persistence.PersistenceModule.BeamReadOnlyReplicaJpaTm;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
@@ -52,14 +50,6 @@ public interface RegistryPipelineComponent {
@BeamJpaTm
Lazy<JpaTransactionManager> getJpaTransactionManager();
/**
* Returns a {@link JpaTransactionManager} optimized for bulk loading multi-level JPA entities
* ({@link Domain} and {@link google.registry.model.domain.DomainHistory}). Please refer to {@link
* google.registry.model.bulkquery.BulkQueryEntities} for more information.
*/
@BeamBulkQueryJpaTm
Lazy<JpaTransactionManager> getBulkQueryJpaTransactionManager();
/**
* A {@link JpaTransactionManager} that uses the Postgres read-only replica if configured (uses
* the standard DB otherwise).
@@ -16,6 +16,7 @@ package google.registry.beam.common;
import google.registry.beam.common.RegistryJpaIO.Write;
import google.registry.config.RegistryEnvironment;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.persistence.PersistenceModule.JpaTransactionManagerType;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import java.util.Objects;
@@ -65,6 +66,17 @@ public interface RegistryPipelineOptions extends GcpOptions {
void setSqlWriteShards(int maxConcurrentSqlWriters);
@DeleteAfterMigration
@Description(
"Whether to use self allocated primary IDs when building entities. This should only be used"
+ " when the IDs are not significant and the resulting entities are not persisted back to"
+ " the database. Use with caution as self allocated IDs are not unique across workers,"
+ " and persisting entities with these IDs can be dangerous.")
@Default.Boolean(false)
boolean getUseSelfAllocatedId();
void setUseSelfAllocatedId(boolean useSelfAllocatedId);
static RegistryPipelineComponent toRegistryPipelineComponent(RegistryPipelineOptions options) {
return DaggerRegistryPipelineComponent.builder()
.isolationOverride(options.getIsolationOverride())
@@ -22,6 +22,8 @@ import dagger.Lazy;
import google.registry.config.RegistryEnvironment;
import google.registry.config.SystemPropertySetter;
import google.registry.model.AppEngineEnvironment;
import google.registry.model.IdService;
import google.registry.model.IdService.SelfAllocatedIdSupplier;
import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.persistence.transaction.TransactionManagerFactory;
import org.apache.beam.sdk.harness.JvmInitializer;
@@ -53,9 +55,6 @@ public class RegistryPipelineWorkerInitializer implements JvmInitializer {
toRegistryPipelineComponent(registryOptions);
Lazy<JpaTransactionManager> transactionManagerLazy;
switch (registryOptions.getJpaTransactionManagerType()) {
case BULK_QUERY:
transactionManagerLazy = registryPipelineComponent.getBulkQueryJpaTransactionManager();
break;
case READ_ONLY_REPLICA:
transactionManagerLazy =
registryPipelineComponent.getReadOnlyReplicaJpaTransactionManager();
@@ -65,12 +64,20 @@ public class RegistryPipelineWorkerInitializer implements JvmInitializer {
transactionManagerLazy = registryPipelineComponent.getJpaTransactionManager();
}
TransactionManagerFactory.setJpaTmOnBeamWorker(transactionManagerLazy::get);
// Masquerade all threads as App Engine threads so we can create Ofy keys in the pipeline. Also
// Masquerade all threads as App Engine threads, so we can create Ofy keys in the pipeline. Also
// loads all ofy entities.
new AppEngineEnvironment("s~" + registryPipelineComponent.getProjectId())
.setEnvironmentForAllThreads();
// Set the system property so that we can call IdService.allocateId() without access to
// datastore.
SystemPropertySetter.PRODUCTION_IMPL.setProperty(PROPERTY, "true");
// Use self-allocated IDs if requested. Note that this inevitably results in duplicate IDs from
// multiple workers, which can also collide with existing IDs in the database. So they cannot be
// dependent upon for comparison or anything significant. The resulting entities can never be
// persisted back into the database. This is a stop-gap measure that should only be used when
// you need to create Buildables in Beam, but do not have control over how the IDs are
// allocated, and you don't care about the generated IDs as long
// as you can build the entities.
if (registryOptions.getUseSelfAllocatedId()) {
IdService.setIdSupplier(SelfAllocatedIdSupplier.getInstance());
}
}
}
@@ -37,6 +37,7 @@ import java.util.Optional;
import java.util.regex.Pattern;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.PipelineResult;
import org.apache.beam.sdk.coders.SerializableCoder;
import org.apache.beam.sdk.coders.StringUtf8Coder;
import org.apache.beam.sdk.io.FileIO;
import org.apache.beam.sdk.io.TextIO;
@@ -93,8 +94,9 @@ public class InvoicingPipeline implements Serializable {
static PCollection<BillingEvent> readFromCloudSql(
InvoicingPipelineOptions options, Pipeline pipeline) {
Read<Object[], BillingEvent> read =
RegistryJpaIO.read(
makeCloudSqlQuery(options.getYearMonth()), false, row -> parseRow(row).orElse(null));
RegistryJpaIO.<Object[], BillingEvent>read(
makeCloudSqlQuery(options.getYearMonth()), false, row -> parseRow(row).orElse(null))
.withCoder(SerializableCoder.of(BillingEvent.class));
PCollection<BillingEvent> billingEventsWithNulls =
pipeline.apply("Read BillingEvents from Cloud SQL", read);
@@ -55,7 +55,7 @@ import google.registry.model.rde.RdeMode;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.Registrar.Type;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.HistoryEntryDao;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.persistence.VKey;
import google.registry.rde.DepositFragment;
@@ -71,11 +71,9 @@ import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;
import java.lang.reflect.InvocationTargetException;
import java.util.HashSet;
import javax.inject.Inject;
import javax.inject.Singleton;
import javax.persistence.IdClass;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.PipelineResult;
import org.apache.beam.sdk.coders.KvCoder;
@@ -128,7 +126,7 @@ import org.joda.time.DateTime;
* <h2>{@link EppResource}</h2>
*
* All EPP resources are loaded from the corresponding {@link HistoryEntry}, which has the resource
* embedded. In general we find most recent history entry before watermark and filter out the ones
* embedded. In general, we find most recent history entry before watermark and filter out the ones
* that are soft-deleted by watermark. The history is emitted as pairs of (resource repo ID: history
* revision ID) from the SQL query.
*
@@ -164,7 +162,7 @@ import org.joda.time.DateTime;
*
* The (pending deposit: deposit fragment) pairs from different resources are combined and grouped
* by pending deposit. For each pending deposit, all the relevant deposit fragments are written into
* a encrypted file stored on GCS. The filename is uniquely determined by the Beam job ID so there
* an encrypted file stored on GCS. The filename is uniquely determined by the Beam job ID so there
* is no need to lock the GCS write operation to prevent stomping. The cursor for staging the
* pending deposit is then rolled forward, and the next action is enqueued. The latter two
* operations are performed in a transaction so the cursor is rolled back if enqueueing failed.
@@ -172,6 +170,7 @@ import org.joda.time.DateTime;
* @see <a href="https://cloud.google.com/dataflow/docs/guides/templates/using-flex-templates">Using
* Flex Templates</a>
*/
@SuppressWarnings("ALL")
@Singleton
public class RdePipeline implements Serializable {
@@ -191,16 +190,6 @@ public class RdePipeline implements Serializable {
private static final ImmutableSet<Type> IGNORED_REGISTRAR_TYPES =
Sets.immutableEnumSet(Registrar.Type.MONITORING, Registrar.Type.TEST);
// The field name of the EPP resource embedded in its corresponding history entry.
private static final ImmutableMap<Class<? extends HistoryEntry>, String> EPP_RESOURCE_FIELD_NAME =
ImmutableMap.of(
DomainHistory.class,
"domainBase",
ContactHistory.class,
"contactBase",
HostHistory.class,
"hostBase");
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
@Inject
@@ -301,10 +290,11 @@ public class RdePipeline implements Serializable {
.apply(
"Read all production Registrars",
RegistryJpaIO.read(
"SELECT registrarId FROM Registrar WHERE type NOT IN (:types)",
ImmutableMap.of("types", IGNORED_REGISTRAR_TYPES),
String.class,
id -> VKey.createSql(Registrar.class, id)))
"SELECT registrarId FROM Registrar WHERE type NOT IN (:types)",
ImmutableMap.of("types", IGNORED_REGISTRAR_TYPES),
String.class,
x -> x)
.withCoder(StringUtf8Coder.of()))
.apply(
"Marshall Registrar into DepositFragment",
FlatMapElements.into(
@@ -312,7 +302,8 @@ public class RdePipeline implements Serializable {
TypeDescriptor.of(PendingDeposit.class),
TypeDescriptor.of(DepositFragment.class)))
.via(
(VKey<Registrar> key) -> {
(String registrarRepoId) -> {
VKey<Registrar> key = VKey.create(Registrar.class, registrarRepoId);
includedRegistrarCounter.inc();
Registrar registrar = jpaTm().transact(() -> jpaTm().loadByKey(key));
DepositFragment fragment = marshaller.marshalRegistrar(registrar);
@@ -335,31 +326,24 @@ public class RdePipeline implements Serializable {
*/
private <T extends HistoryEntry> PCollection<KV<String, Long>> getMostRecentHistoryEntries(
Pipeline pipeline, Class<T> historyClass) {
String repoIdFieldName = HistoryEntryDao.REPO_ID_FIELD_NAMES.get(historyClass);
String resourceFieldName = EPP_RESOURCE_FIELD_NAME.get(historyClass);
return pipeline
.apply(
String.format("Load most recent %s", historyClass.getSimpleName()),
RegistryJpaIO.read(
("SELECT %repoIdField%, id FROM %entity% WHERE (%repoIdField%, modificationTime)"
+ " IN (SELECT %repoIdField%, MAX(modificationTime) FROM %entity% WHERE"
+ " modificationTime <= :watermark GROUP BY %repoIdField%) AND"
+ " %resourceField%.deletionTime > :watermark AND"
+ " COALESCE(%resourceField%.creationClientId, '') NOT LIKE 'prober-%' AND"
+ " COALESCE(%resourceField%.currentSponsorClientId, '') NOT LIKE 'prober-%'"
+ " AND COALESCE(%resourceField%.lastEppUpdateClientId, '') NOT LIKE"
return pipeline.apply(
String.format("Load most recent %s", historyClass.getSimpleName()),
RegistryJpaIO.read(
("SELECT repoId, revisionId FROM %entity% WHERE (repoId, modificationTime) IN"
+ " (SELECT repoId, MAX(modificationTime) FROM %entity% WHERE"
+ " modificationTime <= :watermark GROUP BY repoId) AND resource.deletionTime"
+ " > :watermark AND COALESCE(resource.creationRegistrarId, '') NOT LIKE"
+ " 'prober-%' AND COALESCE(resource.currentSponsorRegistrarId, '') NOT LIKE"
+ " 'prober-%' AND COALESCE(resource.lastEppUpdateRegistrarId, '') NOT LIKE"
+ " 'prober-%' "
+ (historyClass == DomainHistory.class
? "AND %resourceField%.tld IN "
+ "(SELECT id FROM Tld WHERE tldType = 'REAL')"
? "AND resource.tld IN " + "(SELECT id FROM Tld WHERE tldType = 'REAL')"
: ""))
.replace("%entity%", historyClass.getSimpleName())
.replace("%repoIdField%", repoIdFieldName)
.replace("%resourceField%", resourceFieldName),
.replace("%entity%", historyClass.getSimpleName()),
ImmutableMap.of("watermark", watermark),
Object[].class,
row -> KV.of((String) row[0], (long) row[1])))
.setCoder(KvCoder.of(StringUtf8Coder.of(), VarLongCoder.of()));
row -> KV.of((String) row[0], (long) row[1]))
.withCoder(KvCoder.of(StringUtf8Coder.of(), VarLongCoder.of())));
}
private <T extends HistoryEntry> EppResource loadResourceByHistoryEntryId(
@@ -379,38 +363,28 @@ public class RdePipeline implements Serializable {
checkState(
dedupedIds.size() == 1,
"Multiple unique revision IDs detected for %s repo ID %s: %s",
EPP_RESOURCE_FIELD_NAME.get(historyEntryClazz),
historyEntryClazz.getSimpleName(),
repoId,
ids);
logger.atSevere().log(
"Duplicate revision IDs detected for %s repo ID %s: %s",
EPP_RESOURCE_FIELD_NAME.get(historyEntryClazz), repoId, ids);
historyEntryClazz.getSimpleName(), repoId, ids);
}
return loadResourceByHistoryEntryId(historyEntryClazz, repoId, ids.get(0));
}
private <T extends HistoryEntry> EppResource loadResourceByHistoryEntryId(
Class<T> historyEntryClazz, String repoId, long revisionId) {
try {
Class<?> idClazz = historyEntryClazz.getAnnotation(IdClass.class).value();
Serializable idObject =
(Serializable)
idClazz.getConstructor(String.class, long.class).newInstance(repoId, revisionId);
return jpaTm()
.transact(() -> jpaTm().loadByKey(VKey.createSql(historyEntryClazz, idObject)))
.getResourceAtPointInTime()
.map(resource -> resource.cloneProjectedAtTime(watermark))
.get();
} catch (NoSuchMethodException
| InvocationTargetException
| InstantiationException
| IllegalAccessException e) {
throw new RuntimeException(
String.format(
"Cannot load resource from %s with repoId %s and revisionId %s",
historyEntryClazz.getSimpleName(), repoId, revisionId),
e);
}
return jpaTm()
.transact(
() ->
jpaTm()
.loadByKey(
VKey.create(historyEntryClazz, new HistoryEntryId(repoId, revisionId))))
.getResourceAtPointInTime()
.map(resource -> resource.cloneProjectedAtTime(watermark))
.get();
}
/**
@@ -495,12 +469,12 @@ public class RdePipeline implements Serializable {
// Contacts and hosts are only deposited in RDE, not BRDA.
if (pendingDeposit.mode() == RdeMode.FULL) {
HashSet<Serializable> contacts = new HashSet<>();
contacts.add(domain.getAdminContact().getSqlKey());
contacts.add(domain.getTechContact().getSqlKey());
contacts.add(domain.getRegistrant().getSqlKey());
contacts.add(domain.getAdminContact().getKey());
contacts.add(domain.getTechContact().getKey());
contacts.add(domain.getRegistrant().getKey());
// Billing contact is not mandatory.
if (domain.getBillingContact() != null) {
contacts.add(domain.getBillingContact().getSqlKey());
contacts.add(domain.getBillingContact().getKey());
}
referencedContactCounter.inc(contacts.size());
contacts.forEach(
@@ -518,7 +492,7 @@ public class RdePipeline implements Serializable {
.get(REFERENCED_HOSTS)
.output(
KV.of(
(String) hostKey.getSqlKey(),
(String) hostKey.getKey(),
pendingDeposit)));
}
}
@@ -591,7 +565,7 @@ public class RdePipeline implements Serializable {
// The output are pairs of
// (superordinateDomainRepoId,
// (subordinateHostRepoId, (pendingDeposit, revisionId))).
KV.of((String) host.getSuperordinateDomain().getSqlKey(), kv));
KV.of((String) host.getSuperordinateDomain().getKey(), kv));
} else {
externalHostCounter.inc();
DepositFragment fragment = marshaller.marshalExternalHost(host);
@@ -698,8 +672,8 @@ public class RdePipeline implements Serializable {
}
/**
* Encodes the pending deposit set in an URL safe string that is sent to the pipeline worker by
* the pipeline launcher as a pipeline option.
* Encodes the pending deposit set in a URL safe string that is sent to the pipeline worker by the
* pipeline launcher as a pipeline option.
*/
public static String encodePendingDeposits(ImmutableSet<PendingDeposit> pendingDeposits)
throws IOException {
@@ -715,6 +689,12 @@ public class RdePipeline implements Serializable {
PipelineOptionsFactory.register(RdePipelineOptions.class);
RdePipelineOptions options =
PipelineOptionsFactory.fromArgs(args).withValidation().as(RdePipelineOptions.class);
// We need to self allocate the IDs because the pipeline creates EPP resources from history
// entries and projects them to watermark. These buildable entities would otherwise request an
// ID from datastore, which Beam does not have access to. The IDs are not included in the
// deposits or are these entities persisted back to the database, so it is OK to use a self
// allocated ID to get around the limitations of beam.
options.setUseSelfAllocatedId(true);
RegistryPipelineOptions.validateRegistryPipelineOptions(options);
options.setIsolationOverride(TransactionIsolationLevel.TRANSACTION_READ_COMMITTED);
DaggerRdePipeline_RdePipelineComponent.builder().options(options).build().rdePipeline().run();
@@ -36,6 +36,7 @@ import java.io.Serializable;
import java.util.concurrent.ThreadLocalRandom;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.PipelineResult;
import org.apache.beam.sdk.coders.StringUtf8Coder;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.GroupIntoBatches;
@@ -103,10 +104,11 @@ public class ResaveAllEppResourcesPipeline implements Serializable {
private void fastResaveContacts(Pipeline pipeline) {
Read<String, String> repoIdRead =
RegistryJpaIO.read(
"SELECT repoId FROM Contact WHERE transferData.transferStatus = 'PENDING' AND"
+ " transferData.pendingTransferExpirationTime < current_timestamp()",
String.class,
r -> r);
"SELECT repoId FROM Contact WHERE transferData.transferStatus = 'PENDING' AND"
+ " transferData.pendingTransferExpirationTime < current_timestamp()",
String.class,
r -> r)
.withCoder(StringUtf8Coder.of());
projectAndResaveResources(pipeline, Contact.class, repoIdRead);
}
@@ -120,10 +122,11 @@ public class ResaveAllEppResourcesPipeline implements Serializable {
private void fastResaveDomains(Pipeline pipeline) {
Read<String, String> repoIdRead =
RegistryJpaIO.read(
DOMAINS_TO_PROJECT_QUERY,
ImmutableMap.of("END_OF_TIME", DateTimeUtils.END_OF_TIME),
String.class,
r -> r);
DOMAINS_TO_PROJECT_QUERY,
ImmutableMap.of("END_OF_TIME", DateTimeUtils.END_OF_TIME),
String.class,
r -> r)
.withCoder(StringUtf8Coder.of());
projectAndResaveResources(pipeline, Domain.class, repoIdRead);
}
@@ -131,8 +134,9 @@ public class ResaveAllEppResourcesPipeline implements Serializable {
private <T extends EppResource> void forceResaveAllResources(Pipeline pipeline, Class<T> clazz) {
Read<String, String> repoIdRead =
RegistryJpaIO.read(
// Note: cannot use SQL parameters for the table name
String.format("SELECT repoId FROM %s", clazz.getSimpleName()), String.class, r -> r);
// Note: cannot use SQL parameters for the table name
String.format("SELECT repoId FROM %s", clazz.getSimpleName()), String.class, r -> r)
.withCoder(StringUtf8Coder.of());
projectAndResaveResources(pipeline, clazz, repoIdRead);
}
@@ -37,6 +37,8 @@ import java.io.Serializable;
import javax.inject.Singleton;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.PipelineResult;
import org.apache.beam.sdk.coders.KvCoder;
import org.apache.beam.sdk.coders.StringUtf8Coder;
import org.apache.beam.sdk.io.TextIO;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.apache.beam.sdk.transforms.DoFn;
@@ -112,11 +114,12 @@ public class Spec11Pipeline implements Serializable {
static PCollection<DomainNameInfo> readFromCloudSql(Pipeline pipeline) {
Read<Object[], KV<String, String>> read =
RegistryJpaIO.read(
"select d.repoId, r.emailAddress from Domain d join Registrar r on"
+ " d.currentSponsorClientId = r.registrarId where r.type = 'REAL' and"
+ " d.deletionTime > now()",
false,
Spec11Pipeline::parseRow);
"select d.repoId, r.emailAddress from Domain d join Registrar r on"
+ " d.currentSponsorRegistrarId = r.registrarId where r.type = 'REAL' and"
+ " d.deletionTime > now()",
false,
Spec11Pipeline::parseRow)
.withCoder(KvCoder.of(StringUtf8Coder.of(), StringUtf8Coder.of()));
return pipeline
.apply("Read active domains from Cloud SQL", read)
@@ -130,9 +133,7 @@ public class Spec11Pipeline implements Serializable {
Domain domain =
jpaTm()
.transact(
() ->
jpaTm()
.loadByKey(VKey.createSql(Domain.class, input.getKey())));
() -> jpaTm().loadByKey(VKey.create(Domain.class, input.getKey())));
String emailAddress = input.getValue();
if (emailAddress == null) {
emailAddress = "";
@@ -214,8 +215,7 @@ public class Spec11Pipeline implements Serializable {
return output.toString();
} catch (JSONException e) {
throw new RuntimeException(
String.format(
"Encountered an error constructing the JSON for %s", kv.toString()),
String.format("Encountered an error constructing the JSON for %s", kv),
e);
}
}))
@@ -14,15 +14,17 @@
package google.registry.config;
import static com.google.common.base.Preconditions.checkArgument;
import static java.nio.charset.StandardCharsets.UTF_8;
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.auth.ServiceAccountSigner;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.common.collect.ImmutableList;
import dagger.Module;
import dagger.Provides;
import google.registry.config.RegistryConfig.Config;
import google.registry.keyring.api.KeyModule.Key;
import google.registry.util.Clock;
import google.registry.util.GoogleCredentialsBundle;
import java.io.ByteArrayInputStream;
import java.io.IOException;
@@ -30,6 +32,7 @@ import java.io.UncheckedIOException;
import java.lang.annotation.Documented;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.time.Duration;
import javax.inject.Qualifier;
import javax.inject.Singleton;
@@ -37,6 +40,36 @@ import javax.inject.Singleton;
@Module
public abstract class CredentialModule {
/**
* Provides a {@link GoogleCredentialsBundle} backed by the application default credential from
* the Google Cloud Runtime. This credential may be used to access GCP APIs that are NOT part of
* the Google Workspace.
*
* <p>The credential returned by the Cloud Runtime depends on the runtime environment:
*
* <ul>
* <li>On App Engine, returns a scope-less {@code ComputeEngineCredentials} for
* PROJECT_ID@appspot.gserviceaccount.com
* <li>On Compute Engine, returns a scope-less {@code ComputeEngineCredentials} for
* PROJECT_NUMBER-compute@developer.gserviceaccount.com
* <li>On end user host, this returns the credential downloaded by gcloud. Please refer to <a
* href="https://cloud.google.com/sdk/gcloud/reference/auth/application-default/login">Cloud
* SDK documentation</a> for details.
* </ul>
*/
@ApplicationDefaultCredential
@Provides
@Singleton
public static GoogleCredentialsBundle provideApplicationDefaultCredential() {
GoogleCredentials credential;
try {
credential = GoogleCredentials.getApplicationDefault();
} catch (IOException e) {
throw new RuntimeException(e);
}
return GoogleCredentialsBundle.create(credential);
}
/**
* Provides the default {@link GoogleCredentialsBundle} from the Google Cloud runtime.
*
@@ -70,26 +103,19 @@ public abstract class CredentialModule {
}
/**
* Provides the default {@link GoogleCredential} from the Google Cloud runtime for G Suite
* Drive API.
* TODO(b/138195359): Deprecate this credential once we figure out how to use
* {@link GoogleCredentials} for G Suite Drive API.
* Provides a {@link GoogleCredentialsBundle} for accessing Google Workspace APIs, such as Drive
* and Sheets.
*/
@GSuiteDriveCredential
@GoogleWorkspaceCredential
@Provides
@Singleton
public static GoogleCredential provideGSuiteDriveCredential(
public static GoogleCredentialsBundle provideGSuiteDriveCredential(
@ApplicationDefaultCredential GoogleCredentialsBundle applicationDefaultCredential,
@Config("defaultCredentialOauthScopes") ImmutableList<String> requiredScopes) {
GoogleCredential credential;
try {
credential = GoogleCredential.getApplicationDefault();
} catch (IOException e) {
throw new RuntimeException(e);
}
if (credential.createScopedRequired()) {
credential = credential.createScoped(requiredScopes);
}
return credential;
GoogleCredentials credential = applicationDefaultCredential.getGoogleCredentials();
// Although credential is scope-less, its `createScopedRequired` method still returns false.
credential = credential.createScoped(requiredScopes);
return GoogleCredentialsBundle.create(credential);
}
/**
@@ -136,18 +162,66 @@ public abstract class CredentialModule {
.createScoped(requiredScopes));
}
/**
* Provides a {@link GoogleCredentialsBundle} with delegated access to Google Workspace APIs for
* the application default credential user.
*
* <p>The Workspace domain must grant delegated admin access to the default service account user
* (project-id@appspot.gserviceaccount.com on AppEngine) with all scopes in {@code defaultScopes}
* and {@code delegationScopes}.
*/
@AdcDelegatedCredential
@Provides
@Singleton
public static GoogleCredentialsBundle provideSelfSignedDelegatedCredential(
@Config("defaultCredentialOauthScopes") ImmutableList<String> defaultScopes,
@Config("delegatedCredentialOauthScopes") ImmutableList<String> delegationScopes,
@ApplicationDefaultCredential GoogleCredentialsBundle credentialsBundle,
@Config("gSuiteAdminAccountEmailAddress") String gSuiteAdminAccountEmailAddress,
@Config("tokenRefreshDelay") Duration tokenRefreshDelay,
Clock clock) {
GoogleCredentials signer = credentialsBundle.getGoogleCredentials();
checkArgument(
signer instanceof ServiceAccountSigner,
"Expecting a ServiceAccountSigner, found %s.",
signer.getClass().getSimpleName());
try {
// Refreshing as sanity check on the ADC.
signer.refresh();
} catch (IOException e) {
throw new RuntimeException("Cannot refresh the ApplicationDefaultCredential", e);
}
DelegatedCredentials credential =
DelegatedCredentials.createSelfSignedDelegatedCredential(
(ServiceAccountSigner) signer,
ImmutableList.<String>builder().addAll(defaultScopes).addAll(delegationScopes).build(),
gSuiteAdminAccountEmailAddress,
clock,
tokenRefreshDelay);
return GoogleCredentialsBundle.create(credential);
}
/** Dagger qualifier for the scope-less Application Default Credential. */
@Qualifier
@Documented
@Retention(RetentionPolicy.RUNTIME)
public @interface ApplicationDefaultCredential {}
/** Dagger qualifier for the Application Default Credential. */
@Qualifier
@Documented
@Retention(RetentionPolicy.RUNTIME)
@Deprecated // Switching to @ApplicationDefaultCredential
public @interface DefaultCredential {}
/** Dagger qualifier for the credential for G Suite Drive API. */
/** Dagger qualifier for the credential for Google Workspace APIs. */
@Qualifier
@Documented
@Retention(RetentionPolicy.RUNTIME)
public @interface GSuiteDriveCredential {}
public @interface GoogleWorkspaceCredential {}
/**
* Dagger qualifier for a credential from a service account's JSON key, to be used in non-request
@@ -167,6 +241,15 @@ public abstract class CredentialModule {
@Retention(RetentionPolicy.RUNTIME)
public @interface DelegatedCredential {}
/**
* Dagger qualifier for a credential with delegated admin access for a dasher domain (for Google
* Workspace) backed by the application default credential (ADC).
*/
@Qualifier
@Documented
@Retention(RetentionPolicy.RUNTIME)
public @interface AdcDelegatedCredential {}
/** Dagger qualifier for the local credential used in the nomulus tool. */
@Qualifier
@Documented
@@ -0,0 +1,268 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.config;
import static com.google.common.base.Preconditions.checkArgument;
import com.google.api.client.http.GenericUrl;
import com.google.api.client.http.HttpBackOffIOExceptionHandler;
import com.google.api.client.http.HttpBackOffUnsuccessfulResponseHandler;
import com.google.api.client.http.HttpRequest;
import com.google.api.client.http.HttpRequestFactory;
import com.google.api.client.http.HttpResponse;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.http.UrlEncodedContent;
import com.google.api.client.http.javanet.NetHttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.JsonObjectParser;
import com.google.api.client.json.gson.GsonFactory;
import com.google.api.client.json.webtoken.JsonWebSignature;
import com.google.api.client.json.webtoken.JsonWebToken;
import com.google.api.client.util.ExponentialBackOff;
import com.google.api.client.util.GenericData;
import com.google.api.client.util.StringUtils;
import com.google.auth.ServiceAccountSigner;
import com.google.auth.http.HttpTransportFactory;
import com.google.auth.oauth2.AccessToken;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Iterables;
import google.registry.util.Clock;
import java.io.IOException;
import java.math.BigDecimal;
import java.time.Duration;
import java.util.Collection;
import java.util.Date;
import java.util.Map;
import java.util.ServiceLoader;
import org.apache.commons.codec.binary.Base64;
/**
* OAuth2 credentials for accessing Google Workspace APIs with domain-wide delegation. It fetches
* access tokens using JSON Web Tokens (JWT) signed by a user-provided {@link ServiceAccountSigner}.
*
* <p>This class accepts the application-default-credential as {@code ServiceAccountSigner},
* avoiding the need for exported private keys. In this case, the default credential user itself
* (project-id@appspot.gserviceaccount.com on AppEngine) must have domain-wide delegation to the
* Workspace APIs. The default credential user also must have the Token Creator role to itself.
*
* <p>If the user provides a credential {@code S} that carries its own private key, such as {@link
* com.google.auth.oauth2.ServiceAccountCredentials}, this class can use {@code S} to impersonate
* another service account {@code D} and gain delegated access as {@code D}, as long as S has the
* Token Creator role to {@code D}. This usage is documented here for future reference.
*
* <p>As of October 2022, the functionalities described above are not implemented in the GCP Java
* Auth library, although they are available in the Python library. We have filed a <a
* href="https://github.com/googleapis/google-auth-library-java/issues/1064">feature request</a>.
* This class is a stop-gap implementation.
*
* <p>The main body of this class is adapted from {@link
* com.google.auth.oauth2.ServiceAccountCredentials} with cosmetic changes. The important changes
* include the removal of all uses of the private key and the signing of the JWT (in {@link
* #signAssertion}). We choose not to extend {@code ServiceAccountCredentials} because it would add
* dependency to the non-public details of that class.
*/
public class DelegatedCredentials extends GoogleCredentials {
private static final long serialVersionUID = 617127523756785546L;
private static final String DEFAULT_TOKEN_URI = "https://accounts.google.com/o/oauth2/token";
private static final String GRANT_TYPE = "urn:ietf:params:oauth:grant-type:jwt-bearer";
private static final JsonFactory JSON_FACTORY = GsonFactory.getDefaultInstance();
private static final HttpTransport HTTP_TRANSPORT = new NetHttpTransport();
private static final String VALUE_NOT_FOUND_MESSAGE = "%sExpected value %s not found.";
private static final String VALUE_WRONG_TYPE_MESSAGE = "%sExpected %s value %s of wrong type.";
private static final String PARSE_ERROR_PREFIX = "Error parsing token refresh response. ";
private static final Duration MAX_TOKEN_REFRESH_DELAY = Duration.ofHours(1);
private final ServiceAccountSigner signer;
private final String delegatedServiceAccountUser;
private final ImmutableList<String> scopes;
private final String delegatingUserEmail;
private final Clock clock;
private final Duration tokenRefreshDelay;
private final HttpTransportFactory transportFactory;
/**
* Creates a {@link DelegatedCredentials} instance that is self-signed by the signer, which must
* have delegated access to the Workspace APIs.
*
* @param signer Signs for the generated JWT tokens. This may be the application default
* credential
* @param scopes The scopes to use when generating JWT tokens
* @param delegatingUserEmail The Workspace user whose permissions are delegated to the signer
* @param clock Used for setting token expiration times.
* @param tokenRefreshDelay The lifetime of each token. Should not exceed one hour according to
* GCP recommendations.
* @return
*/
static DelegatedCredentials createSelfSignedDelegatedCredential(
ServiceAccountSigner signer,
Collection<String> scopes,
String delegatingUserEmail,
Clock clock,
Duration tokenRefreshDelay) {
return new DelegatedCredentials(
signer, signer.getAccount(), scopes, delegatingUserEmail, clock, tokenRefreshDelay);
}
private DelegatedCredentials(
ServiceAccountSigner signer,
String delegatedServiceAccountUser,
Collection<String> scopes,
String delegatingUserEmail,
Clock clock,
Duration tokenRefreshDelay) {
checkArgument(
tokenRefreshDelay.getSeconds() <= MAX_TOKEN_REFRESH_DELAY.getSeconds(),
"Max refresh delay must not exceed %s.",
MAX_TOKEN_REFRESH_DELAY);
this.signer = signer;
this.delegatedServiceAccountUser = delegatedServiceAccountUser;
this.scopes = ImmutableList.copyOf(scopes);
this.delegatingUserEmail = delegatingUserEmail;
this.clock = clock;
this.tokenRefreshDelay = tokenRefreshDelay;
this.transportFactory =
getFromServiceLoader(
HttpTransportFactory.class, DelegatedCredentials::provideHttpTransport);
}
/**
* Refreshes the OAuth2 access token by getting a new access token using a JSON Web Token (JWT).
*/
@Override
public AccessToken refreshAccessToken() throws IOException {
JsonFactory jsonFactory = JSON_FACTORY;
long currentTime = clock.nowUtc().getMillis();
String assertion = createAssertion(jsonFactory, currentTime);
GenericData tokenRequest = new GenericData();
tokenRequest.set("grant_type", GRANT_TYPE);
tokenRequest.set("assertion", assertion);
UrlEncodedContent content = new UrlEncodedContent(tokenRequest);
HttpRequestFactory requestFactory = transportFactory.create().createRequestFactory();
HttpRequest request =
requestFactory.buildPostRequest(new GenericUrl(DEFAULT_TOKEN_URI), content);
request.setParser(new JsonObjectParser(jsonFactory));
request.setIOExceptionHandler(new HttpBackOffIOExceptionHandler(new ExponentialBackOff()));
request.setUnsuccessfulResponseHandler(
new HttpBackOffUnsuccessfulResponseHandler(new ExponentialBackOff())
.setBackOffRequired(
response -> {
int code = response.getStatusCode();
return (
// Server error --- includes timeout errors, which use 500 instead of 408
code / 100 == 5
// Forbidden error --- for historical reasons, used for rate_limit_exceeded
// errors instead of 429, but there currently seems no robust automatic way
// to
// distinguish these cases: see
// https://github.com/google/google-api-java-client/issues/662
|| code == 403);
}));
HttpResponse response;
try {
response = request.execute();
} catch (IOException e) {
throw new IOException(
String.format("Error getting access token for service account: %s", e.getMessage()), e);
}
GenericData responseData = response.parseAs(GenericData.class);
String accessToken = validateString(responseData, "access_token", PARSE_ERROR_PREFIX);
int expiresInSeconds = validateInt32(responseData, "expires_in", PARSE_ERROR_PREFIX);
long expiresAtMilliseconds = clock.nowUtc().getMillis() + expiresInSeconds * 1000L;
return new AccessToken(accessToken, new Date(expiresAtMilliseconds));
}
String createAssertion(JsonFactory jsonFactory, long currentTime) throws IOException {
JsonWebSignature.Header header = new JsonWebSignature.Header();
header.setAlgorithm("RS256");
header.setType("JWT");
JsonWebToken.Payload payload = new JsonWebToken.Payload();
payload.setIssuer(this.delegatedServiceAccountUser);
payload.setIssuedAtTimeSeconds(currentTime / 1000);
payload.setExpirationTimeSeconds(currentTime / 1000 + tokenRefreshDelay.getSeconds());
payload.setSubject(delegatingUserEmail);
payload.put("scope", Joiner.on(' ').join(scopes));
payload.setAudience(DEFAULT_TOKEN_URI);
return signAssertion(jsonFactory, header, payload);
}
String signAssertion(
JsonFactory jsonFactory, JsonWebSignature.Header header, JsonWebToken.Payload payload)
throws IOException {
String content =
Base64.encodeBase64URLSafeString(jsonFactory.toByteArray(header))
+ "."
+ Base64.encodeBase64URLSafeString(jsonFactory.toByteArray(payload));
byte[] contentBytes = StringUtils.getBytesUtf8(content);
byte[] signature = signer.sign(contentBytes); // Changed from ServiceAccountCredentials.
return content + "." + Base64.encodeBase64URLSafeString(signature);
}
static HttpTransport provideHttpTransport() {
return HTTP_TRANSPORT;
}
protected static <T> T getFromServiceLoader(Class<? extends T> clazz, T defaultInstance) {
return Iterables.getFirst(ServiceLoader.load(clazz), defaultInstance);
}
/** Return the specified string from JSON or throw a helpful error message. */
static String validateString(Map<String, Object> map, String key, String errorPrefix)
throws IOException {
Object value = map.get(key);
if (value == null) {
throw new IOException(String.format(VALUE_NOT_FOUND_MESSAGE, errorPrefix, key));
}
if (!(value instanceof String)) {
throw new IOException(String.format(VALUE_WRONG_TYPE_MESSAGE, errorPrefix, "string", key));
}
return (String) value;
}
/** Return the specified integer from JSON or throw a helpful error message. */
static int validateInt32(Map<String, Object> map, String key, String errorPrefix)
throws IOException {
Object value = map.get(key);
if (value == null) {
throw new IOException(String.format(VALUE_NOT_FOUND_MESSAGE, errorPrefix, key));
}
if (value instanceof BigDecimal) {
BigDecimal bigDecimalValue = (BigDecimal) value;
return bigDecimalValue.intValueExact();
}
if (!(value instanceof Integer)) {
throw new IOException(String.format(VALUE_WRONG_TYPE_MESSAGE, errorPrefix, "integer", key));
}
return (Integer) value;
}
}
@@ -1027,38 +1027,6 @@ public final class RegistryConfig {
return 50;
}
/**
* Returns the delay before executing async delete flow mapreduces.
*
* <p>This delay should be sufficiently longer than a transaction, to solve the following
* problem:
*
* <ul>
* <li>a domain mutation flow starts a transaction
* <li>the domain flow non-transactionally reads a resource and sees that it's not in
* PENDING_DELETE
* <li>the domain flow creates a new reference to this resource
* <li>a contact/host delete flow runs and marks the resource PENDING_DELETE and commits
* <li>the domain flow commits
* </ul>
*
* <p>Although we try not to add references to a PENDING_DELETE resource, strictly speaking that
* is ok as long as the mapreduce eventually sees the new reference (and therefore
* asynchronously fails the delete). Without this delay, the mapreduce might have started before
* the domain flow committed, and could potentially miss the reference.
*
* <p>If you are using EPP resource caching (eppResourceCachingEnabled in YAML), then this
* duration should also be longer than that cache duration (eppResourceCachingSeconds).
*
* @see google.registry.config.RegistryConfigSettings.Caching
* @see google.registry.batch.AsyncTaskEnqueuer
*/
@Provides
@Config("asyncDeleteDelay")
public static Duration provideAsyncDeleteDelay(RegistryConfigSettings config) {
return Duration.standardSeconds(config.misc.asyncDeleteDelaySeconds);
}
/**
* The server ID used in the 'svID' element of an EPP 'greeting'.
*
@@ -1076,24 +1044,6 @@ public final class RegistryConfig {
return config.keyring.activeKeyring;
}
/**
* The name to use for the Cloud KMS KeyRing containing encryption keys for Nomulus secrets.
*
* @see <a
* href="https://cloud.google.com/kms/docs/reference/rest/v1/projects.locations.keyRings#KeyRing">projects.locations.keyRings</a>
*/
@Provides
@Config("cloudKmsKeyRing")
public static String provideCloudKmsKeyRing(RegistryConfigSettings config) {
return config.keyring.kms.keyringName;
}
@Provides
@Config("cloudKmsProjectId")
public static String provideCloudKmsProjectId(RegistryConfigSettings config) {
return config.keyring.kms.projectId;
}
@Provides
@Config("customLogicFactoryClass")
public static String provideCustomLogicFactoryClass(RegistryConfigSettings config) {
@@ -1223,6 +1173,12 @@ public final class RegistryConfig {
return ImmutableList.copyOf(config.credentialOAuth.localCredentialOauthScopes);
}
@Provides
@Config("tokenRefreshDelay")
public static java.time.Duration provideTokenRefreshDelay(RegistryConfigSettings config) {
return java.time.Duration.ofSeconds(config.credentialOAuth.tokenRefreshDelaySeconds);
}
/** OAuth client ID used by the nomulus tool. */
@Provides
@Config("toolsClientId")
@@ -67,6 +67,7 @@ public class RegistryConfigSettings {
public List<String> defaultCredentialOauthScopes;
public List<String> delegatedCredentialOauthScopes;
public List<String> localCredentialOauthScopes;
public int tokenRefreshDelaySeconds;
}
/** Configuration options for the G Suite account used by Nomulus. */
@@ -206,13 +207,13 @@ public class RegistryConfigSettings {
public String alertRecipientEmailAddress;
public String spec11OutgoingEmailAddress;
public List<String> spec11BccEmailAddresses;
public int asyncDeleteDelaySeconds;
public int transientFailureRetries;
}
/** Configuration for keyrings (used to store secrets outside of source). */
public static class Keyring {
public String activeKeyring;
// TODO(b/257276342): Remove after config files in nomulus-internal are updated.
public Kms kms;
}
@@ -340,6 +340,9 @@ credentialOAuth:
- https://www.googleapis.com/auth/userinfo.email
# View and manage your applications deployed on Google App Engine
- https://www.googleapis.com/auth/appengine.admin
# The lifetime of an access token generated by our custom credentials classes
# Must be shorter than one hour.
tokenRefreshDelaySeconds: 1800
icannReporting:
# URL we PUT monthly ICANN transactions reports to.
@@ -418,11 +421,6 @@ misc:
spec11BccEmailAddresses:
- abuse@example.com
# How long to delay processing of asynchronous deletions. This should always
# be longer than eppResourceCachingSeconds, to prevent deleted contacts or
# hosts from being used on domains.
asyncDeleteDelaySeconds: 90
# Number of times to retry a GAE operation when a transient exception is thrown.
# The number of milliseconds it'll sleep before giving up is (2^n - 2) * 100.
transientFailureRetries: 12
@@ -446,7 +444,8 @@ beam:
stagingBucketUrl: gcs-bucket-with-staged-templates
keyring:
# The name of the active keyring, either "KMS" or "Dummy".
# The name of the active keyring, either "Dummy" or "CSM". The latter stands
# for Cloud SecretManager.
activeKeyring: Dummy
# Configuration options specific to Google Cloud KMS.
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>backend</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>default</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>pubapi</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>tools</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>
@@ -293,6 +293,12 @@ have been in the database for a certain period of time. -->
<url-pattern>/_dr/task/wipeOutCloudSql</url-pattern>
</servlet-mapping>
<!-- Action to execute canned scripts -->
<servlet-mapping>
<servlet-name>backend-servlet</servlet-name>
<url-pattern>/_dr/task/executeCannedScript</url-pattern>
</servlet-mapping>
<!-- Security config -->
<security-constraint>
<web-resource-collection>
@@ -18,86 +18,6 @@
</retry-parameters>
</queue>
<queue>
<name>async-delete-pull</name>
<mode>pull</mode>
</queue>
<queue>
<name>async-host-rename-pull</name>
<mode>pull</mode>
</queue>
<queue>
<name>export-commits</name>
<rate>10/s</rate>
<bucket-size>100</bucket-size>
<retry-parameters>
<!-- Retry aggressively since a single delayed export increases our time window of
unrecoverable data loss in the event of a Datastore failure. -->
<min-backoff-seconds>1</min-backoff-seconds>
<max-backoff-seconds>60</max-backoff-seconds>
<!-- No age limit; a failed export should be retried as long as possible to avoid
having data missing from our exported commit log record. -->
</retry-parameters>
</queue>
<!-- Queue for polling export BigQuery jobs for completion. -->
<queue>
<name>export-bigquery-poll</name>
<!-- Limit queue to 5 concurrent tasks and 5 per second to avoid hitting BigQuery quotas. -->
<rate>5/s</rate>
<bucket-size>5</bucket-size>
<max-concurrent-requests>5</max-concurrent-requests>
<!-- Check every 20s and increase interval to every 5 minutes. -->
<retry-parameters>
<min-backoff-seconds>20</min-backoff-seconds>
<max-backoff-seconds>300</max-backoff-seconds>
<max-doublings>2</max-doublings>
</retry-parameters>
</queue>
<!-- Queue for launching new snapshots and for triggering the initial BigQuery load jobs. -->
<queue>
<name>export-snapshot</name>
<rate>1/s</rate>
<retry-parameters>
<!-- Should be less than the exportSnapshot cron interval; see cron.xml. -->
<task-age-limit>22h</task-age-limit>
<!-- Retry starting at a 5m interval and increasing up to a 30m interval. -->
<min-backoff-seconds>300</min-backoff-seconds>
<max-backoff-seconds>1800</max-backoff-seconds>
<task-retry-limit>10</task-retry-limit>
</retry-parameters>
</queue>
<!-- Queue for polling managed backup snapshots for completion. -->
<queue>
<name>export-snapshot-poll</name>
<rate>5/m</rate>
<retry-parameters>
<!-- Should be less than the exportSnapshot cron interval; see cron.xml. -->
<task-age-limit>22h</task-age-limit>
<!-- Retry starting at a 1m interval and increasing up to a 5m interval. -->
<min-backoff-seconds>60</min-backoff-seconds>
<max-backoff-seconds>300</max-backoff-seconds>
</retry-parameters>
</queue>
<!-- Queue for updating BigQuery views after a snapshot kind's load job completes. -->
<queue>
<name>export-snapshot-update-view</name>
<rate>1/s</rate>
<retry-parameters>
<!-- Should be less than the exportSnapshot cron interval; see cron.xml. -->
<task-age-limit>22h</task-age-limit>
<!-- Retry starting at a 10s interval and increasing up to a 1m interval. -->
<min-backoff-seconds>10</min-backoff-seconds>
<max-backoff-seconds>60</max-backoff-seconds>
<task-retry-limit>10</task-retry-limit>
</retry-parameters>
</queue>
<queue>
<name>rde-upload</name>
<rate>10/m</rate>
@@ -149,7 +69,7 @@
</retry-parameters>
</queue>
<!-- Queue for tasks to produce LORDN CSV reports, either by by the query or queue method. -->
<!-- Queue for tasks to produce LORDN CSV reports, either by the query or queue method. -->
<queue>
<name>nordn</name>
<rate>1/s</rate>
@@ -171,17 +91,6 @@
<mode>pull</mode>
</queue>
<!-- Queue used by the MapReduce library for running tasks.
Do not re-use this queue for tasks that our code creates (e.g. tasks to launch MapReduces
that aren't themselves part of a running MapReduce).-->
<queue>
<name>mapreduce</name>
<!-- Warning: DO NOT SET A <target> parameter for this queue. See b/24782801 for why. -->
<rate>500/s</rate>
<bucket-size>100</bucket-size>
</queue>
<!-- Queue for tasks that sync data to Google Spreadsheets. -->
<queue>
<name>sheet</name>
@@ -208,71 +117,4 @@
<max-concurrent-requests>5</max-concurrent-requests>
</queue>
<!-- Queue for replaying commit logs to SQL during the transition from Datastore -> SQL. -->
<queue>
<name>replay-commit-logs-to-sql</name>
<rate>1/s</rate>
</queue>
<!-- The load[0-9] queues are used for load-testing, and can be safely deleted
in any environment that doesn't require load-testing. -->
<queue>
<name>load0</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load1</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load2</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load3</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load4</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load5</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load6</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load7</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load8</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load9</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
</queue-entries>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>backend</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>default</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>pubapi</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>tools</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>backend</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>default</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>pubapi</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>tools</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>backend</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>default</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<manual-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>pubapi</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<manual-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>tools</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>backend</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>default</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>F4_1G</instance-class>
<automatic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>pubapi</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>tools</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>backend</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>default</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<manual-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>pubapi</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<manual-scaling>
@@ -1,9 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java17</runtime>
<app-engine-apis>true</app-engine-apis>
<runtime>java8</runtime>
<service>tools</service>
<threadsafe>true</threadsafe>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>
@@ -14,16 +14,16 @@
package google.registry.export;
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.services.drive.Drive;
import dagger.Component;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule;
import google.registry.config.CredentialModule.GSuiteDriveCredential;
import google.registry.config.CredentialModule.GoogleWorkspaceCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.storage.drive.DriveConnection;
import google.registry.util.GoogleCredentialsBundle;
import javax.inject.Singleton;
/** Dagger module for Google {@link Drive} service connection objects. */
@@ -32,13 +32,13 @@ public final class DriveModule {
@Provides
static Drive provideDrive(
@GSuiteDriveCredential GoogleCredential googleCredential,
@GoogleWorkspaceCredential GoogleCredentialsBundle googleCredential,
@Config("projectId") String projectId) {
return new Drive.Builder(
googleCredential.getTransport(),
googleCredential.getHttpTransport(),
googleCredential.getJsonFactory(),
googleCredential)
googleCredential.getHttpRequestInitializer())
.setApplicationName(projectId)
.build();
}
@@ -163,7 +163,7 @@ public final class SyncGroupMembersAction implements Runnable {
registrarsToSave.add(result.getKey().asBuilder().setContactsRequireSyncing(false).build());
}
}
tm().transactNew(() -> tm().updateAll(registrarsToSave.build()));
tm().transact(() -> tm().updateAll(registrarsToSave.build()));
return errors;
}
@@ -17,7 +17,7 @@ package google.registry.export.sheet;
import com.google.api.services.sheets.v4.Sheets;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule.JsonCredential;
import google.registry.config.CredentialModule.GoogleWorkspaceCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.util.GoogleCredentialsBundle;
@@ -27,7 +27,7 @@ public final class SheetsServiceModule {
@Provides
static Sheets provideSheets(
@JsonCredential GoogleCredentialsBundle credentialsBundle,
@GoogleWorkspaceCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Sheets.Builder(
credentialsBundle.getHttpTransport(),
@@ -23,7 +23,6 @@ import static java.nio.charset.StandardCharsets.UTF_8;
import com.google.common.base.Throwables;
import com.google.common.flogger.FluentLogger;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException.CommandUseErrorException;
import google.registry.flows.EppException.ParameterValueRangeErrorException;
import google.registry.flows.EppException.SyntaxErrorException;
@@ -34,7 +33,7 @@ import google.registry.model.eppcommon.EppXmlTransformer;
import google.registry.model.eppinput.EppInput.WrongProtocolVersionException;
import google.registry.model.eppoutput.EppOutput;
import google.registry.model.host.InetAddressAdapter.IpVersionMismatchException;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.translators.CurrencyUnitAdapter.UnknownCurrencyException;
import google.registry.xml.XmlException;
import java.util.List;
@@ -103,9 +102,8 @@ public final class FlowUtils {
}
}
public static <H extends HistoryEntry> Key<H> createHistoryKey(
EppResource parent, Class<H> clazz) {
return Key.create(Key.create(parent), clazz, allocateId());
public static HistoryEntryId createHistoryEntryId(EppResource parent) {
return new HistoryEntryId(parent.getRepoId(), allocateId());
}
/** Registrar is not logged in. */
@@ -118,7 +116,7 @@ public final class FlowUtils {
/** IP address version mismatch. */
public static class IpAddressVersionMismatchException extends ParameterValueRangeErrorException {
public IpAddressVersionMismatchException() {
super("IP adddress version mismatch");
super("IP address version mismatch");
}
}
@@ -163,7 +163,7 @@ public final class ResourceFlowUtils {
// The roid should match one of the contacts.
Optional<VKey<Contact>> foundContact =
domain.getReferencedContacts().stream()
.filter(key -> key.getSqlKey().equals(authRepoId))
.filter(key -> key.getKey().equals(authRepoId))
.findFirst();
if (!foundContact.isPresent()) {
throw new BadAuthInfoForResourceException();
@@ -20,17 +20,15 @@ import com.google.common.base.CharMatcher;
import com.google.common.base.Preconditions;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Sets;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.EppException.ParameterValuePolicyErrorException;
import google.registry.flows.EppException.ParameterValueSyntaxErrorException;
import google.registry.model.contact.Contact;
import google.registry.model.contact.ContactAddress;
import google.registry.model.contact.ContactHistory;
import google.registry.model.contact.ContactHistory.ContactHistoryId;
import google.registry.model.contact.PostalInfo;
import google.registry.model.poll.PendingActionNotificationResponse.ContactPendingActionNotificationResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.transfer.TransferData;
import google.registry.model.transfer.TransferResponse.ContactTransferResponse;
import java.util.Set;
@@ -69,10 +67,7 @@ public class ContactFlowUtils {
/** Create a poll message for the gaining client in a transfer. */
static PollMessage createGainingTransferPollMessage(
String targetId,
TransferData transferData,
DateTime now,
Key<ContactHistory> contactHistoryKey) {
String targetId, TransferData transferData, DateTime now, HistoryEntryId contactHistoryId) {
return new PollMessage.OneTime.Builder()
.setRegistrarId(transferData.getGainingRegistrarId())
.setEventTime(transferData.getPendingTransferExpirationTime())
@@ -85,23 +80,19 @@ public class ContactFlowUtils {
transferData.getTransferStatus().isApproved(),
transferData.getTransferRequestTrid(),
now)))
.setContactHistoryId(
new ContactHistoryId(
contactHistoryKey.getParent().getName(), contactHistoryKey.getId()))
.setContactHistoryId(contactHistoryId)
.build();
}
/** Create a poll message for the losing client in a transfer. */
static PollMessage createLosingTransferPollMessage(
String targetId, TransferData transferData, Key<ContactHistory> contactHistoryKey) {
String targetId, TransferData transferData, HistoryEntryId contactHistoryId) {
return new PollMessage.OneTime.Builder()
.setRegistrarId(transferData.getLosingRegistrarId())
.setEventTime(transferData.getPendingTransferExpirationTime())
.setMsg(transferData.getTransferStatus().getMessage())
.setResponseData(ImmutableList.of(createTransferResponse(targetId, transferData)))
.setContactHistoryId(
new ContactHistoryId(
contactHistoryKey.getParent().getName(), contactHistoryKey.getId()))
.setContactHistoryId(contactHistoryId)
.build();
}
@@ -90,10 +90,10 @@ public final class ContactInfoFlow implements Flow {
.setVoiceNumber(contact.getVoiceNumber())
.setFaxNumber(contact.getFaxNumber())
.setEmailAddress(contact.getEmailAddress())
.setCurrentSponsorClientId(contact.getCurrentSponsorRegistrarId())
.setCreationClientId(contact.getCreationRegistrarId())
.setCurrentSponsorRegistrarId(contact.getCurrentSponsorRegistrarId())
.setCreationRegistrarId(contact.getCreationRegistrarId())
.setCreationTime(contact.getCreationTime())
.setLastEppUpdateClientId(contact.getLastEppUpdateRegistrarId())
.setLastEppUpdateRegistrarId(contact.getLastEppUpdateRegistrarId())
.setLastEppUpdateTime(contact.getLastEppUpdateTime())
.setLastTransferTime(contact.getLastTransferTime())
.setAuthInfo(includeAuthInfo ? contact.getAuthInfo() : null)
@@ -26,7 +26,6 @@ import static google.registry.model.reporting.HistoryEntry.Type.CONTACT_TRANSFER
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -93,7 +92,7 @@ public final class ContactTransferApproveFlow implements TransactionalFlow {
// Create a poll message for the gaining client.
PollMessage gainingPollMessage =
createGainingTransferPollMessage(
targetId, newContact.getTransferData(), now, Key.create(contactHistory));
targetId, newContact.getTransferData(), now, contactHistory.getHistoryEntryId());
tm().insertAll(ImmutableSet.of(contactHistory, gainingPollMessage));
tm().update(newContact);
// Delete the billing event and poll messages that were written in case the transfer would have
@@ -26,7 +26,6 @@ import static google.registry.model.reporting.HistoryEntry.Type.CONTACT_TRANSFER
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -73,7 +72,7 @@ public final class ContactTransferCancelFlow implements TransactionalFlow {
@Inject ContactTransferCancelFlow() {}
@Override
public final EppResponse run() throws EppException {
public EppResponse run() throws EppException {
extensionManager.register(MetadataExtension.class);
validateRegistrarIsLoggedIn(registrarId);
extensionManager.validate();
@@ -89,7 +88,7 @@ public final class ContactTransferCancelFlow implements TransactionalFlow {
// Create a poll message for the losing client.
PollMessage losingPollMessage =
createLosingTransferPollMessage(
targetId, newContact.getTransferData(), Key.create(contactHistory));
targetId, newContact.getTransferData(), contactHistory.getHistoryEntryId());
tm().insertAll(ImmutableSet.of(contactHistory, losingPollMessage));
tm().update(newContact);
// Delete the billing event and poll messages that were written in case the transfer would have
@@ -26,7 +26,6 @@ import static google.registry.model.reporting.HistoryEntry.Type.CONTACT_TRANSFER
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -86,7 +85,7 @@ public final class ContactTransferRejectFlow implements TransactionalFlow {
historyBuilder.setType(CONTACT_TRANSFER_REJECT).setContact(newContact).build();
PollMessage gainingPollMessage =
createGainingTransferPollMessage(
targetId, newContact.getTransferData(), now, Key.create(contactHistory));
targetId, newContact.getTransferData(), now, contactHistory.getHistoryEntryId());
tm().insertAll(ImmutableSet.of(contactHistory, gainingPollMessage));
tm().update(newContact);
// Delete the billing event and poll messages that were written in case the transfer would have
@@ -14,7 +14,7 @@
package google.registry.flows.contact;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
import static google.registry.flows.ResourceFlowUtils.verifyAuthInfo;
@@ -28,7 +28,6 @@ import static google.registry.model.reporting.HistoryEntry.Type.CONTACT_TRANSFER
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.config.RegistryConfig.Config;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
@@ -46,6 +45,7 @@ import google.registry.model.eppcommon.StatusValue;
import google.registry.model.eppcommon.Trid;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.transfer.ContactTransferData;
import google.registry.model.transfer.TransferStatus;
@@ -74,21 +74,27 @@ import org.joda.time.Duration;
@ReportingSpec(ActivityReportField.CONTACT_TRANSFER_REQUEST)
public final class ContactTransferRequestFlow implements TransactionalFlow {
private static final ImmutableSet<StatusValue> DISALLOWED_STATUSES = ImmutableSet.of(
StatusValue.CLIENT_TRANSFER_PROHIBITED,
StatusValue.PENDING_DELETE,
StatusValue.SERVER_TRANSFER_PROHIBITED);
private static final ImmutableSet<StatusValue> DISALLOWED_STATUSES =
ImmutableSet.of(
StatusValue.CLIENT_TRANSFER_PROHIBITED,
StatusValue.PENDING_DELETE,
StatusValue.SERVER_TRANSFER_PROHIBITED);
@Inject ExtensionManager extensionManager;
@Inject Optional<AuthInfo> authInfo;
@Inject @RegistrarId String gainingClientId;
@Inject @TargetId String targetId;
@Inject @Config("contactAutomaticTransferLength") Duration automaticTransferLength;
@Inject
@Config("contactAutomaticTransferLength")
Duration automaticTransferLength;
@Inject ContactHistory.Builder historyBuilder;
@Inject Trid trid;
@Inject EppResponse.Builder responseBuilder;
@Inject ContactTransferRequestFlow() {}
@Inject
ContactTransferRequestFlow() {}
@Override
public EppResponse run() throws EppException {
@@ -120,29 +126,31 @@ public final class ContactTransferRequestFlow implements TransactionalFlow {
.setPendingTransferExpirationTime(transferExpirationTime)
.setTransferStatus(TransferStatus.SERVER_APPROVED)
.build();
Key<ContactHistory> contactHistoryKey = createHistoryKey(existingContact, ContactHistory.class);
historyBuilder.setId(contactHistoryKey.getId()).setType(CONTACT_TRANSFER_REQUEST);
HistoryEntryId contactHistoryId = createHistoryEntryId(existingContact);
historyBuilder
.setRevisionId(contactHistoryId.getRevisionId())
.setType(CONTACT_TRANSFER_REQUEST);
// If the transfer is server approved, this message will be sent to the losing registrar. */
PollMessage serverApproveLosingPollMessage =
createLosingTransferPollMessage(targetId, serverApproveTransferData, contactHistoryKey);
createLosingTransferPollMessage(targetId, serverApproveTransferData, contactHistoryId);
// If the transfer is server approved, this message will be sent to the gaining registrar. */
PollMessage serverApproveGainingPollMessage =
createGainingTransferPollMessage(
targetId, serverApproveTransferData, now, contactHistoryKey);
targetId, serverApproveTransferData, now, contactHistoryId);
ContactTransferData pendingTransferData =
serverApproveTransferData
.asBuilder()
.setTransferStatus(TransferStatus.PENDING)
.setServerApproveEntities(
serverApproveGainingPollMessage.getContactRepoId(),
contactHistoryKey.getId(),
contactHistoryId.getRevisionId(),
ImmutableSet.of(
serverApproveGainingPollMessage.createVKey(),
serverApproveLosingPollMessage.createVKey()))
.build();
// When a transfer is requested, a poll message is created to notify the losing registrar.
PollMessage requestPollMessage =
createLosingTransferPollMessage(targetId, pendingTransferData, contactHistoryKey)
createLosingTransferPollMessage(targetId, pendingTransferData, contactHistoryId)
.asBuilder()
.setEventTime(now) // Unlike the serverApprove messages, this applies immediately.
.build();
@@ -165,4 +173,3 @@ public final class ContactTransferRequestFlow implements TransactionalFlow {
.build();
}
}
@@ -86,7 +86,6 @@ import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand;
import google.registry.model.domain.DomainCommand.Create;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.Period;
import google.registry.model.domain.fee.FeeCreateCommandExtension;
@@ -110,6 +109,7 @@ import google.registry.model.poll.PollMessage.Autorenew;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.tld.Registry.TldState;
@@ -327,14 +327,14 @@ public final class DomainCreateFlow implements TransactionalFlow {
FeesAndCredits feesAndCredits =
pricingLogic.getCreatePrice(
registry, targetId, now, years, isAnchorTenant, allocationToken);
validateFeeChallenge(targetId, now, feeCreate, feesAndCredits);
validateFeeChallenge(feeCreate, feesAndCredits);
Optional<SecDnsCreateExtension> secDnsCreate =
validateSecDnsExtension(eppInput.getSingleExtension(SecDnsCreateExtension.class));
DateTime registrationExpirationTime = leapSafeAddYears(now, years);
String repoId = createDomainRepoId(allocateId(), registry.getTldStr());
long historyRevisionId = allocateId();
DomainHistoryId domainHistoryId = new DomainHistoryId(repoId, historyRevisionId);
historyBuilder.setId(historyRevisionId);
HistoryEntryId domainHistoryId = new HistoryEntryId(repoId, historyRevisionId);
historyBuilder.setRevisionId(historyRevisionId);
// Bill for the create.
BillingEvent.OneTime createBillingEvent =
createOneTimeBillingEvent(
@@ -406,7 +406,8 @@ public final class DomainCreateFlow implements TransactionalFlow {
if (allocationToken.isPresent()
&& TokenType.SINGLE_USE.equals(allocationToken.get().getTokenType())) {
entitiesToSave.add(
allocationTokenFlowUtils.redeemToken(allocationToken.get(), domainHistory.createVKey()));
allocationTokenFlowUtils.redeemToken(
allocationToken.get(), domainHistory.getHistoryEntryId()));
}
enqueueTasks(domain, hasSignedMarks, hasClaimsNotice);
@@ -562,7 +563,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
boolean isReserved,
int years,
FeesAndCredits feesAndCredits,
DomainHistoryId domainHistoryId,
HistoryEntryId domainHistoryId,
Optional<AllocationToken> allocationToken,
DateTime now) {
ImmutableSet.Builder<Flag> flagsBuilder = new ImmutableSet.Builder<>();
@@ -596,7 +597,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
}
private Recurring createAutorenewBillingEvent(
DomainHistoryId domainHistoryId,
HistoryEntryId domainHistoryId,
DateTime registrationExpirationTime,
RenewalPriceInfo renewalpriceInfo) {
return new BillingEvent.Recurring.Builder()
@@ -613,7 +614,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
}
private Autorenew createAutorenewPollMessage(
DomainHistoryId domainHistoryId, DateTime registrationExpirationTime) {
HistoryEntryId domainHistoryId, DateTime registrationExpirationTime) {
return new PollMessage.Autorenew.Builder()
.setTargetId(targetId)
.setRegistrarId(registrarId)
@@ -634,7 +635,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
.setEventTime(createBillingEvent.getEventTime())
.setBillingTime(createBillingEvent.getBillingTime())
.setFlags(createBillingEvent.getFlags())
.setDomainHistoryId(createBillingEvent.getDomainHistoryId())
.setDomainHistoryId(createBillingEvent.getHistoryEntryId())
.build();
}
@@ -674,11 +675,11 @@ public final class DomainCreateFlow implements TransactionalFlow {
Optional<AllocationToken> allocationToken,
FeesAndCredits feesAndCredits) {
if (isAnchorTenant) {
if (allocationToken.isPresent()) {
checkArgument(
allocationToken.get().getRenewalPriceBehavior() != RenewalPriceBehavior.SPECIFIED,
"Renewal price behavior cannot be SPECIFIED for anchor tenant");
}
allocationToken.ifPresent(
token ->
checkArgument(
token.getRenewalPriceBehavior() != RenewalPriceBehavior.SPECIFIED,
"Renewal price behavior cannot be SPECIFIED for anchor tenant"));
return RenewalPriceInfo.create(RenewalPriceBehavior.NONPREMIUM, null);
} else if (allocationToken.isPresent()
&& allocationToken.get().getRenewalPriceBehavior() == RenewalPriceBehavior.SPECIFIED) {
@@ -705,9 +706,12 @@ public final class DomainCreateFlow implements TransactionalFlow {
private static ImmutableList<FeeTransformResponseExtension> createResponseExtensions(
Optional<FeeCreateCommandExtension> feeCreate, FeesAndCredits feesAndCredits) {
return feeCreate.isPresent()
? ImmutableList.of(createFeeCreateResponse(feeCreate.get(), feesAndCredits))
: ImmutableList.of();
return feeCreate
.map(
feeCreateCommandExtension ->
ImmutableList.of(
createFeeCreateResponse(feeCreateCommandExtension, feesAndCredits)))
.orElseGet(ImmutableList::of);
}
/** Signed marks are only allowed during sunrise. */
@@ -16,7 +16,7 @@ package google.registry.flows.domain;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Strings.isNullOrEmpty;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.persistEntityChanges;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -43,7 +43,6 @@ import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Sets;
import com.googlecode.objectify.Key;
import google.registry.batch.AsyncTaskEnqueuer;
import google.registry.dns.DnsQueue;
import google.registry.flows.EppException;
@@ -66,7 +65,6 @@ import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.fee.BaseFee.FeeType;
import google.registry.model.domain.fee.Credit;
@@ -88,6 +86,7 @@ import google.registry.model.poll.PendingActionNotificationResponse.DomainPendin
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.tld.Registry.TldType;
@@ -181,8 +180,8 @@ public final class DomainDeleteFlow implements TransactionalFlow {
? Duration.ZERO
// By default, this should be 30 days of grace, and 5 days of pending delete.
: redemptionGracePeriodLength.plus(pendingDeleteLength);
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
historyBuilder.setId(domainHistoryKey.getId());
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder.setRevisionId(domainHistoryId.getRevisionId());
DateTime deletionTime = now.plus(durationUntilDelete);
if (durationUntilDelete.equals(Duration.ZERO)) {
builder.setDeletionTime(now).setStatusValues(null);
@@ -214,7 +213,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
// it is synchronous).
if (durationUntilDelete.isLongerThan(Duration.ZERO) || isSuperuser) {
PollMessage.OneTime deletePollMessage =
createDeletePollMessage(existingDomain, domainHistoryKey, deletionTime);
createDeletePollMessage(existingDomain, domainHistoryId, deletionTime);
entitiesToSave.add(deletePollMessage);
builder.setDeletePollMessage(deletePollMessage.createVKey());
}
@@ -224,7 +223,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
if (durationUntilDelete.isLongerThan(Duration.ZERO)
&& !registrarId.equals(existingDomain.getPersistedCurrentSponsorRegistrarId())) {
entitiesToSave.add(
createImmediateDeletePollMessage(existingDomain, domainHistoryKey, now, deletionTime));
createImmediateDeletePollMessage(existingDomain, domainHistoryId, now, deletionTime));
}
// Cancel any grace periods that were still active, and set the expiration time accordingly.
@@ -233,14 +232,9 @@ public final class DomainDeleteFlow implements TransactionalFlow {
// No cancellation is written if the grace period was not for a billable event.
if (gracePeriod.hasBillingEvent()) {
entitiesToSave.add(
BillingEvent.Cancellation.forGracePeriod(
gracePeriod,
now,
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()),
targetId));
BillingEvent.Cancellation.forGracePeriod(gracePeriod, now, domainHistoryId, targetId));
if (gracePeriod.getOneTimeBillingEvent() != null) {
// Take the amount of amount of registration time being refunded off the expiration time.
// Take the amount of registration time being refunded off the expiration time.
// This can be either add grace periods or renew grace periods.
BillingEvent.OneTime oneTime = tm().loadByKey(gracePeriod.getOneTimeBillingEvent());
newExpirationTime = newExpirationTime.minusYears(oneTime.getPeriodYears());
@@ -262,7 +256,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
BillingEvent.Recurring recurringBillingEvent =
updateAutorenewRecurrenceEndTime(
existingDomain, existingRecurring, now, domainHistory.getDomainHistoryId());
existingDomain, existingRecurring, now, domainHistory.getHistoryEntryId());
// If there's a pending transfer, the gaining client's autorenew billing
// event and poll message will already have been deleted in
// ResourceDeleteFlow since it's listed in serverApproveEntities.
@@ -343,7 +337,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
}
private PollMessage.OneTime createDeletePollMessage(
Domain existingDomain, Key<DomainHistory> domainHistoryKey, DateTime deletionTime) {
Domain existingDomain, HistoryEntryId domainHistoryId, DateTime deletionTime) {
Optional<MetadataExtension> metadataExtension =
eppInput.getSingleExtension(MetadataExtension.class);
boolean hasMetadataMessage =
@@ -362,21 +356,16 @@ public final class DomainDeleteFlow implements TransactionalFlow {
ImmutableList.of(
DomainPendingActionNotificationResponse.create(
existingDomain.getDomainName(), true, trid, deletionTime)))
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
}
private PollMessage.OneTime createImmediateDeletePollMessage(
Domain existingDomain,
Key<DomainHistory> domainHistoryKey,
DateTime now,
DateTime deletionTime) {
Domain existingDomain, HistoryEntryId domainHistoryId, DateTime now, DateTime deletionTime) {
return new PollMessage.OneTime.Builder()
.setRegistrarId(existingDomain.getPersistedCurrentSponsorRegistrarId())
.setEventTime(now)
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.setMsg(
String.format(
"Domain %s was deleted by registry administrator with final deletion effective: %s",
@@ -88,7 +88,6 @@ import google.registry.model.domain.DomainCommand.CreateOrUpdate;
import google.registry.model.domain.DomainCommand.InvalidReferencesException;
import google.registry.model.domain.DomainCommand.Update;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.ForeignKeyedDesignatedContact;
import google.registry.model.domain.Period;
import google.registry.model.domain.fee.BaseFee;
@@ -121,7 +120,7 @@ import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.Registrar.State;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.tld.Registry;
import google.registry.model.tld.Registry.TldState;
import google.registry.model.tld.Registry.TldType;
@@ -429,7 +428,7 @@ public class DomainFlowUtils {
contacts.stream()
.collect(
toImmutableSetMultimap(
DesignatedContact::getType, contact -> contact.getContactKey()));
DesignatedContact::getType, DesignatedContact::getContactKey));
// If any contact type has multiple contacts:
if (contactsByType.asMap().values().stream().anyMatch(v -> v.size() > 1)) {
@@ -590,7 +589,7 @@ public class DomainFlowUtils {
Domain domain,
Recurring existingRecurring,
DateTime newEndTime,
@Nullable DomainHistoryId historyId) {
@Nullable HistoryEntryId historyId) {
Optional<PollMessage.Autorenew> autorenewPollMessage =
tm().loadByKeyIfPresent(domain.getAutorenewPollMessage());
@@ -609,7 +608,7 @@ public class DomainFlowUtils {
historyId, "Cannot create a new autorenew poll message without a domain history id");
updatedAutorenewPollMessage =
newAutorenewPollMessage(domain)
.setId((Long) domain.getAutorenewPollMessage().getSqlKey())
.setId((Long) domain.getAutorenewPollMessage().getKey())
.setAutorenewEndTime(newEndTime)
.setDomainHistoryId(historyId)
.build();
@@ -773,8 +772,6 @@ public class DomainFlowUtils {
* domain names.
*/
public static void validateFeeChallenge(
String domainName,
DateTime priceTime,
final Optional<? extends FeeTransformCommandExtension> feeCommand,
FeesAndCredits feesAndCredits)
throws EppException {
@@ -1135,9 +1132,9 @@ public class DomainFlowUtils {
Duration maxSearchPeriod,
final ImmutableSet<TransactionReportField> cancelableFields) {
List<? extends HistoryEntry> recentHistoryEntries =
List<DomainHistory> recentHistoryEntries =
findRecentHistoryEntries(domain, now, maxSearchPeriod);
Optional<? extends HistoryEntry> entryToCancel =
Optional<DomainHistory> entryToCancel =
Streams.findLast(
recentHistoryEntries.stream()
.filter(
@@ -1174,11 +1171,11 @@ public class DomainFlowUtils {
return recordsBuilder.build();
}
private static List<? extends HistoryEntry> findRecentHistoryEntries(
private static List<DomainHistory> findRecentHistoryEntries(
Domain domain, DateTime now, Duration maxSearchPeriod) {
return jpaTm()
.query(
"FROM DomainHistory WHERE modificationTime >= :beginning AND domainRepoId = "
"FROM DomainHistory WHERE modificationTime >= :beginning AND repoId = "
+ ":repoId ORDER BY modificationTime ASC",
DomainHistory.class)
.setParameter("beginning", now.minus(maxSearchPeriod))
@@ -111,7 +111,7 @@ public final class DomainInfoFlow implements Flow {
DomainInfoData.newBuilder()
.setDomainName(domain.getDomainName())
.setRepoId(domain.getRepoId())
.setCurrentSponsorClientId(domain.getCurrentSponsorRegistrarId())
.setCurrentSponsorRegistrarId(domain.getCurrentSponsorRegistrarId())
.setRegistrant(
tm().transact(() -> tm().loadByKey(domain.getRegistrant())).getContactId());
// If authInfo is non-null, then the caller is authorized to see the full information since we
@@ -125,9 +125,9 @@ public final class DomainInfoFlow implements Flow {
.setNameservers(hostsRequest.requestDelegated() ? domain.loadNameserverHostNames() : null)
.setSubordinateHosts(
hostsRequest.requestSubordinate() ? domain.getSubordinateHosts() : null)
.setCreationClientId(domain.getCreationRegistrarId())
.setCreationRegistrarId(domain.getCreationRegistrarId())
.setCreationTime(domain.getCreationTime())
.setLastEppUpdateClientId(domain.getLastEppUpdateRegistrarId())
.setLastEppUpdateRegistrarId(domain.getLastEppUpdateRegistrarId())
.setLastEppUpdateTime(domain.getLastEppUpdateTime())
.setRegistrationExpirationTime(domain.getRegistrationExpirationTime())
.setLastTransferTime(domain.getLastTransferTime())
@@ -14,7 +14,7 @@
package google.registry.flows.domain;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.persistEntityChanges;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -38,7 +38,6 @@ import static google.registry.util.DateTimeUtils.leapSafeAddYears;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.EppException.ParameterValueRangeErrorException;
import google.registry.flows.ExtensionManager;
@@ -62,7 +61,6 @@ import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Renew;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.DomainRenewData;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.Period;
@@ -83,6 +81,7 @@ import google.registry.model.eppoutput.EppResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import java.util.Optional;
@@ -100,7 +99,7 @@ import org.joda.time.Duration;
*
* <p>ICANN prohibits any registration from being longer than ten years so if the request would
* result in a registration greater than ten years long it will fail. In practice this means it's
* impossible to request a ten year renewal, since that will always cause the new registration to be
* impossible to request a ten-year renewal, since that will always cause the new registration to be
* longer than 10 years unless it comes in at the exact millisecond that the domain would have
* expired.
*
@@ -200,44 +199,36 @@ public final class DomainRenewFlow implements TransactionalFlow {
now,
years,
existingRecurringBillingEvent);
validateFeeChallenge(targetId, now, feeRenew, feesAndCredits);
validateFeeChallenge(feeRenew, feesAndCredits);
flowCustomLogic.afterValidation(
AfterValidationParameters.newBuilder()
.setExistingDomain(existingDomain)
.setNow(now)
.setYears(years)
.build());
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
historyBuilder.setId(domainHistoryKey.getId());
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder.setRevisionId(domainHistoryId.getRevisionId());
String tld = existingDomain.getTld();
// Bill for this explicit renew itself.
BillingEvent.OneTime explicitRenewEvent =
createRenewBillingEvent(
tld, feesAndCredits.getTotalCost(), years, domainHistoryKey, allocationToken, now);
tld, feesAndCredits.getTotalCost(), years, domainHistoryId, allocationToken, now);
// Create a new autorenew billing event and poll message starting at the new expiration time.
BillingEvent.Recurring newAutorenewEvent =
newAutorenewBillingEvent(existingDomain)
.setEventTime(newExpirationTime)
.setRenewalPrice(existingRecurringBillingEvent.getRenewalPrice().orElse(null))
.setRenewalPriceBehavior(existingRecurringBillingEvent.getRenewalPriceBehavior())
.setDomainHistoryId(
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
PollMessage.Autorenew newAutorenewPollMessage =
newAutorenewPollMessage(existingDomain)
.setEventTime(newExpirationTime)
.setDomainHistoryId(
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
// End the old autorenew billing event and poll message now. This may delete the poll message.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
updateAutorenewRecurrenceEndTime(
existingDomain,
existingRecurring,
now,
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()));
updateAutorenewRecurrenceEndTime(existingDomain, existingRecurring, now, domainHistoryId);
Domain newDomain =
existingDomain
.asBuilder()
@@ -260,7 +251,8 @@ public final class DomainRenewFlow implements TransactionalFlow {
if (allocationToken.isPresent()
&& TokenType.SINGLE_USE.equals(allocationToken.get().getTokenType())) {
entitiesToSave.add(
allocationTokenFlowUtils.redeemToken(allocationToken.get(), domainHistory.createVKey()));
allocationTokenFlowUtils.redeemToken(
allocationToken.get(), domainHistory.getHistoryEntryId()));
}
EntityChanges entityChanges =
flowCustomLogic.beforeSave(
@@ -339,7 +331,7 @@ public final class DomainRenewFlow implements TransactionalFlow {
String tld,
Money renewCost,
int years,
Key<DomainHistory> domainHistoryKey,
HistoryEntryId domainHistoryId,
Optional<AllocationToken> allocationToken,
DateTime now) {
return new BillingEvent.OneTime.Builder()
@@ -355,27 +347,27 @@ public final class DomainRenewFlow implements TransactionalFlow {
.map(AllocationToken::createVKey)
.orElse(null))
.setBillingTime(now.plus(Registry.get(tld).getRenewGracePeriodLength()))
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
}
private ImmutableList<FeeTransformResponseExtension> createResponseExtensions(
FeesAndCredits feesAndCredits, Optional<FeeRenewCommandExtension> feeRenew) {
return feeRenew.isPresent()
? ImmutableList.of(
feeRenew
.get()
.createResponseBuilder()
.setCurrency(feesAndCredits.getCurrency())
.setFees(
ImmutableList.of(
Fee.create(
feesAndCredits.getRenewCost().getAmount(),
FeeType.RENEW,
feesAndCredits.hasPremiumFeesOfType(FeeType.RENEW))))
.build())
: ImmutableList.of();
return feeRenew
.map(
feeRenewCommandExtension ->
ImmutableList.of(
feeRenewCommandExtension
.createResponseBuilder()
.setCurrency(feesAndCredits.getCurrency())
.setFees(
ImmutableList.of(
Fee.create(
feesAndCredits.getRenewCost().getAmount(),
FeeType.RENEW,
feesAndCredits.hasPremiumFeesOfType(FeeType.RENEW))))
.build()))
.orElseGet(ImmutableList::of);
}
/** The current expiration date is incorrect. */
@@ -14,7 +14,7 @@
package google.registry.flows.domain;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
import static google.registry.flows.ResourceFlowUtils.verifyOptionalAuthInfo;
@@ -34,7 +34,6 @@ import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.net.InternetDomainName;
import com.googlecode.objectify.Key;
import google.registry.dns.DnsQueue;
import google.registry.flows.EppException;
import google.registry.flows.EppException.CommandUseErrorException;
@@ -52,7 +51,6 @@ import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Update;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.fee.BaseFee.FeeType;
import google.registry.model.domain.fee.Fee;
import google.registry.model.domain.fee.FeeTransformResponseExtension;
@@ -67,6 +65,7 @@ import google.registry.model.eppoutput.EppResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import java.util.Optional;
@@ -85,12 +84,12 @@ import org.joda.time.DateTime;
*
* <p>This flow is called a restore "request" because technically it is only supposed to signal that
* the registrar requests the restore, which the registry can choose to process or not based on a
* restore report that is submitted through an out of band process and details the request. However,
* in practice this flow does the restore immediately. This is allowable because all of the fields
* on a restore report are optional or have default values, and so by policy when the request comes
* in we consider it to have been accompanied by a default-initialized report which we auto-approve.
* restore report that is submitted through an out-of-band process and details the request. However,
* in practice this flow does the restore immediately. This is allowable because all the fields on a
* restore report are optional or have default values, and so by policy when the request comes in we
* consider it to have been accompanied by a default-initialized report which we auto-approve.
*
* <p>Restores cost a fixed restore fee plus a one year renewal fee for the domain. The domain is
* <p>Restores cost a fixed restore fee plus a one-year renewal fee for the domain. The domain is
* restored to a single year expiration starting at the restore time, regardless of what the
* original expiration time was.
*
@@ -147,10 +146,8 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
Optional<FeeUpdateCommandExtension> feeUpdate =
eppInput.getSingleExtension(FeeUpdateCommandExtension.class);
verifyRestoreAllowed(command, existingDomain, feeUpdate, feesAndCredits, now);
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
historyBuilder.setId(domainHistoryKey.getId());
DomainHistoryId domainHistoryId =
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId());
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder.setRevisionId(domainHistoryId.getRevisionId());
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
DateTime newExpirationTime =
@@ -175,9 +172,7 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
newAutorenewPollMessage(existingDomain)
.setEventTime(newExpirationTime)
.setAutorenewEndTime(END_OF_TIME)
.setDomainHistoryId(
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
Domain newDomain =
performRestore(
@@ -231,7 +226,7 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
if (!existingDomain.getGracePeriodStatuses().contains(GracePeriodStatus.REDEMPTION)) {
throw new DomainNotEligibleForRestoreException();
}
validateFeeChallenge(targetId, now, feeUpdate, feesAndCredits);
validateFeeChallenge(feeUpdate, feesAndCredits);
}
private static Domain performRestore(
@@ -259,17 +254,17 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
}
private OneTime createRenewBillingEvent(
DomainHistoryId domainHistoryId, Money renewCost, DateTime now) {
HistoryEntryId domainHistoryId, Money renewCost, DateTime now) {
return prepareBillingEvent(domainHistoryId, renewCost, now).setReason(Reason.RENEW).build();
}
private BillingEvent.OneTime createRestoreBillingEvent(
DomainHistoryId domainHistoryId, Money restoreCost, DateTime now) {
HistoryEntryId domainHistoryId, Money restoreCost, DateTime now) {
return prepareBillingEvent(domainHistoryId, restoreCost, now).setReason(Reason.RESTORE).build();
}
private OneTime.Builder prepareBillingEvent(
DomainHistoryId domainHistoryId, Money cost, DateTime now) {
HistoryEntryId domainHistoryId, Money cost, DateTime now) {
return new BillingEvent.OneTime.Builder()
.setTargetId(targetId)
.setRegistrarId(registrarId)
@@ -297,15 +292,16 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
FeeType.RENEW,
feesAndCredits.hasPremiumFeesOfType(FeeType.RENEW)));
}
return feeUpdate.isPresent()
? ImmutableList.of(
feeUpdate
.get()
.createResponseBuilder()
.setCurrency(feesAndCredits.getCurrency())
.setFees(fees.build())
.build())
: ImmutableList.of();
return feeUpdate
.map(
feeUpdateCommandExtension ->
ImmutableList.of(
feeUpdateCommandExtension
.createResponseBuilder()
.setCurrency(feesAndCredits.getCurrency())
.setFees(fees.build())
.build()))
.orElseGet(ImmutableList::of);
}
/** Restore command cannot have other changes specified. */
@@ -15,7 +15,7 @@
package google.registry.flows.domain;
import static com.google.common.collect.Iterables.getOnlyElement;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.computeExDateForApprovalTime;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -36,7 +36,6 @@ import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -52,7 +51,6 @@ import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.domain.rgp.GracePeriodStatus;
@@ -62,6 +60,7 @@ import google.registry.model.eppinput.EppInput;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.DomainTransferData;
@@ -146,8 +145,8 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
// Create a transfer billing event for 1 year, unless the superuser extension was used to set
// the transfer period to zero. There is not a transfer cost if the transfer period is zero.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
historyBuilder.setId(domainHistoryKey.getId());
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder.setRevisionId(domainHistoryId.getRevisionId());
Optional<BillingEvent.OneTime> billingEvent =
transferData.getTransferPeriod().getValue() == 0
? Optional.empty()
@@ -167,9 +166,7 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
.getRenewCost())
.setEventTime(now)
.setBillingTime(now.plus(Registry.get(tld).getTransferGracePeriodLength()))
.setDomainHistoryId(
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build());
ImmutableList.Builder<ImmutableObject> entitiesToSave = new ImmutableList.Builder<>();
// If we are within an autorenew grace period, cancel the autorenew billing event and don't
@@ -185,20 +182,12 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
if (billingEvent.isPresent()) {
entitiesToSave.add(
BillingEvent.Cancellation.forGracePeriod(
autorenewGrace,
now,
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()),
targetId));
autorenewGrace, now, domainHistoryId, targetId));
}
}
// Close the old autorenew event and poll message at the transfer time (aka now). This may end
// up deleting the poll message.
updateAutorenewRecurrenceEndTime(
existingDomain,
existingRecurring,
now,
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()));
updateAutorenewRecurrenceEndTime(existingDomain, existingRecurring, now, domainHistoryId);
DateTime newExpirationTime =
computeExDateForApprovalTime(existingDomain, now, transferData.getTransferPeriod());
// Create a new autorenew event starting at the expiration time.
@@ -212,9 +201,7 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
.setRenewalPriceBehavior(existingRecurring.getRenewalPriceBehavior())
.setRenewalPrice(existingRecurring.getRenewalPrice().orElse(null))
.setRecurrenceEndTime(END_OF_TIME)
.setDomainHistoryId(
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
// Create a new autorenew poll message.
PollMessage.Autorenew gainingClientAutorenewPollMessage =
@@ -224,9 +211,7 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
.setEventTime(newExpirationTime)
.setAutorenewEndTime(END_OF_TIME)
.setMsg("Domain was auto-renewed.")
.setDomainHistoryId(
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
// Construct the post-transfer domain.
Domain partiallyApprovedDomain =
@@ -264,7 +249,7 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
// Create a poll message for the gaining client.
PollMessage gainingClientPollMessage =
createGainingTransferPollMessage(
targetId, newDomain.getTransferData(), newExpirationTime, now, domainHistoryKey);
targetId, newDomain.getTransferData(), newExpirationTime, now, domainHistoryId);
billingEvent.ifPresent(entitiesToSave::add);
entitiesToSave.add(
autorenewEvent,
@@ -14,7 +14,7 @@
package google.registry.flows.domain;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
import static google.registry.flows.ResourceFlowUtils.verifyHasPendingTransfer;
@@ -32,7 +32,6 @@ import static google.registry.persistence.transaction.TransactionManagerFactory.
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -47,6 +46,7 @@ import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.AuthInfo;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.TransferStatus;
@@ -83,7 +83,9 @@ public final class DomainTransferCancelFlow implements TransactionalFlow {
@Inject @Superuser boolean isSuperuser;
@Inject DomainHistory.Builder historyBuilder;
@Inject EppResponse.Builder responseBuilder;
@Inject DomainTransferCancelFlow() {}
@Inject
DomainTransferCancelFlow() {}
@Override
public EppResponse run() throws EppException {
@@ -100,9 +102,9 @@ public final class DomainTransferCancelFlow implements TransactionalFlow {
}
Registry registry = Registry.get(existingDomain.getTld());
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder
.setId(domainHistoryKey.getId())
.setRevisionId(domainHistoryId.getRevisionId())
.setOtherRegistrarId(existingDomain.getTransferData().getLosingRegistrarId());
Domain newDomain =
@@ -112,12 +114,12 @@ public final class DomainTransferCancelFlow implements TransactionalFlow {
newDomain,
domainHistory,
createLosingTransferPollMessage(
targetId, newDomain.getTransferData(), null, domainHistoryKey));
targetId, newDomain.getTransferData(), null, domainHistoryId));
// Reopen the autorenew event and poll message that we closed for the implicit transfer. This
// may recreate the autorenew poll message if it was deleted when the transfer request was made.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
updateAutorenewRecurrenceEndTime(
existingDomain, existingRecurring, END_OF_TIME, domainHistory.getDomainHistoryId());
existingDomain, existingRecurring, END_OF_TIME, domainHistory.getHistoryEntryId());
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
tm().delete(existingDomain.getTransferData().getServerApproveEntities());
@@ -14,7 +14,7 @@
package google.registry.flows.domain;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
import static google.registry.flows.ResourceFlowUtils.verifyHasPendingTransfer;
@@ -34,7 +34,6 @@ import static google.registry.util.CollectionUtils.union;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -49,6 +48,7 @@ import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.AuthInfo;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.TransferStatus;
@@ -95,9 +95,9 @@ public final class DomainTransferRejectFlow implements TransactionalFlow {
DateTime now = tm().getTransactionTime();
Domain existingDomain = loadAndVerifyExistence(Domain.class, targetId, now);
Registry registry = Registry.get(existingDomain.getTld());
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder
.setId(domainHistoryKey.getId())
.setRevisionId(domainHistoryId.getRevisionId())
.setOtherRegistrarId(existingDomain.getTransferData().getGainingRegistrarId());
verifyOptionalAuthInfo(authInfo, existingDomain);
@@ -113,12 +113,12 @@ public final class DomainTransferRejectFlow implements TransactionalFlow {
newDomain,
domainHistory,
createGainingTransferPollMessage(
targetId, newDomain.getTransferData(), null, now, domainHistoryKey));
targetId, newDomain.getTransferData(), null, now, domainHistoryId));
// Reopen the autorenew event and poll message that we closed for the implicit transfer. This
// may end up recreating the poll message if it was deleted upon the transfer request.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
updateAutorenewRecurrenceEndTime(
existingDomain, existingRecurring, END_OF_TIME, domainHistory.getDomainHistoryId());
existingDomain, existingRecurring, END_OF_TIME, domainHistory.getHistoryEntryId());
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
tm().delete(existingDomain.getTransferData().getServerApproveEntities());
@@ -14,7 +14,7 @@
package google.registry.flows.domain;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.computeExDateForApprovalTime;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -38,7 +38,6 @@ import static google.registry.persistence.transaction.TransactionManagerFactory.
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.batch.AsyncTaskEnqueuer;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
@@ -57,13 +56,11 @@ import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Transfer;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.Period;
import google.registry.model.domain.fee.FeeTransferCommandExtension;
import google.registry.model.domain.fee.FeeTransformResponseExtension;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.domain.superuser.DomainTransferRequestSuperuserExtension;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.domain.token.AllocationTokenExtension;
import google.registry.model.eppcommon.AuthInfo;
import google.registry.model.eppcommon.StatusValue;
@@ -74,6 +71,7 @@ import google.registry.model.eppoutput.EppResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.DomainTransferData;
@@ -170,20 +168,19 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
extensionManager.validate();
DateTime now = tm().getTransactionTime();
Domain existingDomain = loadAndVerifyExistence(Domain.class, targetId, now);
Optional<AllocationToken> allocationToken =
allocationTokenFlowUtils.verifyAllocationTokenIfPresent(
existingDomain,
Registry.get(existingDomain.getTld()),
gainingClientId,
now,
eppInput.getSingleExtension(AllocationTokenExtension.class));
allocationTokenFlowUtils.verifyAllocationTokenIfPresent(
existingDomain,
Registry.get(existingDomain.getTld()),
gainingClientId,
now,
eppInput.getSingleExtension(AllocationTokenExtension.class));
Optional<DomainTransferRequestSuperuserExtension> superuserExtension =
eppInput.getSingleExtension(DomainTransferRequestSuperuserExtension.class);
Period period =
superuserExtension.isPresent()
? superuserExtension.get().getRenewalPeriod()
: ((Transfer) resourceCommand).getPeriod();
verifyTransferAllowed(existingDomain, period, now, superuserExtension, allocationToken);
verifyTransferAllowed(existingDomain, period, now, superuserExtension);
String tld = existingDomain.getTld();
Registry registry = Registry.get(tld);
@@ -198,16 +195,16 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
// If the period is zero, then there is no fee for the transfer.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
Optional<FeesAndCredits> feesAndCredits =
(period.getValue() == 0)
period.getValue() == 0
? Optional.empty()
: Optional.of(
pricingLogic.getTransferPrice(registry, targetId, now, existingRecurring));
if (feesAndCredits.isPresent()) {
validateFeeChallenge(targetId, now, feeTransfer, feesAndCredits.get());
validateFeeChallenge(feeTransfer, feesAndCredits.get());
}
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder
.setId(domainHistoryKey.getId())
.setRevisionId(domainHistoryId.getRevisionId())
.setOtherRegistrarId(existingDomain.getCurrentSponsorRegistrarId());
DateTime automaticTransferTime =
superuserExtension
@@ -232,7 +229,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
createTransferServerApproveEntities(
automaticTransferTime,
serverApproveNewExpirationTime,
domainHistoryKey,
domainHistoryId,
existingDomain,
existingRecurring,
trid,
@@ -243,7 +240,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
DomainTransferData pendingTransferData =
createPendingTransferData(
domainAtTransferTime.getRepoId(),
domainHistoryKey.getId(),
domainHistoryId.getRevisionId(),
new DomainTransferData.Builder()
.setTransferRequestTrid(trid)
.setTransferRequestTime(now)
@@ -256,7 +253,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
// Create a poll message to notify the losing registrar that a transfer was requested.
PollMessage requestPollMessage =
createLosingTransferPollMessage(
targetId, pendingTransferData, serverApproveNewExpirationTime, domainHistoryKey)
targetId, pendingTransferData, serverApproveNewExpirationTime, domainHistoryId)
.asBuilder()
.setEventTime(now)
.build();
@@ -265,10 +262,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
// cloneProjectedAtTime() will replace these old autorenew entities with the server approve ones
// that we've created in this flow and stored in pendingTransferData.
updateAutorenewRecurrenceEndTime(
existingDomain,
existingRecurring,
automaticTransferTime,
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()));
existingDomain, existingRecurring, automaticTransferTime, domainHistoryId);
Domain newDomain =
existingDomain
.asBuilder()
@@ -296,8 +290,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
Domain existingDomain,
Period period,
DateTime now,
Optional<DomainTransferRequestSuperuserExtension> superuserExtension,
Optional<AllocationToken> allocationToken)
Optional<DomainTransferRequestSuperuserExtension> superuserExtension)
throws EppException {
verifyNoDisallowedStatuses(existingDomain, DISALLOWED_STATUSES);
if (!isSuperuser) {
@@ -388,7 +381,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
private static ImmutableList<FeeTransformResponseExtension> createResponseExtensions(
Optional<FeesAndCredits> feesAndCredits, Optional<FeeTransferCommandExtension> feeTransfer) {
return (feeTransfer.isPresent() && feesAndCredits.isPresent())
return feeTransfer.isPresent() && feesAndCredits.isPresent()
? ImmutableList.of(
feeTransfer
.get()
@@ -20,20 +20,18 @@ import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.Period;
import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.eppcommon.Trid;
import google.registry.model.poll.PendingActionNotificationResponse.DomainPendingActionNotificationResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.DomainTransferData;
import google.registry.model.transfer.TransferData;
@@ -109,7 +107,7 @@ public final class DomainTransferUtils {
public static ImmutableSet<TransferServerApproveEntity> createTransferServerApproveEntities(
DateTime automaticTransferTime,
DateTime serverApproveNewExpirationTime,
Key<DomainHistory> domainHistoryKey,
HistoryEntryId domainHistoryId,
Domain existingDomain,
Recurring existingRecurring,
Trid trid,
@@ -135,38 +133,38 @@ public final class DomainTransferUtils {
builder.add(
createTransferBillingEvent(
automaticTransferTime,
domainHistoryKey,
domainHistoryId,
targetId,
gainingRegistrarId,
registry,
cost)));
createOptionalAutorenewCancellation(
automaticTransferTime, now, domainHistoryKey, targetId, existingDomain, transferCost)
automaticTransferTime, now, domainHistoryId, targetId, existingDomain, transferCost)
.ifPresent(builder::add);
return builder
.add(
createGainingClientAutorenewEvent(
existingRecurring,
serverApproveNewExpirationTime,
domainHistoryKey,
domainHistoryId,
targetId,
gainingRegistrarId))
.add(
createGainingClientAutorenewPollMessage(
serverApproveNewExpirationTime, domainHistoryKey, targetId, gainingRegistrarId))
serverApproveNewExpirationTime, domainHistoryId, targetId, gainingRegistrarId))
.add(
createGainingTransferPollMessage(
targetId,
serverApproveTransferData,
serverApproveNewExpirationTime,
now,
domainHistoryKey))
domainHistoryId))
.add(
createLosingTransferPollMessage(
targetId,
serverApproveTransferData,
serverApproveNewExpirationTime,
domainHistoryKey))
domainHistoryId))
.build();
}
@@ -176,7 +174,7 @@ public final class DomainTransferUtils {
TransferData transferData,
@Nullable DateTime extendedRegistrationExpirationTime,
DateTime now,
Key<DomainHistory> domainHistoryKey) {
HistoryEntryId domainHistoryId) {
return new PollMessage.OneTime.Builder()
.setRegistrarId(transferData.getGainingRegistrarId())
.setEventTime(transferData.getPendingTransferExpirationTime())
@@ -189,8 +187,7 @@ public final class DomainTransferUtils {
transferData.getTransferStatus().isApproved(),
transferData.getTransferRequestTrid(),
now)))
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
}
@@ -199,7 +196,7 @@ public final class DomainTransferUtils {
String targetId,
TransferData transferData,
@Nullable DateTime extendedRegistrationExpirationTime,
Key<DomainHistory> domainHistoryKey) {
HistoryEntryId domainHistoryId) {
return new PollMessage.OneTime.Builder()
.setRegistrarId(transferData.getLosingRegistrarId())
.setEventTime(transferData.getPendingTransferExpirationTime())
@@ -207,8 +204,7 @@ public final class DomainTransferUtils {
.setResponseData(
ImmutableList.of(
createTransferResponse(targetId, transferData, extendedRegistrationExpirationTime)))
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
}
@@ -230,7 +226,7 @@ public final class DomainTransferUtils {
private static PollMessage.Autorenew createGainingClientAutorenewPollMessage(
DateTime serverApproveNewExpirationTime,
Key<DomainHistory> domainHistoryKey,
HistoryEntryId domainHistoryId,
String targetId,
String gainingRegistrarId) {
return new PollMessage.Autorenew.Builder()
@@ -239,15 +235,14 @@ public final class DomainTransferUtils {
.setEventTime(serverApproveNewExpirationTime)
.setAutorenewEndTime(END_OF_TIME)
.setMsg("Domain was auto-renewed.")
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
}
private static BillingEvent.Recurring createGainingClientAutorenewEvent(
Recurring existingRecurring,
DateTime serverApproveNewExpirationTime,
Key<DomainHistory> domainHistoryKey,
HistoryEntryId domainHistoryId,
String targetId,
String gainingRegistrarId) {
return new BillingEvent.Recurring.Builder()
@@ -259,8 +254,7 @@ public final class DomainTransferUtils {
.setRecurrenceEndTime(END_OF_TIME)
.setRenewalPriceBehavior(existingRecurring.getRenewalPriceBehavior())
.setRenewalPrice(existingRecurring.getRenewalPrice().orElse(null))
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
}
@@ -283,7 +277,7 @@ public final class DomainTransferUtils {
private static Optional<BillingEvent.Cancellation> createOptionalAutorenewCancellation(
DateTime automaticTransferTime,
DateTime now,
Key<DomainHistory> domainHistoryKey,
HistoryEntryId domainHistoryId,
String targetId,
Domain existingDomain,
Optional<Money> transferCost) {
@@ -294,11 +288,7 @@ public final class DomainTransferUtils {
if (autorenewGracePeriod != null && transferCost.isPresent()) {
return Optional.of(
BillingEvent.Cancellation.forGracePeriod(
autorenewGracePeriod,
now,
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()),
targetId)
autorenewGracePeriod, now, domainHistoryId, targetId)
.asBuilder()
.setEventTime(automaticTransferTime)
.build());
@@ -308,7 +298,7 @@ public final class DomainTransferUtils {
private static BillingEvent.OneTime createTransferBillingEvent(
DateTime automaticTransferTime,
Key<DomainHistory> domainHistoryKey,
HistoryEntryId domainHistoryId,
String targetId,
String gainingRegistrarId,
Registry registry,
@@ -321,8 +311,7 @@ public final class DomainTransferUtils {
.setPeriodYears(1)
.setEventTime(automaticTransferTime)
.setBillingTime(automaticTransferTime.plus(registry.getTransferGracePeriodLength()))
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
}
@@ -87,6 +87,7 @@ import google.registry.model.poll.PendingActionNotificationResponse.DomainPendin
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import java.util.Objects;
import java.util.Optional;
import javax.inject.Inject;
import org.joda.time.DateTime;
@@ -181,7 +182,9 @@ public final class DomainUpdateFlow implements TransactionalFlow {
DomainHistory domainHistory =
historyBuilder.setType(DOMAIN_UPDATE).setDomain(newDomain).build();
validateNewState(newDomain);
dnsQueue.addDomainRefreshTask(targetId);
if (requiresDnsUpdate(existingDomain, newDomain)) {
dnsQueue.addDomainRefreshTask(targetId);
}
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
entitiesToSave.add(newDomain, domainHistory);
Optional<BillingEvent.OneTime> statusUpdateBillingEvent =
@@ -203,6 +206,13 @@ public final class DomainUpdateFlow implements TransactionalFlow {
return responseBuilder.build();
}
/** Determines if any of the changes to new domain should trigger DNS update. */
private boolean requiresDnsUpdate(Domain existingDomain, Domain newDomain) {
return existingDomain.shouldPublishToDns() != newDomain.shouldPublishToDns()
|| !Objects.equals(newDomain.getDsData(), existingDomain.getDsData())
|| !Objects.equals(newDomain.getNsHosts(), existingDomain.getNsHosts());
}
/** Fail if the object doesn't exist or was deleted. */
private void verifyUpdateAllowed(Update command, Domain existingDomain, DateTime now)
throws EppException {
@@ -267,12 +277,18 @@ public final class DomainUpdateFlow implements TransactionalFlow {
.setLastEppUpdateRegistrarId(registrarId)
.addStatusValues(add.getStatusValues())
.removeStatusValues(remove.getStatusValues())
.addNameservers(add.getNameservers().stream().collect(toImmutableSet()))
.removeNameservers(remove.getNameservers().stream().collect(toImmutableSet()))
.removeContacts(remove.getContacts())
.addContacts(add.getContacts())
.setRegistrant(firstNonNull(change.getRegistrant(), domain.getRegistrant()))
.setAuthInfo(firstNonNull(change.getAuthInfo(), domain.getAuthInfo()));
if (!add.getNameservers().isEmpty()) {
domainBuilder.addNameservers(add.getNameservers().stream().collect(toImmutableSet()));
}
if (!remove.getNameservers().isEmpty()) {
domainBuilder.removeNameservers(remove.getNameservers().stream().collect(toImmutableSet()));
}
Optional<DomainUpdateSuperuserExtension> superuserExt =
eppInput.getSingleExtension(DomainUpdateSuperuserExtension.class);
if (superuserExt.isPresent()) {
@@ -344,7 +360,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
.map(StatusValue::getXmlName)
.collect(toImmutableSortedSet(Ordering.natural()));
String msg = "";
String msg;
if (addedServerStatuses.size() > 0 && removedServerStatuses.size() > 0) {
msg =
String.format(
@@ -36,7 +36,7 @@ import google.registry.model.domain.token.AllocationToken.TokenBehavior;
import google.registry.model.domain.token.AllocationToken.TokenStatus;
import google.registry.model.domain.token.AllocationToken.TokenType;
import google.registry.model.domain.token.AllocationTokenExtension;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.tld.Registry;
import google.registry.persistence.VKey;
import java.util.List;
@@ -95,12 +95,11 @@ public class AllocationTokenFlowUtils {
}
/** Redeems a SINGLE_USE {@link AllocationToken}, returning the redeemed copy. */
public AllocationToken redeemToken(
AllocationToken token, VKey<? extends HistoryEntry> redemptionHistoryEntry) {
public AllocationToken redeemToken(AllocationToken token, HistoryEntryId redemptionHistoryId) {
checkArgument(
TokenType.SINGLE_USE.equals(token.getTokenType()),
"Only SINGLE_USE tokens can be marked as redeemed");
return token.asBuilder().setRedemptionHistoryEntry(redemptionHistoryEntry).build();
return token.asBuilder().setRedemptionHistoryId(redemptionHistoryId).build();
}
/**
@@ -110,7 +109,7 @@ public class AllocationTokenFlowUtils {
*
* @throws EppException if the token is invalid in any way
*/
private void validateToken(
private static void validateToken(
InternetDomainName domainName, AllocationToken token, String registrarId, DateTime now)
throws EppException {
@@ -138,7 +137,7 @@ public class AllocationTokenFlowUtils {
}
/** Loads a given token and validates that it is not redeemed */
private AllocationToken loadToken(String token) throws EppException {
private static AllocationToken loadToken(String token) throws EppException {
if (Strings.isNullOrEmpty(token)) {
// We load the token directly from the input XML. If it's null or empty we should throw
// an InvalidAllocationTokenException before the database load attempt fails.
@@ -152,7 +151,7 @@ public class AllocationTokenFlowUtils {
}
maybeTokenEntity =
tm().transact(() -> tm().loadByKeyIfPresent(VKey.createSql(AllocationToken.class, token)));
tm().transact(() -> tm().loadByKeyIfPresent(VKey.create(AllocationToken.class, token)));
if (!maybeTokenEntity.isPresent()) {
throw new InvalidAllocationTokenException();
@@ -80,14 +80,14 @@ public final class HostInfoFlow implements Flow {
tm().transact(
() -> tm().loadByKey(host.getSuperordinateDomain()).cloneProjectedAtTime(now));
hostInfoDataBuilder
.setCurrentSponsorClientId(superordinateDomain.getCurrentSponsorRegistrarId())
.setCurrentSponsorRegistrarId(superordinateDomain.getCurrentSponsorRegistrarId())
.setLastTransferTime(host.computeLastTransferTime(superordinateDomain));
if (superordinateDomain.getStatusValues().contains(StatusValue.PENDING_TRANSFER)) {
statusValues.add(StatusValue.PENDING_TRANSFER);
}
} else {
hostInfoDataBuilder
.setCurrentSponsorClientId(host.getPersistedCurrentSponsorRegistrarId())
.setCurrentSponsorRegistrarId(host.getPersistedCurrentSponsorRegistrarId())
.setLastTransferTime(host.getLastTransferTime());
}
return responseBuilder
@@ -97,9 +97,9 @@ public final class HostInfoFlow implements Flow {
.setRepoId(host.getRepoId())
.setStatusValues(statusValues.build())
.setInetAddresses(host.getInetAddresses())
.setCreationClientId(host.getCreationRegistrarId())
.setCreationRegistrarId(host.getCreationRegistrarId())
.setCreationTime(host.getCreationTime())
.setLastEppUpdateClientId(host.getLastEppUpdateRegistrarId())
.setLastEppUpdateRegistrarId(host.getLastEppUpdateRegistrarId())
.setLastEppUpdateTime(host.getLastEppUpdateTime())
.build())
.build();
@@ -268,7 +268,7 @@ public final class HostUpdateFlow implements TransactionalFlow {
}
// We must also enqueue updates for all domains that use this host as their nameserver so
// that their NS records can be updated to point at the new name.
asyncTaskEnqueuer.enqueueAsyncDnsRefresh(existingHost, tm().getTransactionTime());
// TODO(jianglai): implement a SQL based solution.
}
}
@@ -108,7 +108,7 @@ public final class PollAckFlow implements TransactionalFlow {
// acked, then we return a special status code indicating that. Note that the query will
// include the message being acked.
int messageCount = tm().doTransactionless(() -> getPollMessageCount(registrarId, now));
int messageCount = tm().transact(() -> getPollMessageCount(registrarId, now));
if (messageCount <= 0) {
return responseBuilder.setResultFromCode(SUCCESS_WITH_NO_MESSAGES).build();
}
@@ -1,41 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.keyring.kms;
import com.google.auto.value.AutoValue;
import com.google.common.annotations.VisibleForTesting;
/**
* A value type class containing a Cloud KMS encrypted and encoded ciphertext, and the name of the
* CryptoKeyVersion used to encrypt it.
*/
@AutoValue
abstract class EncryptResponse {
static EncryptResponse create(
com.google.api.services.cloudkms.v1.model.EncryptResponse cloudKmsEncryptResponse) {
return new AutoValue_EncryptResponse(
cloudKmsEncryptResponse.getCiphertext(), cloudKmsEncryptResponse.getName());
}
@VisibleForTesting
static EncryptResponse create(String ciphertext, String cryptoKeyVersionName) {
return new AutoValue_EncryptResponse(ciphertext, cryptoKeyVersionName);
}
abstract String ciphertext();
abstract String cryptoKeyVersionName();
}
@@ -1,48 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.keyring.kms;
import google.registry.keyring.api.KeyringException;
/** An abstraction to simplify Cloud KMS operations. */
public interface KmsConnection {
/**
* The maximum allowable secret size, as set by Cloud KMS.
*
* @see <a
* href="https://cloud.google.com/kms/docs/reference/rest/v1/projects.locations.keyRings.cryptoKeys/encrypt#request-body">projects.locations.keyRings.cryptoKeys.encrypt</a>
*/
int MAX_SECRET_SIZE_BYTES = 64 * 1024;
/**
* Encrypts a plaintext with CryptoKey {@code cryptoKeyName} on KeyRing {@code keyRingName}.
*
* <p>The latest CryptoKeyVersion is used to encrypt the value. The value must not be larger than
* {@code MAX_SECRET_SIZE_BYTES}.
*
* <p>If no applicable CryptoKey or CryptoKeyVersion exist, they will be created.
*
* @throws KeyringException on encryption failure.
*/
EncryptResponse encrypt(String cryptoKeyName, byte[] plaintext);
/**
* Decrypts a Cloud KMS encrypted and encoded value with CryptoKey {@code cryptoKeyName}.
*
* @throws KeyringException on decryption failure.
*/
byte[] decrypt(String cryptoKeyName, String encodedCiphertext);
}
@@ -1,177 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.keyring.kms;
import static com.google.common.base.Preconditions.checkArgument;
import com.google.api.client.googleapis.json.GoogleJsonResponseException;
import com.google.api.client.http.HttpStatusCodes;
import com.google.api.services.cloudkms.v1.CloudKMS;
import com.google.api.services.cloudkms.v1.model.CryptoKey;
import com.google.api.services.cloudkms.v1.model.CryptoKeyVersion;
import com.google.api.services.cloudkms.v1.model.DecryptRequest;
import com.google.api.services.cloudkms.v1.model.EncryptRequest;
import com.google.api.services.cloudkms.v1.model.KeyRing;
import com.google.api.services.cloudkms.v1.model.UpdateCryptoKeyPrimaryVersionRequest;
import google.registry.keyring.api.KeyringException;
import google.registry.util.Retrier;
import java.io.IOException;
/** The {@link KmsConnection} which talks to Cloud KMS. */
class KmsConnectionImpl implements KmsConnection {
private static final String KMS_LOCATION_FORMAT = "projects/%s/locations/global";
private static final String KMS_KEYRING_NAME_FORMAT = "projects/%s/locations/global/keyRings/%s";
private static final String KMS_CRYPTO_KEY_NAME_FORMAT =
"projects/%s/locations/global/keyRings/%s/cryptoKeys/%s";
private final CloudKMS kms;
private final String kmsKeyRingName;
private final String projectId;
private final Retrier retrier;
KmsConnectionImpl(String projectId, String kmsKeyRingName, Retrier retrier, CloudKMS kms) {
this.projectId = projectId;
this.kmsKeyRingName = kmsKeyRingName;
this.retrier = retrier;
this.kms = kms;
}
@Override
public EncryptResponse encrypt(String cryptoKeyName, byte[] value) {
checkArgument(
value.length <= MAX_SECRET_SIZE_BYTES,
"Value to encrypt was larger than %s bytes",
MAX_SECRET_SIZE_BYTES);
try {
return attemptEncrypt(cryptoKeyName, value);
} catch (IOException e) {
throw new KeyringException(
String.format("CloudKMS encrypt operation failed for secret %s", cryptoKeyName), e);
}
}
private EncryptResponse attemptEncrypt(String cryptoKeyName, byte[] value) throws IOException {
String fullKeyRingName = getKeyRingName(projectId, kmsKeyRingName);
try {
kms.projects().locations().keyRings().get(fullKeyRingName).execute();
} catch (GoogleJsonResponseException jsonException) {
if (jsonException.getStatusCode() == HttpStatusCodes.STATUS_CODE_NOT_FOUND) {
// Create the KeyRing in the "global" namespace. Encryption keys will be accessible from all
// GCP regions.
kms.projects()
.locations()
.keyRings()
.create(getLocationName(projectId), new KeyRing())
.setKeyRingId(kmsKeyRingName)
.execute();
} else {
throw jsonException;
}
}
String fullKeyName = getCryptoKeyName(projectId, kmsKeyRingName, cryptoKeyName);
boolean newCryptoKey = false;
try {
kms.projects().locations().keyRings().cryptoKeys().get(fullKeyName).execute();
} catch (GoogleJsonResponseException jsonException) {
if (jsonException.getStatusCode() == HttpStatusCodes.STATUS_CODE_NOT_FOUND) {
newCryptoKey = true;
kms.projects()
.locations()
.keyRings()
.cryptoKeys()
.create(fullKeyRingName, new CryptoKey().setPurpose("ENCRYPT_DECRYPT"))
.setCryptoKeyId(cryptoKeyName)
.execute();
} else {
throw jsonException;
}
}
// New CryptoKeys start with a CryptoKeyVersion, so we only create a new CryptoKeyVersion and
// rotate to it if we're dealing with an existing CryptoKey.
if (!newCryptoKey) {
CryptoKeyVersion cryptoKeyVersion =
kms.projects()
.locations()
.keyRings()
.cryptoKeys()
.cryptoKeyVersions()
.create(fullKeyName, new CryptoKeyVersion())
.execute();
kms.projects()
.locations()
.keyRings()
.cryptoKeys()
.updatePrimaryVersion(
fullKeyName,
new UpdateCryptoKeyPrimaryVersionRequest()
.setCryptoKeyVersionId(getCryptoKeyVersionId(cryptoKeyVersion)))
.execute();
}
return EncryptResponse.create(
kms.projects()
.locations()
.keyRings()
.cryptoKeys()
.encrypt(fullKeyName, new EncryptRequest().encodePlaintext(value))
.execute());
}
@Override
public byte[] decrypt(final String cryptoKeyName, final String encodedCiphertext) {
try {
return retrier.callWithRetry(
() -> attemptDecrypt(cryptoKeyName, encodedCiphertext), IOException.class);
} catch (RuntimeException e) {
throw new KeyringException(
String.format("CloudKMS decrypt operation failed for secret %s", cryptoKeyName), e);
}
}
private byte[] attemptDecrypt(String cryptoKeyName, String encodedCiphertext) throws IOException {
return kms.projects()
.locations()
.keyRings()
.cryptoKeys()
.decrypt(
getCryptoKeyName(projectId, kmsKeyRingName, cryptoKeyName),
new DecryptRequest().setCiphertext(encodedCiphertext))
.execute()
.decodePlaintext();
}
private static String getLocationName(String projectId) {
return String.format(KMS_LOCATION_FORMAT, projectId);
}
private static String getKeyRingName(String projectId, String kmsKeyRingName) {
return String.format(KMS_KEYRING_NAME_FORMAT, projectId, kmsKeyRingName);
}
private static String getCryptoKeyName(
String projectId, String kmsKeyRingName, String cryptoKeyName) {
return String.format(KMS_CRYPTO_KEY_NAME_FORMAT, projectId, kmsKeyRingName, cryptoKeyName);
}
private static String getCryptoKeyVersionId(CryptoKeyVersion cryptoKeyVersion) {
String cryptoKeyVersionName = cryptoKeyVersion.getName();
return cryptoKeyVersionName.substring(cryptoKeyVersionName.lastIndexOf('/') + 1);
}
}
@@ -1,84 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.keyring.kms;
import com.google.api.services.cloudkms.v1.CloudKMS;
import dagger.Binds;
import dagger.Module;
import dagger.Provides;
import dagger.multibindings.IntoMap;
import dagger.multibindings.StringKey;
import google.registry.config.CredentialModule.DefaultCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.keyring.api.Keyring;
import google.registry.util.GoogleCredentialsBundle;
import google.registry.util.Retrier;
/** Dagger module for Cloud KMS. */
@Module
public abstract class KmsModule {
public static final String NAME = "KMS";
@Provides
@Config("defaultKms")
static CloudKMS provideKms(
@DefaultCredential GoogleCredentialsBundle credentialsBundle,
@Config("cloudKmsProjectId") String projectId) {
return createKms(credentialsBundle, projectId);
}
@Provides
@Config("beamKms")
static CloudKMS provideBeamKms(
@DefaultCredential GoogleCredentialsBundle credentialsBundle,
@Config("beamCloudKmsProjectId") String projectId) {
return createKms(credentialsBundle, projectId);
}
private static CloudKMS createKms(GoogleCredentialsBundle credentialsBundle, String projectId) {
return new CloudKMS.Builder(
credentialsBundle.getHttpTransport(),
credentialsBundle.getJsonFactory(),
credentialsBundle.getHttpRequestInitializer())
.setApplicationName(projectId)
.build();
}
@Provides
@Config("defaultKmsConnection")
static KmsConnection provideKmsConnection(
@Config("cloudKmsProjectId") String projectId,
@Config("cloudKmsKeyRing") String keyringName,
Retrier retrier,
@Config("defaultKms") CloudKMS defaultKms) {
return new KmsConnectionImpl(projectId, keyringName, retrier, defaultKms);
}
@Provides
@Config("beamKmsConnection")
static KmsConnection provideBeamKmsConnection(
@Config("beamCloudKmsProjectId") String projectId,
@Config("beamCloudKmsKeyRing") String keyringName,
Retrier retrier,
@Config("beamKms") CloudKMS defaultKms) {
return new KmsConnectionImpl(projectId, keyringName, retrier, defaultKms);
}
@Binds
@IntoMap
@StringKey(NAME)
abstract Keyring provideKeyring(KmsKeyring keyring);
}
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.keyring.kms;
package google.registry.keyring.secretmanager;
import static com.google.common.base.CaseFormat.LOWER_HYPHEN;
import static com.google.common.base.CaseFormat.UPPER_UNDERSCORE;
@@ -29,8 +29,7 @@ import org.bouncycastle.openpgp.PGPPrivateKey;
import org.bouncycastle.openpgp.PGPPublicKey;
/** A {@link Keyring} implementation which stores sensitive data in the Secret Manager. */
// TODO(2021-08-01): rename this class to SecretManagerKeyring and update config files.
public class KmsKeyring implements Keyring {
public class SecretManagerKeyring implements Keyring {
/** Key labels for private key secrets. */
enum PrivateKeyLabel {
@@ -75,7 +74,7 @@ public class KmsKeyring implements Keyring {
private final KeyringSecretStore secretStore;
@Inject
KmsKeyring(KeyringSecretStore secretStore) {
SecretManagerKeyring(KeyringSecretStore secretStore) {
this.secretStore = secretStore;
}
@@ -0,0 +1,40 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.keyring.secretmanager;
import dagger.Binds;
import dagger.Module;
import dagger.multibindings.IntoMap;
import dagger.multibindings.StringKey;
import google.registry.keyring.api.Keyring;
/** Dagger module for {@link Keyring} backed by the Cloud SecretManager. */
@Module
public abstract class SecretManagerKeyringModule {
public static final String NAME = "CSM";
// TODO(b/257276342): Remove after configs in nomulus-internal are updated.
public static final String DEPRECATED_NAME = "KMS";
@Binds
@IntoMap
@StringKey(DEPRECATED_NAME)
abstract Keyring provideDeprecatedKeyring(SecretManagerKeyring keyring);
@Binds
@IntoMap
@StringKey(NAME)
abstract Keyring provideKeyring(SecretManagerKeyring keyring);
}
@@ -12,33 +12,33 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.keyring.kms;
package google.registry.keyring.secretmanager;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.keyring.kms.KmsKeyring.PrivateKeyLabel.BRDA_SIGNING_PRIVATE;
import static google.registry.keyring.kms.KmsKeyring.PrivateKeyLabel.RDE_SIGNING_PRIVATE;
import static google.registry.keyring.kms.KmsKeyring.PrivateKeyLabel.RDE_STAGING_PRIVATE;
import static google.registry.keyring.kms.KmsKeyring.PublicKeyLabel.BRDA_RECEIVER_PUBLIC;
import static google.registry.keyring.kms.KmsKeyring.PublicKeyLabel.BRDA_SIGNING_PUBLIC;
import static google.registry.keyring.kms.KmsKeyring.PublicKeyLabel.RDE_RECEIVER_PUBLIC;
import static google.registry.keyring.kms.KmsKeyring.PublicKeyLabel.RDE_SIGNING_PUBLIC;
import static google.registry.keyring.kms.KmsKeyring.PublicKeyLabel.RDE_STAGING_PUBLIC;
import static google.registry.keyring.kms.KmsKeyring.StringKeyLabel.ICANN_REPORTING_PASSWORD_STRING;
import static google.registry.keyring.kms.KmsKeyring.StringKeyLabel.JSON_CREDENTIAL_STRING;
import static google.registry.keyring.kms.KmsKeyring.StringKeyLabel.MARKSDB_DNL_LOGIN_STRING;
import static google.registry.keyring.kms.KmsKeyring.StringKeyLabel.MARKSDB_LORDN_PASSWORD_STRING;
import static google.registry.keyring.kms.KmsKeyring.StringKeyLabel.MARKSDB_SMDRL_LOGIN_STRING;
import static google.registry.keyring.kms.KmsKeyring.StringKeyLabel.RDE_SSH_CLIENT_PRIVATE_STRING;
import static google.registry.keyring.kms.KmsKeyring.StringKeyLabel.RDE_SSH_CLIENT_PUBLIC_STRING;
import static google.registry.keyring.kms.KmsKeyring.StringKeyLabel.SAFE_BROWSING_API_KEY;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.PrivateKeyLabel.BRDA_SIGNING_PRIVATE;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.PrivateKeyLabel.RDE_SIGNING_PRIVATE;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.PrivateKeyLabel.RDE_STAGING_PRIVATE;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.PublicKeyLabel.BRDA_RECEIVER_PUBLIC;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.PublicKeyLabel.BRDA_SIGNING_PUBLIC;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.PublicKeyLabel.RDE_RECEIVER_PUBLIC;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.PublicKeyLabel.RDE_SIGNING_PUBLIC;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.PublicKeyLabel.RDE_STAGING_PUBLIC;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.StringKeyLabel.ICANN_REPORTING_PASSWORD_STRING;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.StringKeyLabel.JSON_CREDENTIAL_STRING;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.StringKeyLabel.MARKSDB_DNL_LOGIN_STRING;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.StringKeyLabel.MARKSDB_LORDN_PASSWORD_STRING;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.StringKeyLabel.MARKSDB_SMDRL_LOGIN_STRING;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.StringKeyLabel.RDE_SSH_CLIENT_PRIVATE_STRING;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.StringKeyLabel.RDE_SSH_CLIENT_PUBLIC_STRING;
import static google.registry.keyring.secretmanager.SecretManagerKeyring.StringKeyLabel.SAFE_BROWSING_API_KEY;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.google.common.flogger.FluentLogger;
import google.registry.keyring.api.KeySerializer;
import google.registry.keyring.kms.KmsKeyring.PrivateKeyLabel;
import google.registry.keyring.kms.KmsKeyring.PublicKeyLabel;
import google.registry.keyring.kms.KmsKeyring.StringKeyLabel;
import google.registry.keyring.secretmanager.SecretManagerKeyring.PrivateKeyLabel;
import google.registry.keyring.secretmanager.SecretManagerKeyring.PublicKeyLabel;
import google.registry.keyring.secretmanager.SecretManagerKeyring.StringKeyLabel;
import google.registry.privileges.secretmanager.KeyringSecretStore;
import java.io.IOException;
import java.util.HashMap;
@@ -50,73 +50,77 @@ import org.bouncycastle.openpgp.PGPKeyPair;
import org.bouncycastle.openpgp.PGPPublicKey;
/**
* The {@link KmsUpdater} accumulates updates to a {@link KmsKeyring} and persists them to KMS and
* Datastore when closed.
* The {@link SecretManagerKeyringUpdater} accumulates updates to a {@link SecretManagerKeyring} and
* persists them to KMS and Datastore when closed.
*/
// TODO(2021-06-01): rename this class to SecretManagerKeyringUpdater
public final class KmsUpdater {
public final class SecretManagerKeyringUpdater {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final KeyringSecretStore secretStore;
private final HashMap<String, byte[]> secretValues;
@Inject
public KmsUpdater(KeyringSecretStore secretStore) {
public SecretManagerKeyringUpdater(KeyringSecretStore secretStore) {
this.secretStore = secretStore;
// Use LinkedHashMap to preserve insertion order on update() to simplify testing and debugging
this.secretValues = new LinkedHashMap<>();
}
public KmsUpdater setRdeSigningKey(PGPKeyPair keyPair) throws IOException, PGPException {
public SecretManagerKeyringUpdater setRdeSigningKey(PGPKeyPair keyPair)
throws IOException, PGPException {
return setKeyPair(keyPair, RDE_SIGNING_PRIVATE, RDE_SIGNING_PUBLIC);
}
public KmsUpdater setRdeStagingKey(PGPKeyPair keyPair) throws IOException, PGPException {
public SecretManagerKeyringUpdater setRdeStagingKey(PGPKeyPair keyPair)
throws IOException, PGPException {
return setKeyPair(keyPair, RDE_STAGING_PRIVATE, RDE_STAGING_PUBLIC);
}
public KmsUpdater setRdeReceiverPublicKey(PGPPublicKey publicKey) throws IOException {
public SecretManagerKeyringUpdater setRdeReceiverPublicKey(PGPPublicKey publicKey)
throws IOException {
return setPublicKey(publicKey, RDE_RECEIVER_PUBLIC);
}
public KmsUpdater setBrdaSigningKey(PGPKeyPair keyPair) throws IOException, PGPException {
public SecretManagerKeyringUpdater setBrdaSigningKey(PGPKeyPair keyPair)
throws IOException, PGPException {
return setKeyPair(keyPair, BRDA_SIGNING_PRIVATE, BRDA_SIGNING_PUBLIC);
}
public KmsUpdater setBrdaReceiverPublicKey(PGPPublicKey publicKey) throws IOException {
public SecretManagerKeyringUpdater setBrdaReceiverPublicKey(PGPPublicKey publicKey)
throws IOException {
return setPublicKey(publicKey, BRDA_RECEIVER_PUBLIC);
}
public KmsUpdater setRdeSshClientPublicKey(String asciiPublicKey) {
public SecretManagerKeyringUpdater setRdeSshClientPublicKey(String asciiPublicKey) {
return setString(asciiPublicKey, RDE_SSH_CLIENT_PUBLIC_STRING);
}
public KmsUpdater setRdeSshClientPrivateKey(String asciiPrivateKey) {
public SecretManagerKeyringUpdater setRdeSshClientPrivateKey(String asciiPrivateKey) {
return setString(asciiPrivateKey, RDE_SSH_CLIENT_PRIVATE_STRING);
}
public KmsUpdater setSafeBrowsingAPIKey(String apiKey) {
public SecretManagerKeyringUpdater setSafeBrowsingAPIKey(String apiKey) {
return setString(apiKey, SAFE_BROWSING_API_KEY);
}
public KmsUpdater setIcannReportingPassword(String password) {
public SecretManagerKeyringUpdater setIcannReportingPassword(String password) {
return setString(password, ICANN_REPORTING_PASSWORD_STRING);
}
public KmsUpdater setMarksdbDnlLoginAndPassword(String login) {
public SecretManagerKeyringUpdater setMarksdbDnlLoginAndPassword(String login) {
return setString(login, MARKSDB_DNL_LOGIN_STRING);
}
public KmsUpdater setMarksdbLordnPassword(String password) {
public SecretManagerKeyringUpdater setMarksdbLordnPassword(String password) {
return setString(password, MARKSDB_LORDN_PASSWORD_STRING);
}
public KmsUpdater setMarksdbSmdrlLoginAndPassword(String login) {
public SecretManagerKeyringUpdater setMarksdbSmdrlLoginAndPassword(String login) {
return setString(login, MARKSDB_SMDRL_LOGIN_STRING);
}
public KmsUpdater setJsonCredential(String credential) {
public SecretManagerKeyringUpdater setJsonCredential(String credential) {
return setString(credential, JSON_CREDENTIAL_STRING);
}
@@ -144,22 +148,22 @@ public final class KmsUpdater {
}
}
private KmsUpdater setString(String key, StringKeyLabel stringKeyLabel) {
private SecretManagerKeyringUpdater setString(String key, StringKeyLabel stringKeyLabel) {
checkArgumentNotNull(key);
setSecret(stringKeyLabel.getLabel(), KeySerializer.serializeString(key));
return this;
}
private KmsUpdater setPublicKey(PGPPublicKey publicKey, PublicKeyLabel publicKeyLabel)
throws IOException {
private SecretManagerKeyringUpdater setPublicKey(
PGPPublicKey publicKey, PublicKeyLabel publicKeyLabel) throws IOException {
checkArgumentNotNull(publicKey);
setSecret(publicKeyLabel.getLabel(), KeySerializer.serializePublicKey(publicKey));
return this;
}
private KmsUpdater setKeyPair(
private SecretManagerKeyringUpdater setKeyPair(
PGPKeyPair keyPair, PrivateKeyLabel privateKeyLabel, PublicKeyLabel publicKeyLabel)
throws IOException, PGPException {
checkArgumentNotNull(keyPair);
@@ -1,45 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model;
import com.google.common.collect.ImmutableSet;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.common.GaeUserIdConverter;
import google.registry.model.contact.Contact;
import google.registry.model.contact.ContactHistory;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.host.Host;
import google.registry.model.host.HostHistory;
import google.registry.model.reporting.HistoryEntry;
/** Sets of classes of the Objectify-registered entities in use throughout the model. */
@DeleteAfterMigration
public final class EntityClasses {
/** Set of entity classes. */
public static final ImmutableSet<Class<? extends ImmutableObject>> ALL_CLASSES =
ImmutableSet.of(
Contact.class,
ContactHistory.class,
Domain.class,
DomainHistory.class,
GaeUserIdConverter.class,
HistoryEntry.class,
Host.class,
HostHistory.class);
private EntityClasses() {}
}
@@ -31,11 +31,9 @@ import com.github.benmanes.caffeine.cache.LoadingCache;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.annotation.Id;
import com.googlecode.objectify.annotation.Ignore;
import com.googlecode.objectify.annotation.Index;
import google.registry.config.RegistryConfig;
import google.registry.model.CacheUtils.AppEngineEnvironmentCacheLoader;
import google.registry.model.annotations.OfyIdAllocation;
import google.registry.model.eppcommon.StatusValue;
import google.registry.model.transfer.TransferData;
import google.registry.persistence.VKey;
@@ -47,7 +45,6 @@ import java.util.Set;
import javax.persistence.Access;
import javax.persistence.AccessType;
import javax.persistence.AttributeOverride;
import javax.persistence.AttributeOverrides;
import javax.persistence.Column;
import javax.persistence.MappedSuperclass;
import javax.persistence.Transient;
@@ -55,8 +52,8 @@ import org.joda.time.DateTime;
/** An EPP entity object (i.e. a domain, contact, or host). */
@MappedSuperclass
@Access(AccessType.FIELD) // otherwise it'll use the default if the repoId (property)
public abstract class EppResource extends BackupGroupRoot implements Buildable {
@Access(AccessType.FIELD) // otherwise it'll use the default of the repoId (property)
public abstract class EppResource extends UpdateAutoTimestampEntity implements Buildable {
private static final long serialVersionUID = -252782773382339534L;
@@ -70,7 +67,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
*
* @see <a href="https://tools.ietf.org/html/rfc5730">RFC 5730</a>
*/
@Id @Transient String repoId;
@OfyIdAllocation @Transient String repoId;
/**
* The ID of the registrar that is currently sponsoring this resource.
@@ -78,9 +75,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
* <p>This can be null in the case of pre-Registry-3.0-migration history objects with null
* resource fields.
*/
@Index
@Column(name = "currentSponsorRegistrarId")
String currentSponsorClientId;
String currentSponsorRegistrarId;
/**
* The ID of the registrar that created this resource.
@@ -88,8 +83,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
* <p>This can be null in the case of pre-Registry-3.0-migration history objects with null
* resource fields.
*/
@Column(name = "creationRegistrarId")
String creationClientId;
String creationRegistrarId;
/**
* The ID of the registrar that last updated this resource.
@@ -98,8 +92,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
* edits; it only includes EPP-visible modifications such as {@literal <update>}. Can be null if
* the resource has never been modified.
*/
@Column(name = "lastEppUpdateRegistrarId")
String lastEppUpdateClientId;
String lastEppUpdateRegistrarId;
/**
* The time when this resource was created.
@@ -111,8 +104,8 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
* <p>This can be null in the case of pre-Registry-3.0-migration history objects with null
* resource fields.
*/
@AttributeOverrides(@AttributeOverride(name = "creationTime", column = @Column))
@Ignore
// Need to override the default non-null column attribute.
@AttributeOverride(name = "creationTime", column = @Column)
CreateAutoTimestamp creationTime = CreateAutoTimestamp.create(null);
/**
@@ -128,7 +121,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
* out of the index at that time, as long as we query for resources whose deletion time is before
* now.
*/
@Index DateTime deletionTime;
DateTime deletionTime;
/**
* The time that this resource was last updated.
@@ -140,7 +133,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
DateTime lastEppUpdateTime;
/** Status values associated with this resource. */
@Ignore Set<StatusValue> statuses;
Set<StatusValue> statuses;
public String getRepoId() {
return repoId;
@@ -164,7 +157,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
}
public String getCreationRegistrarId() {
return creationClientId;
return creationRegistrarId;
}
public DateTime getLastEppUpdateTime() {
@@ -172,17 +165,17 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
}
public String getLastEppUpdateRegistrarId() {
return lastEppUpdateClientId;
return lastEppUpdateRegistrarId;
}
/**
* Get the stored value of {@link #currentSponsorClientId}.
* Get the stored value of {@link #currentSponsorRegistrarId}.
*
* <p>For subordinate hosts, this value may not represent the actual current client id, which is
* the client id of the superordinate host. For all other resources this is the true client id.
*/
public final String getPersistedCurrentSponsorRegistrarId() {
return currentSponsorClientId;
return currentSponsorRegistrarId;
}
public final ImmutableSet<StatusValue> getStatusValues() {
@@ -272,13 +265,13 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
/** Set the current sponsoring registrar. */
public B setPersistedCurrentSponsorRegistrarId(String currentSponsorRegistrarId) {
getInstance().currentSponsorClientId = currentSponsorRegistrarId;
getInstance().currentSponsorRegistrarId = currentSponsorRegistrarId;
return thisCastToDerived();
}
/** Set the registrar that created this resource. */
public B setCreationRegistrarId(String creationRegistrarId) {
getInstance().creationClientId = creationRegistrarId;
getInstance().creationRegistrarId = creationRegistrarId;
return thisCastToDerived();
}
@@ -290,7 +283,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
/** Set the registrar who last performed a {@literal <update>} on this resource. */
public B setLastEppUpdateRegistrarId(String lastEppUpdateRegistrarId) {
getInstance().lastEppUpdateClientId = lastEppUpdateRegistrarId;
getInstance().lastEppUpdateRegistrarId = lastEppUpdateRegistrarId;
return thisCastToDerived();
}
@@ -320,14 +313,14 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
/** Add to this resource's status values. */
public B addStatusValues(ImmutableSet<StatusValue> statusValues) {
return setStatusValues(ImmutableSet.copyOf(
union(getInstance().getStatusValues(), statusValues)));
return setStatusValues(
ImmutableSet.copyOf(union(getInstance().getStatusValues(), statusValues)));
}
/** Remove from this resource's status values. */
public B removeStatusValues(ImmutableSet<StatusValue> statusValues) {
return setStatusValues(ImmutableSet.copyOf(
difference(getInstance().getStatusValues(), statusValues)));
return setStatusValues(
ImmutableSet.copyOf(difference(getInstance().getStatusValues(), statusValues)));
}
/** Set this resource's repoId. */
@@ -339,7 +332,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
/**
* Set the update timestamp.
*
* <p>This is provided at EppResource since BackupGroupRoot doesn't have a Builder.
* <p>This is provided at EppResource since UpdateAutoTimestampEntity doesn't have a Builder.
*/
public B setUpdateTimestamp(UpdateAutoTimestamp updateTimestamp) {
getInstance().setUpdateTimestamp(updateTimestamp);
@@ -365,13 +358,13 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
@Override
public EppResource load(VKey<? extends EppResource> key) {
return replicaTm().doTransactionless(() -> replicaTm().loadByKey(key));
return replicaTm().transact(() -> replicaTm().loadByKey(key));
}
@Override
public Map<VKey<? extends EppResource>, EppResource> loadAll(
Iterable<? extends VKey<? extends EppResource>> keys) {
return replicaTm().doTransactionless(() -> replicaTm().loadByKeys(keys));
return replicaTm().transact(() -> replicaTm().loadByKeys(keys));
}
};
@@ -16,7 +16,6 @@ package google.registry.model;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
@@ -27,7 +26,6 @@ import static google.registry.util.DateTimeUtils.latestOf;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import com.googlecode.objectify.Key;
import google.registry.config.RegistryConfig;
import google.registry.model.EppResource.BuilderWithTransferData;
import google.registry.model.EppResource.ForeignKeyedEppResource;
@@ -67,7 +65,7 @@ public final class EppResourceUtils {
+ "AND deletionTime > :now";
// We have to use the native SQL query here because DomainHost table doesn't have its entity
// class so we cannot reference its property like domainHost.hostRepoId in a JPQL query.
// class, so we cannot reference its property like domainHost.hostRepoId in a JPQL query.
private static final String HOST_LINKED_DOMAIN_QUERY =
"SELECT d.repo_id FROM \"Domain\" d "
+ "JOIN \"DomainHost\" dh ON dh.domain_repo_id = d.repo_id "
@@ -260,7 +258,7 @@ public final class EppResourceUtils {
/**
* Rewinds an {@link EppResource} object to a given point in time.
*
* <p>This method costs nothing if {@code resource} is already current. Otherwise it needs to
* <p>This method costs nothing if {@code resource} is already current. Otherwise, it needs to
* perform a single fetch operation.
*
* <p><b>Warning:</b> A resource can only be rolled backwards in time, not forwards; therefore
@@ -292,7 +290,7 @@ public final class EppResourceUtils {
/**
* Rewinds an {@link EppResource} object to a given point in time.
*
* <p>This method costs nothing if {@code resource} is already current. Otherwise it returns an
* <p>This method costs nothing if {@code resource} is already current. Otherwise, it returns an
* async operation that performs a single fetch operation.
*
* @return an asynchronous operation returning resource at {@code timestamp} or {@code null} if
@@ -346,50 +344,35 @@ public final class EppResourceUtils {
"key must be either VKey<Contact> or VKey<Host>, but it is %s",
key);
boolean isContactKey = key.getKind().equals(Contact.class);
if (tm().isOfy()) {
com.googlecode.objectify.cmd.Query<Domain> query =
auditedOfy()
.load()
.type(Domain.class)
.filter(isContactKey ? "allContacts.contact" : "nsHosts", key.getOfyKey())
.filter("deletionTime >", now);
if (limit != null) {
query.limit(limit);
}
return query.keys().list().stream().map(Domain::createVKey).collect(toImmutableSet());
} else {
return tm().transact(
() -> {
Query query;
if (isContactKey) {
query =
jpaTm()
.query(CONTACT_LINKED_DOMAIN_QUERY, String.class)
.setParameter("fkRepoId", key)
.setParameter("now", now);
} else {
query =
jpaTm()
.getEntityManager()
.createNativeQuery(HOST_LINKED_DOMAIN_QUERY)
.setParameter("fkRepoId", key.getSqlKey())
.setParameter("now", now.toDate());
}
if (limit != null) {
query.setMaxResults(limit);
}
@SuppressWarnings("unchecked")
ImmutableSet<VKey<Domain>> domainKeySet =
(ImmutableSet<VKey<Domain>>)
query
.getResultStream()
.map(
repoId ->
Domain.createVKey(Key.create(Domain.class, (String) repoId)))
.collect(toImmutableSet());
return domainKeySet;
});
}
return tm().transact(
() -> {
Query query;
if (isContactKey) {
query =
jpaTm()
.query(CONTACT_LINKED_DOMAIN_QUERY, String.class)
.setParameter("fkRepoId", key)
.setParameter("now", now);
} else {
query =
jpaTm()
.getEntityManager()
.createNativeQuery(HOST_LINKED_DOMAIN_QUERY)
.setParameter("fkRepoId", key.getKey())
.setParameter("now", now.toDate());
}
if (limit != null) {
query.setMaxResults(limit);
}
@SuppressWarnings("unchecked")
ImmutableSet<VKey<Domain>> domainKeySet =
(ImmutableSet<VKey<Domain>>)
query
.getResultStream()
.map(repoId -> Domain.createVKey((String) repoId))
.collect(toImmutableSet());
return domainKeySet;
});
}
/**
@@ -89,7 +89,7 @@ public final class ForeignKeyUtils {
Class<E> clazz, Collection<String> foreignKeys, final DateTime now) {
return load(clazz, foreignKeys, false).entrySet().stream()
.filter(e -> now.isBefore(e.getValue().deletionTime()))
.collect(toImmutableMap(Entry::getKey, e -> VKey.createSql(clazz, e.getValue().repoId())));
.collect(toImmutableMap(Entry::getKey, e -> VKey.create(clazz, e.getValue().repoId())));
}
/**
@@ -146,16 +146,14 @@ public final class ForeignKeyUtils {
// called, it is always passed with a list of VKeys with the same type.
Class<? extends EppResource> clazz = keys.iterator().next().getKind();
ImmutableList<String> foreignKeys =
Streams.stream(keys)
.map(key -> (String) key.getSqlKey())
.collect(toImmutableList());
Streams.stream(keys).map(key -> (String) key.getKey()).collect(toImmutableList());
ImmutableMap<String, MostRecentResource> existingKeys =
ForeignKeyUtils.load(clazz, foreignKeys, true);
// The above map only contains keys that exist in the database, so we re-add the
// missing ones with Optional.empty() values for caching.
return Maps.asMap(
ImmutableSet.copyOf(keys),
key -> Optional.ofNullable(existingKeys.get((String) key.getSqlKey())));
key -> Optional.ofNullable(existingKeys.get((String) key.getKey())));
}
};
@@ -211,15 +209,14 @@ public final class ForeignKeyUtils {
return load(clazz, foreignKeys, now);
}
return foreignKeyCache
.getAll(
foreignKeys.stream().map(fk -> VKey.createSql(clazz, fk)).collect(toImmutableList()))
.getAll(foreignKeys.stream().map(fk -> VKey.create(clazz, fk)).collect(toImmutableList()))
.entrySet()
.stream()
.filter(e -> e.getValue().isPresent() && now.isBefore(e.getValue().get().deletionTime()))
.collect(
toImmutableMap(
e -> (String) e.getKey().getSqlKey(),
e -> VKey.createSql(clazz, e.getValue().get().repoId())));
e -> (String) e.getKey().getKey(),
e -> VKey.create(clazz, e.getValue().get().repoId())));
}
@AutoValue
@@ -17,58 +17,114 @@ package google.registry.model;
import static com.google.common.base.Preconditions.checkState;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.flogger.FluentLogger;
import google.registry.beam.common.RegistryPipelineWorkerInitializer;
import google.registry.config.RegistryEnvironment;
import google.registry.model.annotations.DeleteAfterMigration;
import java.util.concurrent.atomic.AtomicLong;
import java.util.function.Supplier;
/**
* Allocates a globally unique {@link Long} number to use as an Ofy {@code @Id}.
* Allocates a {@link long} to use as a {@code @Id}, (part) of the primary SQL key for an entity.
*
* <p>In non-test, non-beam environments the Id is generated by Datastore, otherwise it's from an
* atomic long number that's incremented every time this method is called.
* <p>Normally, the ID is globally unique and allocated by Datastore. It is possible to override
* this behavior by providing an ID supplier, such as in unit tests, where a self-allocated ID based
* on a monotonically increasing atomic {@link long} is used. Such an ID supplier can also be used
* in other scenarios, such as in a Beam pipeline to get around the limitation of Beam's inability
* to use GAE SDK to access Datastore. The override should be used with great care lest it results
* in irreversible data corruption.
*
* @see #setIdSupplier(Supplier)
*/
@DeleteAfterMigration
public final class IdService {
/**
* A placeholder String passed into DatastoreService.allocateIds that ensures that all ids are
* initialized from the same id pool.
*/
private static final String APP_WIDE_ALLOCATION_KIND = "common";
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private IdService() {}
private static Supplier<Long> idSupplier =
RegistryEnvironment.UNITTEST.equals(RegistryEnvironment.get())
? SelfAllocatedIdSupplier.getInstance()
: DatastoreIdSupplier.getInstance();
/**
* Counts of used ids for use in unit tests or Beam.
* Provides a {@link Supplier} of ID that overrides the default.
*
* <p>Note that one should only use self-allocate Ids in Beam for entities whose Ids are not
* important and are not persisted back to the database, i. e. nowhere the uniqueness of the ID is
* required.
* <p>Currently, the only use case for an override is in the Beam pipeline, where access to
* Datastore is not possible through the App Engine API. As such, the setter explicitly checks if
* the runtime is Beam.
*
* <p>Because the provided supplier is not guaranteed to be globally unique and compatible with
* existing IDs in the database, one should proceed with great care. It is safe to use an
* arbitrary supplier when the resulting IDs are not significant and not persisted back to the
* database, i.e. the IDs are only required by the {@link Buildable} contract but are not used in
* any meaningful way. One example is the RDE pipeline where we project EPP resource entities from
* history entries to watermark time, which are then marshalled into XML elements in the RDE
* deposits, where the IDs are omitted.
*/
private static final AtomicLong nextSelfAllocatedId = new AtomicLong(1); // ids cannot be zero
private static final boolean isSelfAllocated() {
return RegistryEnvironment.UNITTEST.equals(RegistryEnvironment.get())
|| "true".equals(System.getProperty(RegistryPipelineWorkerInitializer.PROPERTY, "false"));
public static void setIdSupplier(Supplier<Long> idSupplier) {
checkState(
"true".equals(System.getProperty(RegistryPipelineWorkerInitializer.PROPERTY, "false")),
"Can only set ID supplier in a Beam pipeline");
logger.atWarning().log("Using ID supplier override!");
IdService.idSupplier = idSupplier;
}
/** Allocates an id. */
// TODO(b/201547855): Find a way to allocate a unique ID without datastore.
public static long allocateId() {
return isSelfAllocated()
? nextSelfAllocatedId.getAndIncrement()
: DatastoreServiceFactory.getDatastoreService()
.allocateIds(APP_WIDE_ALLOCATION_KIND, 1)
.iterator()
.next()
.getId();
return idSupplier.get();
}
/** Resets the global self-allocated id counter (i.e. sets the next id to 1). */
@VisibleForTesting
public static void resetSelfAllocatedId() {
checkState(
isSelfAllocated(), "Can only call resetSelfAllocatedId() in unit tests or Beam pipelines");
nextSelfAllocatedId.set(1); // ids cannot be zero
// TODO(b/201547855): Find a way to allocate a unique ID without datastore.
private static class DatastoreIdSupplier implements Supplier<Long> {
private static final DatastoreIdSupplier INSTANCE = new DatastoreIdSupplier();
/**
* A placeholder String passed into {@code DatastoreService.allocateIds} that ensures that all
* IDs are initialized from the same ID pool.
*/
private static final String APP_WIDE_ALLOCATION_KIND = "common";
public static DatastoreIdSupplier getInstance() {
return INSTANCE;
}
@Override
public Long get() {
return DatastoreServiceFactory.getDatastoreService()
.allocateIds(APP_WIDE_ALLOCATION_KIND, 1)
.iterator()
.next()
.getId();
}
}
/**
* An ID supplier that allocates an ID from a monotonically increasing atomic {@link long}.
*
* <p>The generated IDs are only unique within the same JVM. It is not suitable for production use
* unless in cases the IDs are not significant.
*/
public static class SelfAllocatedIdSupplier implements Supplier<Long> {
private static final SelfAllocatedIdSupplier INSTANCE = new SelfAllocatedIdSupplier();
/** Counts of used ids for self allocating IDs. */
private static final AtomicLong nextSelfAllocatedId = new AtomicLong(1); // ids cannot be zero
public static SelfAllocatedIdSupplier getInstance() {
return INSTANCE;
}
@Override
public Long get() {
return nextSelfAllocatedId.getAndIncrement();
}
public void reset() {
nextSelfAllocatedId.set(1);
}
}
}
@@ -14,17 +14,13 @@
package google.registry.model;
import static com.google.common.collect.Iterables.transform;
import static com.google.common.collect.Maps.transformValues;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static java.lang.annotation.ElementType.FIELD;
import static java.lang.annotation.RetentionPolicy.RUNTIME;
import static java.util.stream.Collectors.toCollection;
import static java.util.stream.Collectors.toList;
import com.google.common.base.Joiner;
import com.google.common.collect.Maps;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.Ignore;
import google.registry.persistence.VKey;
@@ -56,15 +52,6 @@ public abstract class ImmutableObject implements Cloneable {
@Target(FIELD)
public @interface DoNotHydrate {}
/**
* Indicates that the field should be ignored when comparing an object in the datastore to the
* corresponding object in Cloud SQL.
*/
@Documented
@Retention(RUNTIME)
@Target(FIELD)
public @interface DoNotCompare {}
/**
* Indicates that the field stores a null value to indicate an empty set. This is also used in
* object comparison.
@@ -105,7 +92,7 @@ public abstract class ImmutableObject implements Cloneable {
*/
protected Map<Field, Object> getSignificantFields() {
// Can't use streams or ImmutableMap because we can have null values.
Map<Field, Object> result = new LinkedHashMap();
Map<Field, Object> result = new LinkedHashMap<>();
for (Map.Entry<Field, Object> entry : ModelUtils.getFieldValues(this).entrySet()) {
if (!entry.getKey().isAnnotationPresent(Insignificant.class)) {
result.put(entry.getKey(), entry.getValue());
@@ -190,15 +177,15 @@ public abstract class ImmutableObject implements Cloneable {
/** Helper function to recursively hydrate an ImmutableObject. */
private static Object hydrate(Object value) {
if (value instanceof Key) {
if (tm().isOfy()) {
return hydrate(auditedOfy().load().key((Key<?>) value).now());
}
return value;
} else if (value instanceof Map) {
}
if (value instanceof Map) {
return transformValues((Map<?, ?>) value, ImmutableObject::hydrate);
} else if (value instanceof Collection) {
return transform((Collection<?>) value, ImmutableObject::hydrate);
} else if (value instanceof ImmutableObject) {
}
if (value instanceof Collection) {
return ((Collection<?>) value).stream().map(ImmutableObject::hydrate);
}
if (value instanceof ImmutableObject) {
return ((ImmutableObject) value).toHydratedString();
}
return value;
@@ -220,7 +207,7 @@ public abstract class ImmutableObject implements Cloneable {
}
return result;
} else if (o instanceof Map) {
return Maps.transformValues((Map<?, ?>) o, ImmutableObject::toMapRecursive);
return transformValues((Map<?, ?>) o, ImmutableObject::toMapRecursive);
} else if (o instanceof Set) {
return ((Set<?>) o)
.stream()
@@ -183,11 +183,11 @@ public final class ResourceTransferUtils {
* sets the last EPP update client id to the given client id.
*/
public static <R extends EppResource & ResourceWithTransferData> R denyPendingTransfer(
R resource, TransferStatus transferStatus, DateTime now, String lastEppUpdateClientId) {
R resource, TransferStatus transferStatus, DateTime now, String lastEppUpdateRegistrarId) {
checkArgument(transferStatus.isDenied(), "Not a denial transfer status");
return resolvePendingTransfer(resource, transferStatus, now)
.setLastEppUpdateTime(now)
.setLastEppUpdateRegistrarId(lastEppUpdateClientId)
.setLastEppUpdateRegistrarId(lastEppUpdateRegistrarId)
.build();
}
}
@@ -1,69 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model;
import static com.google.common.base.Predicates.or;
import static com.google.common.base.Predicates.subtypeOf;
import static java.util.stream.Collectors.joining;
import com.google.common.collect.Ordering;
import google.registry.model.annotations.DeleteAfterMigration;
import java.util.ArrayDeque;
import java.util.Queue;
import java.util.SortedSet;
import java.util.TreeSet;
/** Utility methods for getting the version of the model schema from the model code. */
@DeleteAfterMigration
public final class SchemaVersion {
/**
* Returns a set of classes corresponding to all types persisted within the model classes, sorted
* by the string representation.
*/
private static SortedSet<Class<?>> getAllPersistedTypes() {
SortedSet<Class<?>> persistedTypes = new TreeSet<>(Ordering.usingToString());
// Do a breadth-first search for persisted types, starting with @Entity types and expanding each
// ImmutableObject by querying it for all its persisted field types.
persistedTypes.addAll(EntityClasses.ALL_CLASSES);
Queue<Class<?>> queue = new ArrayDeque<>(persistedTypes);
while (!queue.isEmpty()) {
Class<?> clazz = queue.remove();
if (ImmutableObject.class.isAssignableFrom(clazz)) {
for (Class<?> persistedFieldType : ModelUtils.getPersistedFieldTypes(clazz)) {
if (persistedTypes.add(persistedFieldType)) {
// If we haven't seen this type before, add it to the queue to query its field types.
queue.add(persistedFieldType);
}
}
}
}
return persistedTypes;
}
/**
* Return a string representing the schema which includes the definition of all persisted entity
* types (and their field types, recursively). Each definition contains the field names and their
* types (for classes), or else a list of all possible values (for enums).
*/
public static String getSchema() {
return getAllPersistedTypes().stream()
.filter(or(subtypeOf(Enum.class), subtypeOf(ImmutableObject.class)))
.map(ModelUtils::getSchema)
.collect(joining("\n"));
}
private SchemaVersion() {}
}
@@ -29,12 +29,7 @@ import org.joda.time.DateTime;
@Embeddable
public class UpdateAutoTimestamp extends ImmutableObject implements UnsafeSerializable {
// When set to true, database converters/translators should do the auto update. When set to
// false, auto update should be suspended (this exists to allow us to preserve the original value
// during a replay).
private static final ThreadLocal<Boolean> autoUpdateEnabled = ThreadLocal.withInitial(() -> true);
@Column(nullable = false)
@Column(name = "updateTimestamp")
DateTime lastUpdateTime;
// Unfortunately, we cannot use the @UpdateTimestamp annotation on "lastUpdateTime" in this class
@@ -43,9 +38,7 @@ public class UpdateAutoTimestamp extends ImmutableObject implements UnsafeSerial
@PrePersist
@PreUpdate
void setTimestamp() {
if (autoUpdateEnabled() || lastUpdateTime == null) {
lastUpdateTime = jpaTm().getTransactionTime();
}
lastUpdateTime = jpaTm().getTransactionTime();
}
/** Returns the timestamp, or {@code START_OF_TIME} if it's null. */
@@ -58,30 +51,4 @@ public class UpdateAutoTimestamp extends ImmutableObject implements UnsafeSerial
instance.lastUpdateTime = timestamp;
return instance;
}
// TODO(b/175610935): Remove the auto-update disabling code below after migration.
/** Class to allow us to safely disable auto-update in a try-with-resources block. */
public static class DisableAutoUpdateResource implements AutoCloseable {
DisableAutoUpdateResource() {
autoUpdateEnabled.set(false);
}
@Override
public void close() {
autoUpdateEnabled.set(true);
}
}
/**
* Resturns a resource that disables auto-updates on all {@link UpdateAutoTimestamp}s in the
* current thread, suitable for use with in a try-with-resources block.
*/
public static DisableAutoUpdateResource disableAutoUpdate() {
return new DisableAutoUpdateResource();
}
public static boolean autoUpdateEnabled() {
return autoUpdateEnabled.get();
}
}
@@ -18,24 +18,19 @@ import com.googlecode.objectify.annotation.Ignore;
import google.registry.util.PreconditionsUtils;
import javax.persistence.Access;
import javax.persistence.AccessType;
import javax.persistence.AttributeOverride;
import javax.persistence.Column;
import javax.persistence.MappedSuperclass;
import javax.xml.bind.annotation.XmlTransient;
/**
* Base class for entities that are the root of a Registry 2.0 entity group that gets enrolled in
* commit logs for backup purposes.
*
* <p>The commit log system needs to preserve the ordering of closely timed mutations to entities in
* a single entity group. We require an {@link UpdateAutoTimestamp} field on the root of a group so
* that we can enforce strictly increasing timestamps.
* Base class for entities that contains an {@link UpdateAutoTimestamp} which is updated every time
* the entity is persisted.
*/
@MappedSuperclass
public abstract class BackupGroupRoot extends ImmutableObject implements UnsafeSerializable {
public abstract class UpdateAutoTimestampEntity extends ImmutableObject
implements UnsafeSerializable {
/**
* An automatically managed timestamp of when this object was last written to Datastore.
* An automatically managed timestamp of when this object was last written to the database.
*
* <p>Note that this is distinct from the EPP-specified {@link EppResource#lastEppUpdateTime}, in
* that this is updated on every save, rather than only in response to an {@code <update>} command
@@ -44,7 +39,6 @@ public abstract class BackupGroupRoot extends ImmutableObject implements UnsafeS
// Prevents subclasses from unexpectedly accessing as property (e.g., Host), which would
// require an unnecessary non-private setter method.
@Access(AccessType.FIELD)
@AttributeOverride(name = "lastUpdateTime", column = @Column(name = "updateTimestamp"))
@Ignore
UpdateAutoTimestamp updateTimestamp = UpdateAutoTimestamp.create(null);
@@ -59,7 +53,7 @@ public abstract class BackupGroupRoot extends ImmutableObject implements UnsafeS
* <p>This method is for the few cases when {@code updateTimestamp} is copied between different
* types of entities. Use {@link #clone} for same-type copying.
*/
protected void copyUpdateTimestamp(BackupGroupRoot other) {
protected void copyUpdateTimestamp(UpdateAutoTimestampEntity other) {
this.updateTimestamp = PreconditionsUtils.checkArgumentNotNull(other, "other").updateTimestamp;
}
@@ -32,10 +32,10 @@ import google.registry.model.annotations.OfyIdAllocation;
import google.registry.model.annotations.ReportedOn;
import google.registry.model.common.TimeOfYear;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
import google.registry.persistence.VKey;
import google.registry.persistence.WithVKey;
@@ -204,8 +204,8 @@ public abstract class BillingEvent extends ImmutableObject
return targetId;
}
public DomainHistoryId getDomainHistoryId() {
return new DomainHistoryId(domainRepoId, domainHistoryRevisionId);
public HistoryEntryId getHistoryEntryId() {
return new HistoryEntryId(domainRepoId, domainHistoryRevisionId);
}
public ImmutableSet<Flag> getFlags() {
@@ -259,14 +259,14 @@ public abstract class BillingEvent extends ImmutableObject
return thisCastToDerived();
}
public B setDomainHistoryId(DomainHistoryId domainHistoryId) {
getInstance().domainHistoryRevisionId = domainHistoryId.getId();
getInstance().domainRepoId = domainHistoryId.getDomainRepoId();
public B setDomainHistoryId(HistoryEntryId domainHistoryId) {
getInstance().domainHistoryRevisionId = domainHistoryId.getRevisionId();
getInstance().domainRepoId = domainHistoryId.getRepoId();
return thisCastToDerived();
}
public B setDomainHistory(DomainHistory domainHistory) {
return setDomainHistoryId(domainHistory.getDomainHistoryId());
return setDomainHistoryId(domainHistory.getHistoryEntryId());
}
@Override
@@ -375,7 +375,7 @@ public abstract class BillingEvent extends ImmutableObject
}
public static VKey<OneTime> createVKey(long id) {
return VKey.createSql(OneTime.class, id);
return VKey.create(OneTime.class, id);
}
@Override
@@ -538,7 +538,7 @@ public abstract class BillingEvent extends ImmutableObject
}
public static VKey<Recurring> createVKey(Long id) {
return VKey.createSql(Recurring.class, id);
return VKey.create(Recurring.class, id);
}
@Override
@@ -653,7 +653,7 @@ public abstract class BillingEvent extends ImmutableObject
public static Cancellation forGracePeriod(
GracePeriod gracePeriod,
DateTime eventTime,
DomainHistoryId domainHistoryId,
HistoryEntryId domainHistoryId,
String targetId) {
checkArgument(
gracePeriod.hasBillingEvent(),
@@ -682,7 +682,7 @@ public abstract class BillingEvent extends ImmutableObject
}
public static VKey<Cancellation> createVKey(long id) {
return VKey.createSql(Cancellation.class, id);
return VKey.create(Cancellation.class, id);
}
@Override
@@ -1,112 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.bulkquery;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.GracePeriod.GracePeriodHistory;
import google.registry.model.domain.secdns.DomainDsData;
import google.registry.model.domain.secdns.DomainDsDataHistory;
import google.registry.model.host.Host;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.persistence.VKey;
import google.registry.persistence.transaction.JpaTransactionManager;
/**
* Utilities for managing an alternative JPA entity model optimized for bulk loading multi-level
* entities such as {@link Domain} and {@link DomainHistory}.
*
* <p>In a bulk query for a multi-level JPA entity type, the JPA framework only generates a bulk
* query (SELECT * FROM table) for the base table. Then, for each row in the base table, additional
* queries are issued to load associated rows in child tables. This can be very slow when an entity
* type has multiple child tables.
*
* <p>We have defined an alternative entity model for {@link Domain} and {@link DomainHistory},
* where the base table as well as the child tables are mapped to single-level entity types. The
* idea is to load each of these types using a bulk query, and assemble them into the target type in
* memory in a pipeline. The main use case is Datastore-Cloud SQL validation during the Registry
* database migration, where we will need the full database snapshots frequently.
*/
public class BulkQueryEntities {
/**
* The JPA entity classes in persistence.xml to replace when creating the {@link
* JpaTransactionManager} for bulk query.
*/
public static final ImmutableMap<String, String> JPA_ENTITIES_REPLACEMENTS =
ImmutableMap.of(
Domain.class.getCanonicalName(),
DomainLite.class.getCanonicalName(),
DomainHistory.class.getCanonicalName(),
DomainHistoryLite.class.getCanonicalName());
/* The JPA entity classes that are not included in persistence.xml and need to be added to
* the {@link JpaTransactionManager} for bulk query.*/
public static final ImmutableList<String> JPA_ENTITIES_NEW =
ImmutableList.of(
DomainHost.class.getCanonicalName(), DomainHistoryHost.class.getCanonicalName());
public static Domain assembleDomain(
DomainLite domainLite,
ImmutableSet<GracePeriod> gracePeriods,
ImmutableSet<DomainDsData> domainDsData,
ImmutableSet<VKey<Host>> nsHosts) {
Domain.Builder builder = new Domain.Builder();
builder.copyFrom(domainLite);
builder.setGracePeriods(gracePeriods);
builder.setDsData(domainDsData);
builder.setNameservers(nsHosts);
// Restore the original update timestamp (this gets cleared when we set nameservers or DS data).
builder.setUpdateTimestamp(domainLite.getUpdateTimestamp());
return builder.build();
}
public static DomainHistory assembleDomainHistory(
DomainHistoryLite domainHistoryLite,
ImmutableSet<DomainDsDataHistory> dsDataHistories,
ImmutableSet<VKey<Host>> domainHistoryHosts,
ImmutableSet<GracePeriodHistory> gracePeriodHistories,
ImmutableSet<DomainTransactionRecord> transactionRecords) {
DomainHistory.Builder builder = new DomainHistory.Builder();
builder.copyFrom(domainHistoryLite);
DomainBase rawDomainBase = domainHistoryLite.domainBase;
if (rawDomainBase != null) {
DomainBase newDomainBase =
domainHistoryLite
.domainBase
.asBuilder()
.setNameservers(domainHistoryHosts)
.setGracePeriods(
gracePeriodHistories.stream()
.map(GracePeriod::createFromHistory)
.collect(toImmutableSet()))
.setDsData(
dsDataHistories.stream().map(DomainDsData::create).collect(toImmutableSet()))
// Restore the original update timestamp (this gets cleared when we set nameservers or
// DS data).
.setUpdateTimestamp(domainHistoryLite.domainBase.getUpdateTimestamp())
.build();
builder.setDomain(newDomainBase);
}
return builder.buildAndAssemble(
dsDataHistories, domainHistoryHosts, gracePeriodHistories, transactionRecords);
}
}
@@ -1,69 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.bulkquery;
import com.google.common.base.Objects;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.host.Host;
import google.registry.persistence.VKey;
import java.io.Serializable;
import javax.persistence.Access;
import javax.persistence.AccessType;
import javax.persistence.Entity;
import javax.persistence.Id;
import javax.persistence.IdClass;
/**
* A name server host referenced by a {@link google.registry.model.domain.DomainHistory} record.
* Please refer to {@link BulkQueryEntities} for usage.
*/
@Entity
@Access(AccessType.FIELD)
@IdClass(DomainHistoryHost.class)
public class DomainHistoryHost implements Serializable {
@Id private Long domainHistoryHistoryRevisionId;
@Id private String domainHistoryDomainRepoId;
@Id private String hostRepoId;
private DomainHistoryHost() {}
public DomainHistoryId getDomainHistoryId() {
return new DomainHistoryId(domainHistoryDomainRepoId, domainHistoryHistoryRevisionId);
}
public VKey<Host> getHostVKey() {
return VKey.create(Host.class, hostRepoId);
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof DomainHistoryHost)) {
return false;
}
DomainHistoryHost that = (DomainHistoryHost) o;
return Objects.equal(domainHistoryHistoryRevisionId, that.domainHistoryHistoryRevisionId)
&& Objects.equal(domainHistoryDomainRepoId, that.domainHistoryDomainRepoId)
&& Objects.equal(hostRepoId, that.hostRepoId);
}
@Override
public int hashCode() {
return Objects.hashCode(domainHistoryHistoryRevisionId, domainHistoryDomainRepoId, hostRepoId);
}
}
@@ -1,125 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.bulkquery;
import com.googlecode.objectify.Key;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.Period;
import google.registry.model.reporting.HistoryEntry;
import google.registry.persistence.VKey;
import javax.annotation.Nullable;
import javax.persistence.Access;
import javax.persistence.AccessType;
import javax.persistence.AttributeOverride;
import javax.persistence.AttributeOverrides;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.Id;
import javax.persistence.IdClass;
import javax.persistence.PostLoad;
/**
* A 'light' version of {@link DomainHistory} with only base table ("DomainHistory") attributes,
* which allows fast bulk loading. They are used in in-memory assembly of {@code DomainHistory}
* instances along with bulk-loaded child entities ({@code GracePeriodHistory} etc). The in-memory
* assembly achieves much higher performance than loading {@code DomainHistory} directly.
*
* <p>Please refer to {@link BulkQueryEntities} for more information.
*
* <p>This class is adapted from {@link DomainHistory} by removing the {@code dsDataHistories},
* {@code gracePeriodHistories}, and {@code nsHosts} fields and associated methods.
*/
@Entity(name = "DomainHistory")
@Access(AccessType.FIELD)
@IdClass(DomainHistoryId.class)
public class DomainHistoryLite extends HistoryEntry {
// Store DomainBase instead of Domain so we don't pick up its @Id
// Nullable for the sake of pre-Registry-3.0 history objects
@Nullable DomainBase domainBase;
@Id
@Access(AccessType.PROPERTY)
public String getDomainRepoId() {
// We need to handle null case here because Hibernate sometimes accesses this method before
// parent gets initialized
return parent == null ? null : parent.getName();
}
/** This method is private because it is only used by Hibernate. */
@SuppressWarnings("unused")
private void setDomainRepoId(String domainRepoId) {
parent = Key.create(Domain.class, domainRepoId);
}
@Override
@Nullable
@Access(AccessType.PROPERTY)
@AttributeOverrides({
@AttributeOverride(name = "unit", column = @Column(name = "historyPeriodUnit")),
@AttributeOverride(name = "value", column = @Column(name = "historyPeriodValue"))
})
public Period getPeriod() {
return super.getPeriod();
}
/**
* For transfers, the id of the other registrar.
*
* <p>For requests and cancels, the other registrar is the losing party (because the registrar
* sending the EPP transfer command is the gaining party). For approves and rejects, the other
* registrar is the gaining party.
*/
@Nullable
@Access(AccessType.PROPERTY)
@Column(name = "historyOtherRegistrarId")
@Override
public String getOtherRegistrarId() {
return super.getOtherRegistrarId();
}
@Id
@Column(name = "historyRevisionId")
@Access(AccessType.PROPERTY)
@Override
public long getId() {
return super.getId();
}
/** The key to the {@link Domain} this is based off of. */
public VKey<Domain> getParentVKey() {
return VKey.create(Domain.class, getDomainRepoId());
}
public DomainHistoryId getDomainHistoryId() {
return new DomainHistoryId(getDomainRepoId(), getId());
}
@PostLoad
void postLoad() {
if (domainBase == null) {
return;
}
// See inline comments in DomainHistory.postLoad for reasons for the following lines.
if (domainBase.getDomainName() == null) {
domainBase = null;
} else if (domainBase.getRepoId() == null) {
domainBase.setRepoId(parent.getName());
}
}
}
@@ -1,64 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.bulkquery;
import com.google.common.base.Objects;
import google.registry.model.host.Host;
import google.registry.persistence.VKey;
import java.io.Serializable;
import javax.persistence.Access;
import javax.persistence.AccessType;
import javax.persistence.Entity;
import javax.persistence.Id;
import javax.persistence.IdClass;
/** A name server host of a domain. Please refer to {@link BulkQueryEntities} for usage. */
@Entity
@Access(AccessType.FIELD)
@IdClass(DomainHost.class)
public class DomainHost implements Serializable {
@Id private String domainRepoId;
@Id private String hostRepoId;
DomainHost() {}
public String getDomainRepoId() {
return domainRepoId;
}
public VKey<Host> getHostVKey() {
return VKey.create(Host.class, hostRepoId);
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof DomainHost)) {
return false;
}
DomainHost that = (DomainHost) o;
return Objects.equal(domainRepoId, that.domainRepoId)
&& Objects.equal(hostRepoId, that.hostRepoId);
}
@Override
public int hashCode() {
return Objects.hashCode(domainRepoId, hostRepoId);
}
}
@@ -1,48 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.bulkquery;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainBase;
import google.registry.persistence.VKey;
import google.registry.persistence.WithVKey;
import javax.persistence.Access;
import javax.persistence.AccessType;
import javax.persistence.Entity;
/**
* A 'light' version of {@link Domain} with only base table ("Domain") attributes, which allows fast
* bulk loading. They are used in in-memory assembly of {@code Domain} instances along with
* bulk-loaded child entities ({@code GracePeriod} etc). The in-memory assembly achieves much higher
* performance than loading {@code Domain} directly.
*
* <p>Please refer to {@link BulkQueryEntities} for more information.
*/
@Entity(name = "Domain")
@WithVKey(String.class)
@Access(AccessType.FIELD)
public class DomainLite extends DomainBase {
@Override
@javax.persistence.Id
@Access(AccessType.PROPERTY)
public String getRepoId() {
return super.getRepoId();
}
public static VKey<DomainLite> createVKey(String repoId) {
return VKey.createSql(DomainLite.class, repoId);
}
}
@@ -1,68 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.common;
import static com.google.common.base.Functions.identity;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.model.EntityClasses.ALL_CLASSES;
import com.google.common.annotations.VisibleForTesting;
import com.googlecode.objectify.annotation.EntitySubclass;
import java.util.Map;
import java.util.stream.Collectors;
/** A helper to manage class name and class path mapping. */
public class ClassPathManager {
/**
* Class registry allowing us to restore the original class object from the unqualified class
* name, which is all the datastore key gives us. Note that entities annotated
* with @EntitySubclass are removed because they share the same kind of the key with their parent
* class.
*/
public static final Map<String, Class<?>> CLASS_REGISTRY =
ALL_CLASSES.stream()
.filter(clazz -> !clazz.isAnnotationPresent(EntitySubclass.class))
.collect(Collectors.toMap(com.googlecode.objectify.Key::getKind, identity()));
/**
* Class name registry allowing us to obtain the class name the unqualified class, which is all
* the datastore key gives us. Note that entities annotated with @EntitySubclass are removed
* because they share the same kind of the key with their parent class.
*/
public static final Map<Class<?>, String> CLASS_NAME_REGISTRY =
ALL_CLASSES.stream()
.filter(clazz -> !clazz.isAnnotationPresent(EntitySubclass.class))
.collect(Collectors.toMap(identity(), com.googlecode.objectify.Key::getKind));
@VisibleForTesting
public static void addTestEntityClass(Class<?> clazz) {
CLASS_REGISTRY.put(clazz.getSimpleName(), clazz);
CLASS_NAME_REGISTRY.put(clazz, clazz.getSimpleName());
}
public static <T> Class<T> getClass(String className) {
checkArgument(
CLASS_REGISTRY.containsKey(className), "Class %s not found in class registry", className);
return (Class<T>) CLASS_REGISTRY.get(className);
}
public static <T> String getClassName(Class<T> clazz) {
checkArgument(
CLASS_NAME_REGISTRY.containsKey(clazz),
"Class %s not found in class name registry",
clazz.getSimpleName());
return CLASS_NAME_REGISTRY.get(clazz);
}
}

Some files were not shown because too many files have changed in this diff Show More