1
0
mirror of https://github.com/google/nomulus synced 2026-07-14 12:02:28 +00:00

Compare commits

...

38 Commits

Author SHA1 Message Date
gbrodman f62473542f Use the requested server host when creating the registry lock verification URL (#624)
* Use the server host when creating the registry lock verification URL

The app doesn't know about any external configuration that may point to
this app, so there's no way of finding out that, for instance,
registry.google points to the app. Thus, we have to use what the user
gives us so that, in our case, the registry-lock verification
emails can point to https://registry.google/registry-lock-verify instead
of https://domain-registry.appspot.com/registry-lock-verify. The former
is used by clients / users to authenticate, and unfortunately
authenticating on registry.google does not give authentication to
domain-registry.apspot.com.

Tested using the RDAP code that uses getServerName() -- in that case, if
you access registry.google/rdap/<>, it uses registry.google in the URLs
but if you use domain-registry.appspot.com/rdap/<>, it uses
domain-registry.appspot.com in the URLs.

Relatedly, frontend_config_prod-appengine.asciiproto in Piper
is what configures registry.google to point to
domain-registry.appspot.com
2020-06-12 10:11:53 -04:00
Lai Jiang 484173b659 Log client certificate type and length (#622)
* Log client certificate type and length

It appears that most client certificates are RSA certs, but we should
make sure that is indeed the case. Print out the strength of the cert
if it is RSA.

Also adds supports for TLS 1.3 and print out the supported cipher suites.

* Add a comment about zero length certificate

* Make length of non-RSA keys -1

* Don't use TLS 1.3 if JDK SSL provider is used
2020-06-11 17:11:40 -04:00
Weimin Yu d3fd826dc1 Load CommitLog into BEAM pipeline (#618)
* Load CommitLog into BEAM pipeline

Created tools that can generate CommitLogs on a test
datastore.

Defined BEAM transforms that load from CommitLog files and
added a few simple tests.

This is work-in-progress. Next step is to build a consistent
backup by merging Datastore exports and CommitLogs.
2020-06-11 11:38:53 -04:00
gbrodman 1c62728886 Rename V30 -> V31 to avoid duplicates (#621) 2020-06-10 16:08:31 -04:00
Lai Jiang b5d3186e67 Update Netty to the latest version (#620)
* Upgrade to the latest version of Netty

* Update lock files
2020-06-10 16:08:11 -04:00
gbrodman b4dfec5fd5 Rename client_id to registrar_id in SQL (#619)
We'll eventually want to shift everything over to using registrar_id and
registrarId rather than client_id and clientId but for the sake of the
Datastore schema and existing code, we won't change the Java identifier
for now. Once we're completely and only on SQL, we can rename the Java
field easily.
2020-06-10 15:11:27 -04:00
gbrodman 40b14fb695 Create a converter for sets of InetAddresses and use it in HostResource (#612)
* Create a converter for sets of inetAddresses and use it in HostResource

This can just be a set of strings where each string represents an
address;  there's no need for it to be a separate table. This allows
for simplification of the SQL schema.

* Regenerate golden SQL file after renaming v28 -> v29

* Add more tests and rename a typo in the file

* Refactor common test code and use tm methods

* Use JUnit5 API

* Rename test entity
2020-06-10 13:04:20 -04:00
Shicong Huang fdac686250 Add columns for TransferData in Domain and Contact (#577)
* Add columns for TransferData in Domain and Contact

* Rename flyway file and foreign key

* Rebase on master and address comment

* Compileable commit

* Fix unit test

* Refactor TransferServerApproveEntity

* Use tm().delete(vkeys)

* Rename transfer_period fields

* Rename client_id to registrar_id

* Rebase on master

* Resolve comment

* Rebase on master
2020-06-09 16:39:55 -04:00
Shicong Huang f0765dc893 Make JpaUnitTestRule not depend on the nomulus schema (#613) 2020-06-09 14:51:13 -04:00
Lai Jiang 2995bb03fd Update Javadoc URL (#615) 2020-06-09 10:25:56 -04:00
gbrodman 0f415f78a6 Use the correct text VKey for HostResource's superordinateDomain (#608)
* Store the superordinateDomain reference as a VKey rather than Key

This is a reference to a Domain object, so we should store it as a VKey
in reference to the Domain table. This should not affect any business
logic, but rather will allow us to set up the SQL tables for
HostResource et al. properly.
2020-06-08 12:21:51 -04:00
gbrodman b324fb98d3 Only use asymmetric VKeys for EPP resources (#611)
Given that we currently have no way of reconstituting a symmetric key
from the asymmetric key (or at least, we don't have a 100% reliable way
of doing so) it's best to keep keys as asymmetrical, referring to the
correct database. That way, we don't get situations where we cannot
compare equality of two keys due to one being asymmetrical and one being
symmetrical.
2020-06-05 16:55:12 -04:00
Lai Jiang d27fe8ead5 Fix a typo (#610) 2020-06-05 15:53:17 -04:00
Lai Jiang da65a38782 Add a GCB job to build and publish javadoc (#609) 2020-06-05 13:00:15 -04:00
Legina Chen 5a1f3d0376 Remove platformType and threatEntryMetaData fields from ThreatMatch (#607)
* Remove platformType and threatEntryMetaData fields from ThreatMatch

* Run google-java-format on both files

* Add test for removal of unnecessary fields

* Removed unnecessary fields from Spec11PipelineTest.testEndToEndPipeline_generatesExpectedFiles

* Added style check

* Fix typo
2020-06-05 09:00:07 -07:00
Shicong Huang b1241b98b2 Generate sql schema for PollMessage (#582)
* Generate sql schema for PollMessage

* Rework columns and resolve comments

* Fix datastore schema
2020-06-04 18:24:59 -04:00
Shicong Huang b42ded9451 Add test to verify the behavior of @DualDatabaseTest (#606) 2020-06-03 14:55:37 -04:00
Shicong Huang 472503541b Add deleteAll method to TransactionManager (#604)
* Add deleteAll method to TransactionManager

* Rename deleteAll to delete

* Add bucket.getLastWrittenTime() before second mutation
2020-06-03 10:02:48 -04:00
Weimin Yu ed64dd3548 Load raw records from Datastore export (#605)
* Load raw records from Datastore export

Created a tool that can export from a test instance of Datastore.

Defined Beam pipeline transforms for loading raw records back from
the export.

This is the first part of the effort to create a consistent snapshot
of Datastore. The next step is to load entity records from CommitLog
files.
2020-06-02 18:55:03 -04:00
Michael Muller 6a96b1a9cd Use TransactionManager for hosts and contacts (#603)
* Use TransactionManager for hosts and contacts

Replace Ofy calls with TransactionManager for most interactions involving
hosts and contacts.  In the course of this, also convert ForeignKeyIndex and
the EppResourceCache.

* Minor formatting fix
2020-06-02 13:17:16 -04:00
Michael Muller c23d4f3ba5 Add createVKey() at the EppResource level (#600)
* Add createVKey() at the EppResource level

Also convert createKey() to createVKey() to normalize with what we've settled
on.
2020-05-29 08:36:57 -04:00
Weimin Yu 2b794347e6 Refactor LevelDbFileBuilder to accept DS Entity (#599)
* Refactor LevelDbFileBuilder to accept DS Entity

Builder now can directly work with Datastore Entity objects.
No need to wrap data in ComparableEntity.
2020-05-28 13:38:00 -04:00
Shicong Huang 26fb5388a4 Generate sql schema for BillingEvent (#565)
* Generate sql schema for BillingEvent

* Change to use sequence

* Address comments

* Resolve warnings and remove duplicate cost related fields

* Increase the flayway file version to V25

* Remove extra space

* Split to 3 tables, merge VKey

* Rename talbes

* Rename repoId to domainRepoId

* Exclude VKey in schema.txt

* Rename target_id to domain_name

* Fix javadoc

* Resolve comments
2020-05-27 15:59:19 -04:00
Lai Jiang bd443633f6 Add a task to compile javadoc across all packages (#597)
Also fixes various issues that prevent javadoc compliation.
2020-05-27 10:33:46 -04:00
Weimin Yu d87f119b36 Add a test for SQL logging config (#598)
* Add a test for SQL logging config

Verifies that SQL statements are logged by Hibernate when
configured to do so.
2020-05-26 16:25:33 -04:00
Weimin Yu 54f1357d83 Fix show-sql which stopped working (#596)
* Fix show-sql which stopped working

Made show-sql property configurable in JpaUnitTestRules.

Added a few comments on foreign key constraint behavior.
2020-05-21 12:20:56 -04:00
Lai Jiang c73d154084 Do not enqueue update snapshot task if import fails (#578)
If the import from Datastore to BigQuery fails, there is no point
enqueuing a job to update the snapshot view.

Also when there's an error updating the snapshot view, log it at severe
level. The HTTP exception thrown is logged at info and triggers a retry
implicitly. I'm not sure if we want this behavior though. Do we want to
retry upon snapshot updating failures? Unless the failurs are transient,
retrying doesn't help. In our case the failure (End of time out of range
in Standard SQL) is not transient.
2020-05-21 11:40:45 -04:00
gbrodman 259d2e2cdc Run "npm audit fix" to fix a vulnerability (#592) 2020-05-20 15:12:27 -04:00
Shicong Huang 0f174d9ce0 Add all existing entities to VKeyTranslatorFactory (#595)
EntityClasses.ALL_CLASSES has all of our registered entities so
we can use it to initialize VKeyTranslatorFactory.classRegistry
to avoid adding them one by one.

Also, this PR changed to use Key.getKind() to get the kind of
the entity to solve the problem that when the entity class
is an inner class, its kind should still be the class name
instead of OuterClass$InnerClass.
2020-05-20 14:24:45 -04:00
Weimin Yu ca2edb6a17 Close input channel in LevelDbLogReader (#594)
* Close input channel in LevelDbLogReader

Input channel should be closed when all data has been read.
2020-05-20 12:54:13 -04:00
Weimin Yu 3947ac6ef7 Read LevelDb incrementally (#593)
* Read LevelDb incrementally

Made LevelDbLogReader an iterator over a LevelDb data stream,
Reducing memory footprint which is important when used in a
Dataflow pipeline.
2020-05-20 10:26:34 -04:00
Michael Muller 579a3d0ac1 Make VKey persist to datastore as a key (#591)
* Make VKey persist to datastore as a key

Convert nsHosts entirely to VKey as a proof-of-concept.

Tested as follows:
    1) Deployed to crash, verified that nameservers were visible for several
       domains (indicating that we are able to load a set of Keys as VKeys)
    2) Updated the set of nameservers for a domain (removing some initial
       hosts) and verified that the changes went through.
    3) Deployed the old version to crash, verified that I was able to retrieve
       the newly saved VKeys as Keys.
    4) Modified the hosts for the same domain (adding back one of the hosts)
       and verified that the change took effect.
    5) Redeployed this change to crash, again updated the nameservers to add
       another host.
    6) Again restored the old version, verified that the new hosts were
       visible.

* Changes in response to review

* Convert to a single VKeyTranslatorFactory instance

* Moved vkey field rename to V25
2020-05-19 14:10:28 -04:00
Lai Jiang 5fe929b027 Log InternalServerErrorException at SEVERE (#585)
Normal HttpException logs at INFO because they usual do not indicate
anything out of the ordinary and is meant to convey to the client that
there is some expected error. However InternalServerErrorException is
something that we do care about being alerted for so we log it at SEVERE.
2020-05-18 22:55:13 -04:00
Lai Jiang fb335b7d89 Upgrade to Gradle 6.4.1 (#590) 2020-05-18 16:47:02 -04:00
Shicong Huang a0f4013d53 Add JUnit5 extension to run test twice against different databases (#588)
* Add JUnit5 extension to run test against different databases

* Fix typos

* Add some explanation
2020-05-18 11:06:21 -04:00
Lai Jiang 5e596bb389 Upgrade to Gradle 6.4 (#589) 2020-05-14 14:57:24 -04:00
Lai Jiang f62fd82803 Log information about SSL connection from the client (#586) 2020-05-14 09:38:33 -04:00
sarahcaseybot b7353ef338 Add TimedTransitionProperty Converters (#561)
* Add TimedTldStateTransitionMapConverter

* Move timedTransitions to a base class and add BillingCostTransitionConverter

* Add test of TimedTransitionPropertyConverterBase

* clean up tests

* Switch tests to JUnit 5

* Make JpaUnitTestRule an extension
2020-05-12 11:46:19 -04:00
268 changed files with 6424 additions and 1958 deletions
+1 -1
View File
@@ -33,7 +33,7 @@ running system:
* View the source code for the [GAE app](https://github.com/google/nomulus/tree/master/core/src/main/java/google/registry)
and for the [GKE proxy](https://github.com/google/nomulus/tree/master/proxy/src/main/java/google/registry)
* [Other docs](https://github.com/google/nomulus/tree/master/docs)
* [Javadoc](https://nomulus.foo/javadoc/latest/)
* [Javadoc](https://javadoc.nomulus.foo/)
* [Nomulus discussion
group](https://groups.google.com/forum/#!forum/nomulus-discuss), for any
other questions
+24
View File
@@ -197,6 +197,10 @@ task runPresubmits(type: Exec) {
args('config/presubmits.py')
}
def javadocSource = []
def javadocClasspath = []
def javadocDependentTasks = []
subprojects {
// Skip no-op project
if (project.name == 'services') return
@@ -317,6 +321,10 @@ subprojects {
}
}
}
javadocSource << project.sourceSets.main.allJava
javadocClasspath << project.sourceSets.main.compileClasspath
javadocDependentTasks << project.tasks.compileJava
}
// If "-P verboseTestOutput=true" is passed in, configure all subprojects to dump all of their
@@ -445,3 +453,19 @@ task javaIncrementalFormatApply {
}
tasks.build.dependsOn(tasks.javaIncrementalFormatCheck)
task javadoc(type: Javadoc) {
source javadocSource
classpath = files(javadocClasspath)
destinationDir = file("${buildDir}/docs/javadoc")
options.encoding = "UTF-8"
// In a lot of places we don't write @return so suppress warnings about that.
options.addBooleanOption('Xdoclint:all,-missing', true)
options.addBooleanOption("-allow-script-in-comments",true)
options.tags = ["type:a:Generic Type",
"error:a:Expected Error"]
}
tasks.build.dependsOn(tasks.javadoc)
javadocDependentTasks.each { tasks.javadoc.dependsOn(it) }
@@ -21,7 +21,6 @@ import org.joda.time.ReadableDuration;
* An object which accepts requests to put the current thread to sleep.
*
* @see SystemSleeper
* @see google.registry.testing.FakeSleeper
*/
@ThreadSafe
public interface Sleeper {
+3
View File
@@ -880,6 +880,9 @@ task standardTest(type: FilteringTest) {
// forkEvery 1
// Sets the maximum number of test executors that may exist at the same time.
// Also, Gradle executes tests in 1 thread and some of our test infrastructures
// depend on that, e.g. DualDatabaseTestInvocationContextProvider injects
// different implementation of TransactionManager into TransactionManagerFactory.
maxParallelForks 5
systemProperty 'test.projectRoot', rootProject.projectRootDir
@@ -0,0 +1,66 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import com.google.apphosting.api.ApiProxy;
import com.google.apphosting.api.ApiProxy.Environment;
import com.google.common.collect.ImmutableMap;
import java.io.Closeable;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
/**
* Sets up a placeholder {@link Environment} on a non-AppEngine platform so that Datastore Entities
* can be deserialized. See {@code DatastoreEntityExtension} in test source for more information.
*/
public class AppEngineEnvironment implements Closeable {
private static final Environment PLACEHOLDER_ENV = createAppEngineEnvironment();
private boolean isPlaceHolderNeeded;
AppEngineEnvironment() {
isPlaceHolderNeeded = ApiProxy.getCurrentEnvironment() == null;
// isPlaceHolderNeeded may be true when we are invoked in a test with AppEngineRule.
if (isPlaceHolderNeeded) {
ApiProxy.setEnvironmentForCurrentThread(PLACEHOLDER_ENV);
}
}
@Override
public void close() {
if (isPlaceHolderNeeded) {
ApiProxy.setEnvironmentForCurrentThread(null);
}
}
/** Returns a placeholder {@link Environment} that can return hardcoded AppId and Attributes. */
private static Environment createAppEngineEnvironment() {
return (Environment)
Proxy.newProxyInstance(
Environment.class.getClassLoader(),
new Class[] {Environment.class},
(Object proxy, Method method, Object[] args) -> {
switch (method.getName()) {
case "getAppId":
return "PlaceholderAppId";
case "getAttributes":
return ImmutableMap.<String, Object>of();
default:
throw new UnsupportedOperationException(method.getName());
}
});
}
}
@@ -0,0 +1,95 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.backup.BackupUtils.createDeserializingIterator;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Streams;
import google.registry.model.ImmutableObject;
import google.registry.model.ofy.CommitLogCheckpoint;
import google.registry.model.ofy.CommitLogManifest;
import google.registry.model.ofy.CommitLogMutation;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.channels.Channels;
import java.nio.channels.ReadableByteChannel;
import java.util.Iterator;
import java.util.stream.Stream;
/**
* Helpers for reading CommitLog records from a file.
*
* <p>This class is adapted from {@link RestoreCommitLogsAction}, and will be used in the initial
* population of the Cloud SQL database.
*/
public final class CommitLogImports {
private CommitLogImports() {}
/**
* Returns entities in an {@code inputStream} (from a single CommitLog file) as an {@link
* ImmutableList} of {@link VersionedEntity} records. Upon completion the {@code inputStream} is
* closed.
*
* <p>The returned list may be empty, since CommitLogs are written at fixed intervals regardless
* if actual changes exist.
*
* <p>A CommitLog file starts with a {@link CommitLogCheckpoint}, followed by (repeated)
* subsequences of [{@link CommitLogManifest}, [{@link CommitLogMutation}] ...]. Each subsequence
* represents the changes in one transaction. The {@code CommitLogManifest} contains deleted
* entity keys, whereas each {@code CommitLogMutation} contains one whole entity.
*/
public static ImmutableList<VersionedEntity> loadEntities(InputStream inputStream) {
try (AppEngineEnvironment appEngineEnvironment = new AppEngineEnvironment();
InputStream input = new BufferedInputStream(inputStream)) {
Iterator<ImmutableObject> commitLogs = createDeserializingIterator(input);
checkState(commitLogs.hasNext());
checkState(commitLogs.next() instanceof CommitLogCheckpoint);
return Streams.stream(commitLogs)
.map(
e ->
e instanceof CommitLogManifest
? VersionedEntity.fromManifest((CommitLogManifest) e)
: Stream.of(VersionedEntity.fromMutation((CommitLogMutation) e)))
.flatMap(s -> s)
.collect(ImmutableList.toImmutableList());
} catch (IOException e) {
throw new RuntimeException(e);
}
}
/** Covenience method that adapts {@link #loadEntities(InputStream)} to a {@link File}. */
public static ImmutableList<VersionedEntity> loadEntities(File commitLogFile) {
try {
return loadEntities(new FileInputStream(commitLogFile));
} catch (IOException e) {
throw new RuntimeException(e);
}
}
/**
* Covenience method that adapts {@link #loadEntities(InputStream)} to a {@link
* ReadableByteChannel}.
*/
public static ImmutableList<VersionedEntity> loadEntities(ReadableByteChannel channel) {
return loadEntities(Channels.newInputStream(channel));
}
}
@@ -0,0 +1,160 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.EntityTranslator;
import com.google.appengine.api.datastore.Key;
import com.google.auto.value.AutoValue;
import com.google.auto.value.extension.memoized.Memoized;
import google.registry.model.ofy.CommitLogManifest;
import google.registry.model.ofy.CommitLogMutation;
import java.io.Serializable;
import java.util.Arrays;
import java.util.Optional;
import java.util.stream.Stream;
import javax.annotation.Nullable;
/**
* A Datastore {@link Entity Entity's} timestamped state.
*
* <p>For a new or updated Entity, its ProtocolBuffer bytes are stored along with its {@link Key}.
* For a deleted entity, only its {@link Key} is stored, and the {@link #entityProtoBytes} is left
* as null.
*
* <p>Note that {@link Optional java.util.Optional} is not serializable, therefore cannot be used as
* property type in this class.
*/
@AutoValue
public abstract class VersionedEntity implements Serializable {
private static final long serialVersionUID = 1L;
public abstract long commitTimeMills();
/** The {@link Key} of the {@link Entity}. */
public abstract Key key();
/** Serialized form of the {@link Entity}. This property is {@code null} for a deleted Entity. */
@Nullable
abstract ImmutableBytes entityProtoBytes();
@Memoized
public Optional<Entity> getEntity() {
return Optional.ofNullable(entityProtoBytes())
.map(ImmutableBytes::getBytes)
.map(EntityTranslator::createFromPbBytes);
}
public boolean isDelete() {
return entityProtoBytes() == null;
}
/**
* Converts deleted entity keys in {@code manifest} into a {@link Stream} of {@link
* VersionedEntity VersionedEntities}. See {@link CommitLogImports#loadEntities} for more
* information.
*/
public static Stream<VersionedEntity> fromManifest(CommitLogManifest manifest) {
long commitTimeMillis = manifest.getCommitTime().getMillis();
return manifest.getDeletions().stream()
.map(com.googlecode.objectify.Key::getRaw)
.map(key -> builder().commitTimeMills(commitTimeMillis).key(key).build());
}
/* Converts a {@link CommitLogMutation} to a {@link VersionedEntity}. */
public static VersionedEntity fromMutation(CommitLogMutation mutation) {
return from(
com.googlecode.objectify.Key.create(mutation).getParent().getId(),
mutation.getEntityProtoBytes());
}
public static VersionedEntity from(long commitTimeMillis, byte[] entityProtoBytes) {
return builder()
.entityProtoBytes(entityProtoBytes)
.key(EntityTranslator.createFromPbBytes(entityProtoBytes).getKey())
.commitTimeMills(commitTimeMillis)
.build();
}
static Builder builder() {
return new AutoValue_VersionedEntity.Builder();
}
@AutoValue.Builder
public abstract static class Builder {
public abstract Builder commitTimeMills(long commitTimeMillis);
abstract Builder entityProtoBytes(ImmutableBytes bytes);
public abstract Builder key(Key key);
public abstract VersionedEntity build();
public Builder entityProtoBytes(byte[] bytes) {
return entityProtoBytes(new ImmutableBytes(bytes));
}
}
/**
* Wraps a byte array and prevents it from being modified by its original owner.
*
* <p>While this class seems an overkill, it exists for two reasons:
*
* <ul>
* <li>It is easier to override the {@link #equals} method here (for value-equivalence check)
* than to override the AutoValue-generated {@code equals} method.
* <li>To appease the style checker, which forbids arrays as AutoValue property.
* </ul>
*/
static final class ImmutableBytes implements Serializable {
private static final long serialVersionUID = 1L;
private final byte[] bytes;
ImmutableBytes(byte[] bytes) {
this.bytes = Arrays.copyOf(bytes, bytes.length);
}
/**
* Returns the saved byte array. Invocation is restricted to trusted callers, who must not
* modify the array.
*/
byte[] getBytes() {
return bytes;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof ImmutableBytes)) {
return false;
}
ImmutableBytes that = (ImmutableBytes) o;
// Do not use Objects.equals, which checks reference identity instead of data in array.
return Arrays.equals(bytes, that.bytes);
}
@Override
public int hashCode() {
// Do not use Objects.hashCode, which hashes the reference, not the data in array.
return Arrays.hashCode(bytes);
}
}
}
@@ -150,7 +150,7 @@ public final class AsyncTaskEnqueuer {
/** Enqueues a task to asynchronously refresh DNS for a renamed host. */
public void enqueueAsyncDnsRefresh(HostResource host, DateTime now) {
VKey<HostResource> hostKey = host.createKey();
VKey<HostResource> hostKey = host.createVKey();
logger.atInfo().log("Enqueuing async DNS refresh for renamed host %s.", hostKey);
addTaskToQueueWithRetry(
asyncDnsRefreshPullQueue,
@@ -347,7 +347,7 @@ public class DeleteContactsAndHostsAction implements Runnable {
String resourceClientId = resource.getPersistedCurrentSponsorClientId();
if (resource instanceof HostResource && ((HostResource) resource).isSubordinate()) {
resourceClientId =
ofy().load().key(((HostResource) resource).getSuperordinateDomain()).now()
tm().load(((HostResource) resource).getSuperordinateDomain())
.cloneProjectedAtTime(now)
.getCurrentSponsorClientId();
}
@@ -466,10 +466,11 @@ public class DeleteContactsAndHostsAction implements Runnable {
HostResource host = (HostResource) existingResource;
if (host.isSubordinate()) {
dnsQueue.addHostRefreshTask(host.getFullyQualifiedHostName());
ofy().save().entity(
ofy().load().key(host.getSuperordinateDomain()).now().asBuilder()
.removeSubordinateHost(host.getFullyQualifiedHostName())
.build());
tm().saveNewOrUpdate(
tm().load(host.getSuperordinateDomain())
.asBuilder()
.removeSubordinateHost(host.getFullyQualifiedHostName())
.build());
}
} else {
throw new IllegalStateException(
@@ -155,89 +155,100 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
}
int numBillingEventsSaved = 0;
try {
numBillingEventsSaved = tm().transactNew(() -> {
ImmutableSet.Builder<OneTime> syntheticOneTimesBuilder =
new ImmutableSet.Builder<>();
final Registry tld = Registry.get(getTldFromDomainName(recurring.getTargetId()));
numBillingEventsSaved =
tm().transactNew(
() -> {
ImmutableSet.Builder<OneTime> syntheticOneTimesBuilder =
new ImmutableSet.Builder<>();
final Registry tld =
Registry.get(getTldFromDomainName(recurring.getTargetId()));
// Determine the complete set of times at which this recurring event should occur
// (up to and including the runtime of the mapreduce).
Iterable<DateTime> eventTimes =
recurring.getRecurrenceTimeOfYear().getInstancesInRange(Range.closed(
recurring.getEventTime(),
earliestOf(recurring.getRecurrenceEndTime(), executeTime)));
// Determine the complete set of times at which this recurring event should
// occur (up to and including the runtime of the mapreduce).
Iterable<DateTime> eventTimes =
recurring
.getRecurrenceTimeOfYear()
.getInstancesInRange(
Range.closed(
recurring.getEventTime(),
earliestOf(recurring.getRecurrenceEndTime(), executeTime)));
// Convert these event times to billing times
final ImmutableSet<DateTime> billingTimes =
getBillingTimesInScope(eventTimes, cursorTime, executeTime, tld);
// Convert these event times to billing times
final ImmutableSet<DateTime> billingTimes =
getBillingTimesInScope(eventTimes, cursorTime, executeTime, tld);
Key<? extends EppResource> domainKey = recurring.getParentKey().getParent();
Iterable<OneTime> oneTimesForDomain =
ofy().load().type(OneTime.class).ancestor(domainKey);
Key<? extends EppResource> domainKey = recurring.getParentKey().getParent();
Iterable<OneTime> oneTimesForDomain =
ofy().load().type(OneTime.class).ancestor(domainKey);
// Determine the billing times that already have OneTime events persisted.
ImmutableSet<DateTime> existingBillingTimes =
getExistingBillingTimes(oneTimesForDomain, recurring);
// Determine the billing times that already have OneTime events persisted.
ImmutableSet<DateTime> existingBillingTimes =
getExistingBillingTimes(oneTimesForDomain, recurring);
ImmutableSet.Builder<HistoryEntry> historyEntriesBuilder =
new ImmutableSet.Builder<>();
// Create synthetic OneTime events for all billing times that do not yet have an event
// persisted.
for (DateTime billingTime : difference(billingTimes, existingBillingTimes)) {
// Construct a new HistoryEntry that parents over the OneTime
HistoryEntry historyEntry = new HistoryEntry.Builder()
.setBySuperuser(false)
.setClientId(recurring.getClientId())
.setModificationTime(tm().getTransactionTime())
.setParent(domainKey)
.setPeriod(Period.create(1, YEARS))
.setReason("Domain autorenewal by ExpandRecurringBillingEventsAction")
.setRequestedByRegistrar(false)
.setType(DOMAIN_AUTORENEW)
// Don't write a domain transaction record if the recurrence was ended prior to the
// billing time (i.e. a domain was deleted during the autorenew grace period).
.setDomainTransactionRecords(
recurring.getRecurrenceEndTime().isBefore(billingTime)
? ImmutableSet.of()
: ImmutableSet.of(
DomainTransactionRecord.create(
tld.getTldStr(),
// We report this when the autorenew grace period ends
billingTime,
TransactionReportField.netRenewsFieldFromYears(1),
1)))
.build();
historyEntriesBuilder.add(historyEntry);
ImmutableSet.Builder<HistoryEntry> historyEntriesBuilder =
new ImmutableSet.Builder<>();
// Create synthetic OneTime events for all billing times that do not yet have
// an event persisted.
for (DateTime billingTime : difference(billingTimes, existingBillingTimes)) {
// Construct a new HistoryEntry that parents over the OneTime
HistoryEntry historyEntry =
new HistoryEntry.Builder()
.setBySuperuser(false)
.setClientId(recurring.getClientId())
.setModificationTime(tm().getTransactionTime())
.setParent(domainKey)
.setPeriod(Period.create(1, YEARS))
.setReason(
"Domain autorenewal by ExpandRecurringBillingEventsAction")
.setRequestedByRegistrar(false)
.setType(DOMAIN_AUTORENEW)
// Don't write a domain transaction record if the recurrence was
// ended prior to the billing time (i.e. a domain was deleted
// during the autorenew grace period).
.setDomainTransactionRecords(
recurring.getRecurrenceEndTime().isBefore(billingTime)
? ImmutableSet.of()
: ImmutableSet.of(
DomainTransactionRecord.create(
tld.getTldStr(),
// We report this when the autorenew grace period
// ends
billingTime,
TransactionReportField.netRenewsFieldFromYears(1),
1)))
.build();
historyEntriesBuilder.add(historyEntry);
DateTime eventTime = billingTime.minus(tld.getAutoRenewGracePeriodLength());
// Determine the cost for a one-year renewal.
Money renewCost = getDomainRenewCost(recurring.getTargetId(), eventTime, 1);
syntheticOneTimesBuilder.add(new OneTime.Builder()
.setBillingTime(billingTime)
.setClientId(recurring.getClientId())
.setCost(renewCost)
.setEventTime(eventTime)
.setFlags(union(recurring.getFlags(), Flag.SYNTHETIC))
.setParent(historyEntry)
.setPeriodYears(1)
.setReason(recurring.getReason())
.setSyntheticCreationTime(executeTime)
.setCancellationMatchingBillingEvent(Key.create(recurring))
.setTargetId(recurring.getTargetId())
.build());
}
Set<HistoryEntry> historyEntries = historyEntriesBuilder.build();
Set<OneTime> syntheticOneTimes = syntheticOneTimesBuilder.build();
if (!isDryRun) {
ImmutableSet<ImmutableObject> entitiesToSave =
new ImmutableSet.Builder<ImmutableObject>()
.addAll(historyEntries)
.addAll(syntheticOneTimes)
.build();
ofy().save().entities(entitiesToSave).now();
}
return syntheticOneTimes.size();
});
DateTime eventTime = billingTime.minus(tld.getAutoRenewGracePeriodLength());
// Determine the cost for a one-year renewal.
Money renewCost = getDomainRenewCost(recurring.getTargetId(), eventTime, 1);
syntheticOneTimesBuilder.add(
new OneTime.Builder()
.setBillingTime(billingTime)
.setClientId(recurring.getClientId())
.setCost(renewCost)
.setEventTime(eventTime)
.setFlags(union(recurring.getFlags(), Flag.SYNTHETIC))
.setParent(historyEntry)
.setPeriodYears(1)
.setReason(recurring.getReason())
.setSyntheticCreationTime(executeTime)
.setCancellationMatchingBillingEvent(recurring.createVKey())
.setTargetId(recurring.getTargetId())
.build());
}
Set<HistoryEntry> historyEntries = historyEntriesBuilder.build();
Set<OneTime> syntheticOneTimes = syntheticOneTimesBuilder.build();
if (!isDryRun) {
ImmutableSet<ImmutableObject> entitiesToSave =
new ImmutableSet.Builder<ImmutableObject>()
.addAll(historyEntries)
.addAll(syntheticOneTimes)
.build();
ofy().save().entities(entitiesToSave).now();
}
return syntheticOneTimes.size();
});
} catch (Throwable t) {
getContext().incrementCounter("error: " + t.getClass().getSimpleName());
getContext().incrementCounter(ERROR_COUNTER);
@@ -279,7 +290,8 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
return Streams.stream(oneTimesForDomain)
.filter(
billingEvent ->
Key.create(recurringEvent)
recurringEvent
.createVKey()
.equals(billingEvent.getCancellationMatchingBillingEvent()))
.map(OneTime::getBillingTime)
.collect(toImmutableSet());
@@ -0,0 +1,74 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Strings.isNullOrEmpty;
import org.joda.time.DateTime;
/**
* Helpers for determining the fully qualified paths to Nomulus backup files. A backup consists of a
* Datastore export and Nomulus CommitLogs that overlap with the export.
*/
public final class BackupPaths {
private BackupPaths() {}
private static final String WILDCARD_CHAR = "*";
private static final String EXPORT_PATTERN_TEMPLATE = "%s/all_namespaces/kind_%s/input-%s";
public static final String COMMIT_LOG_NAME_PREFIX = "commit_diff_until_";
private static final String COMMIT_LOG_PATTERN_TEMPLATE = "%s/" + COMMIT_LOG_NAME_PREFIX + "*";
/**
* Returns a regex pattern that matches all Datastore export files of a given {@code kind}.
*
* @param exportDir path to the top directory of a Datastore export
* @param kind the 'kind' of the Datastore entity
*/
public static String getExportFileNamePattern(String exportDir, String kind) {
checkArgument(!isNullOrEmpty(exportDir), "Null or empty exportDir.");
checkArgument(!isNullOrEmpty(kind), "Null or empty kind.");
return String.format(EXPORT_PATTERN_TEMPLATE, exportDir, kind, WILDCARD_CHAR);
}
/**
* Returns the fully qualified path of a Datastore export file with the given {@code kind} and
* {@code shard}.
*
* @param exportDir path to the top directory of a Datastore export
* @param kind the 'kind' of the Datastore entity
* @param shard an integer suffix of the file name
*/
public static String getExportFileNameByShard(String exportDir, String kind, int shard) {
checkArgument(!isNullOrEmpty(exportDir), "Null or empty exportDir.");
checkArgument(!isNullOrEmpty(kind), "Null or empty kind.");
checkArgument(shard >= 0, "Negative shard %s not allowed.", shard);
return String.format(EXPORT_PATTERN_TEMPLATE, exportDir, kind, Integer.toString(shard));
}
public static String getCommitLogFileNamePattern(String commitLogDir) {
return String.format(COMMIT_LOG_PATTERN_TEMPLATE, commitLogDir);
}
/** Gets the Commit timestamp from a CommitLog file name. */
public static DateTime getCommitLogTimestamp(String fileName) {
checkArgument(!isNullOrEmpty(fileName), "Null or empty fileName.");
int start = fileName.lastIndexOf(COMMIT_LOG_NAME_PREFIX);
checkArgument(start >= 0, "Illegal file name %s.", fileName);
return DateTime.parse(fileName.substring(start + COMMIT_LOG_NAME_PREFIX.length()));
}
}
@@ -0,0 +1,109 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static google.registry.beam.initsql.BackupPaths.getCommitLogFileNamePattern;
import static google.registry.beam.initsql.BackupPaths.getCommitLogTimestamp;
import static google.registry.beam.initsql.Transforms.processFiles;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import google.registry.backup.CommitLogImports;
import google.registry.backup.VersionedEntity;
import java.io.IOException;
import org.apache.beam.sdk.coders.StringUtf8Coder;
import org.apache.beam.sdk.io.FileIO.ReadableFile;
import org.apache.beam.sdk.io.fs.MatchResult.Metadata;
import org.apache.beam.sdk.transforms.Create;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.PTransform;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.values.PBegin;
import org.apache.beam.sdk.values.PCollection;
import org.joda.time.DateTime;
/**
* {@link org.apache.beam.sdk.transforms.PTransform Pipeline transforms} for loading from Nomulus
* CommitLog files. They are all part of a transformation that loads raw records from a sequence of
* Datastore CommitLog files, and are broken apart for testing.
*/
public class CommitLogTransforms {
/**
* Returns a {@link PTransform transform} that can generate a collection of patterns that match
* all Datastore CommitLog files.
*/
public static PTransform<PBegin, PCollection<String>> getCommitLogFilePatterns(
String commitLogDir) {
return Create.of(getCommitLogFileNamePattern(commitLogDir)).withCoder(StringUtf8Coder.of());
}
/**
* Returns files with timestamps between {@code fromTime} (inclusive) and {@code endTime}
* (exclusive).
*/
public static PTransform<PCollection<? extends String>, PCollection<String>>
filterCommitLogsByTime(DateTime fromTime, DateTime toTime) {
checkNotNull(fromTime, "fromTime");
checkNotNull(toTime, "toTime");
checkArgument(
fromTime.isBefore(toTime),
"Invalid time range: fromTime (%s) is before endTime (%s)",
fromTime,
toTime);
return ParDo.of(new FilterCommitLogFileByTime(fromTime, toTime));
}
/** Returns a {@link PTransform} from file {@link Metadata} to {@link VersionedEntity}. */
public static PTransform<PCollection<Metadata>, PCollection<VersionedEntity>>
loadCommitLogsFromFiles() {
return processFiles(new LoadOneCommitLogsFile());
}
static class FilterCommitLogFileByTime extends DoFn<String, String> {
private final DateTime fromTime;
private final DateTime toTime;
public FilterCommitLogFileByTime(DateTime fromTime, DateTime toTime) {
this.fromTime = fromTime;
this.toTime = toTime;
}
@ProcessElement
public void processElement(@Element String fileName, OutputReceiver<String> out) {
DateTime timestamp = getCommitLogTimestamp(fileName);
if (isBeforeOrAt(fromTime, timestamp) && timestamp.isBefore(toTime)) {
out.output(fileName);
}
}
}
/**
* Reads a CommitLog file and converts its content into {@link VersionedEntity VersionedEntities}.
*/
static class LoadOneCommitLogsFile extends DoFn<ReadableFile, VersionedEntity> {
@ProcessElement
public void processElement(@Element ReadableFile file, OutputReceiver<VersionedEntity> out) {
try {
CommitLogImports.loadEntities(file.open()).forEach(out::output);
} catch (IOException e) {
// Let the pipeline retry the whole file.
throw new RuntimeException(e);
}
}
}
}
@@ -0,0 +1,82 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import static google.registry.beam.initsql.BackupPaths.getExportFileNamePattern;
import static google.registry.beam.initsql.Transforms.processFiles;
import com.google.common.collect.ImmutableList;
import google.registry.backup.VersionedEntity;
import google.registry.tools.LevelDbLogReader;
import java.io.IOException;
import java.util.Collection;
import org.apache.beam.sdk.coders.StringUtf8Coder;
import org.apache.beam.sdk.io.FileIO.ReadableFile;
import org.apache.beam.sdk.io.fs.MatchResult.Metadata;
import org.apache.beam.sdk.transforms.Create;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.PTransform;
import org.apache.beam.sdk.values.PBegin;
import org.apache.beam.sdk.values.PCollection;
/**
* {@link PTransform Pipeline transforms} for loading from Datastore export files. They are all part
* of a transformation that loads raw records from a Datastore export, and are broken apart for
* testing.
*/
public class ExportLoadingTransforms {
/**
* Returns a {@link PTransform transform} that can generate a collection of patterns that match
* all Datastore export files of the given {@code kinds}.
*/
public static PTransform<PBegin, PCollection<String>> getDatastoreExportFilePatterns(
String exportDir, Collection<String> kinds) {
return Create.of(
kinds.stream()
.map(kind -> getExportFileNamePattern(exportDir, kind))
.collect(ImmutableList.toImmutableList()))
.withCoder(StringUtf8Coder.of());
}
/** Returns a {@link PTransform} from file {@link Metadata} to {@link VersionedEntity}. */
public static PTransform<PCollection<Metadata>, PCollection<VersionedEntity>>
loadExportDataFromFiles() {
return processFiles(new LoadOneExportShard());
}
/**
* Reads a LevelDb file and converts each raw record into a {@link VersionedEntity}. All such
* entities use {@link Long#MIN_VALUE} as timestamp, so that they go before data from CommitLogs.
*
* <p>LevelDb files are not seekable because a large object may span multiple blocks. If a
* sequential read fails, the file needs to be retried from the beginning.
*/
private static class LoadOneExportShard extends DoFn<ReadableFile, VersionedEntity> {
private static final long TIMESTAMP = Long.MIN_VALUE;
@ProcessElement
public void processElement(@Element ReadableFile file, OutputReceiver<VersionedEntity> output) {
try {
LevelDbLogReader.from(file.open())
.forEachRemaining(record -> output.output(VersionedEntity.from(TIMESTAMP, record)));
} catch (IOException e) {
// Let the pipeline retry the whole file.
throw new RuntimeException(e);
}
}
}
}
@@ -0,0 +1,3 @@
## Summary
This package contains a BEAM pipeline that populates a Cloud SQL database from a Datastore backup.
@@ -0,0 +1,61 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import google.registry.backup.VersionedEntity;
import org.apache.beam.sdk.io.Compression;
import org.apache.beam.sdk.io.FileIO;
import org.apache.beam.sdk.io.FileIO.ReadableFile;
import org.apache.beam.sdk.io.fs.EmptyMatchTreatment;
import org.apache.beam.sdk.io.fs.MatchResult.Metadata;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.PTransform;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.values.PCollection;
/**
* Common {@link PTransform pipeline transforms} used in pipelines that load from both Datastore
* export files and Nomulus CommitLog files.
*/
public class Transforms {
/**
* Returns a {@link PTransform} from file name patterns to file {@link Metadata Metadata records}.
*/
public static PTransform<PCollection<String>, PCollection<Metadata>> getFilesByPatterns() {
return new PTransform<PCollection<String>, PCollection<Metadata>>() {
@Override
public PCollection<Metadata> expand(PCollection<String> input) {
return input.apply(FileIO.matchAll().withEmptyMatchTreatment(EmptyMatchTreatment.DISALLOW));
}
};
}
/**
* Returns a {@link PTransform} from file {@link Metadata} to {@link VersionedEntity} using
* caller-provided {@code transformer}.
*/
public static PTransform<PCollection<Metadata>, PCollection<VersionedEntity>> processFiles(
DoFn<ReadableFile, VersionedEntity> transformer) {
return new PTransform<PCollection<Metadata>, PCollection<VersionedEntity>>() {
@Override
public PCollection<VersionedEntity> expand(PCollection<Metadata> input) {
return input
.apply(FileIO.readMatches().withCompression(Compression.UNCOMPRESSED))
.apply(transformer.getClass().getSimpleName(), ParDo.of(transformer));
}
};
}
}
@@ -24,22 +24,10 @@ import org.json.JSONObject;
public abstract class ThreatMatch implements Serializable {
private static final String THREAT_TYPE_FIELD = "threatType";
private static final String PLATFORM_TYPE_FIELD = "platformType";
private static final String METADATA_FIELD = "threatEntryMetadata";
private static final String DOMAIN_NAME_FIELD = "fullyQualifiedDomainName";
/** Returns what kind of threat it is (malware, phishing etc.) */
public abstract String threatType();
/** Returns what platforms it affects (Windows, Linux etc.) */
abstract String platformType();
/**
* Returns a String representing a JSON Object containing arbitrary metadata associated with this
* threat, or "NONE" if there is no metadata to retrieve.
*
* <p>This ideally would be a {@link JSONObject} type, but can't be due to serialization
* requirements.
*/
abstract String metadata();
/** Returns the fully qualified domain name [SLD].[TLD] of the matched threat. */
public abstract String fullyQualifiedDomainName();
@@ -52,29 +40,19 @@ public abstract class ThreatMatch implements Serializable {
static ThreatMatch create(JSONObject threatMatchJSON, String fullyQualifiedDomainName)
throws JSONException {
return new AutoValue_ThreatMatch(
threatMatchJSON.getString(THREAT_TYPE_FIELD),
threatMatchJSON.getString(PLATFORM_TYPE_FIELD),
threatMatchJSON.has(METADATA_FIELD)
? threatMatchJSON.getJSONObject(METADATA_FIELD).toString()
: "NONE",
fullyQualifiedDomainName);
threatMatchJSON.getString(THREAT_TYPE_FIELD), fullyQualifiedDomainName);
}
/** Returns a {@link JSONObject} representing a subset of this object's data. */
JSONObject toJSON() throws JSONException {
return new JSONObject()
.put(THREAT_TYPE_FIELD, threatType())
.put(PLATFORM_TYPE_FIELD, platformType())
.put(METADATA_FIELD, metadata())
.put(DOMAIN_NAME_FIELD, fullyQualifiedDomainName());
}
/** Parses a {@link JSONObject} and returns an equivalent {@link ThreatMatch}. */
public static ThreatMatch fromJSON(JSONObject threatMatch) throws JSONException {
return new AutoValue_ThreatMatch(
threatMatch.getString(THREAT_TYPE_FIELD),
threatMatch.getString(PLATFORM_TYPE_FIELD),
threatMatch.getString(METADATA_FIELD),
threatMatch.getString(DOMAIN_NAME_FIELD));
threatMatch.getString(THREAT_TYPE_FIELD), threatMatch.getString(DOMAIN_NAME_FIELD));
}
}
@@ -442,10 +442,8 @@ public class BigqueryConnection implements AutoCloseable {
* Returns the result of calling queryToLocalTable, but synchronously to avoid spawning new
* background threads, which App Engine doesn't support.
*
* @see <a href="https://cloud.google.com/appengine/docs/standard/java/runtime#Threads">
* App Engine Runtime</a>
*
* <p>Returns the results of the query in an ImmutableTable on success.
* @see <a href="https://cloud.google.com/appengine/docs/standard/java/runtime#Threads">App Engine
* Runtime</a>
*/
public ImmutableTable<Integer, TableFieldSchema, Object> queryToLocalTableSync(String querySql) {
Job job = new Job()
@@ -576,8 +574,6 @@ public class BigqueryConnection implements AutoCloseable {
/**
* Launch a job, wait for it to complete, but <i>do not</i> check for errors.
*
* @throws BigqueryJobFailureException
*/
public Job runJob(Job job, @Nullable AbstractInputStreamContent data) {
return checkJob(waitForJob(launchJob(job, data)));
@@ -84,7 +84,7 @@ public class DnsMessageTransport {
* @param query a message to send
* @return the response received from the server
* @throws IOException if the Socket input/output streams throws one
* @throws IllegalArgumentException if the query is too large to be sent (> 65535 bytes)
* @throws IllegalArgumentException if the query is too large to be sent (&gt; 65535 bytes)
*/
public Message send(Message query) throws IOException {
try (Socket socket = factory.createSocket(InetAddress.getByName(updateHost), DNS_PORT)) {
@@ -73,8 +73,10 @@ public class BigqueryPollJobAction implements Runnable {
@Override
public void run() {
checkJobOutcome(); // Throws a NotModifiedException if the job hasn't completed.
if (payload == null || payload.length == 0) {
boolean jobOutcome =
checkJobOutcome(); // Throws a NotModifiedException if the job hasn't completed.
// If the job failed, do not enqueue the next step.
if (!jobOutcome || payload == null || payload.length == 0) {
return;
}
// If there is a payload, it's a chained task, so enqueue it.
@@ -42,8 +42,8 @@ import javax.xml.stream.events.XMLEvent;
/**
* Sanitizes sensitive data in incoming/outgoing EPP XML messages.
*
* <p>Current implementation masks user credentials (text following <pw> and <newPW> tags) as
* follows:
* <p>Current implementation masks user credentials (text following &lt;pw&gt; and &lt;newPW&gt;
* tags) as follows:
*
* <ul>
* <li>A control character (in ranges [0 - 1F] and [7F - 9F]) is replaced with 'C'.
@@ -94,10 +94,10 @@ public final class ResourceFlowUtils {
* trust the query and need to do the full mapreduce.
*/
Iterable<Key<DomainBase>> keys =
queryForLinkedDomains(fki.getResourceKey(), now)
queryForLinkedDomains(fki.getResourceKey().getOfyKey(), now)
.limit(FAILFAST_CHECK_COUNT)
.keys();
VKey<R> resourceVKey = VKey.createOfy(resourceClass, fki.getResourceKey());
VKey<R> resourceVKey = fki.getResourceKey();
Predicate<DomainBase> predicate =
domain -> getPotentialReferences.apply(domain).contains(resourceVKey);
return ofy().load().keys(keys).values().stream().anyMatch(predicate)
@@ -136,9 +136,9 @@ public final class ResourceFlowUtils {
public static <R extends EppResource> void verifyResourceDoesNotExist(
Class<R> clazz, String targetId, DateTime now, String clientId) throws EppException {
Key<R> key = loadAndGetKey(clazz, targetId, now);
VKey<R> key = loadAndGetKey(clazz, targetId, now);
if (key != null) {
R resource = ofy().load().key(key).now();
R resource = tm().load(key);
// These are similar exceptions, but we can track them internally as log-based metrics.
if (Objects.equals(clientId, resource.getPersistedCurrentSponsorClientId())) {
throw new ResourceAlreadyExistsForThisClientException(targetId);
@@ -97,7 +97,7 @@ public final class ContactTransferApproveFlow implements TransactionalFlow {
ofy().save().<Object>entities(newContact, historyEntry, gainingPollMessage);
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
ofy().delete().keys(existingContact.getTransferData().getServerApproveEntities());
tm().delete(existingContact.getTransferData().getServerApproveEntities());
return responseBuilder
.setResData(createTransferResponse(targetId, newContact.getTransferData()))
.build();
@@ -93,7 +93,7 @@ public final class ContactTransferCancelFlow implements TransactionalFlow {
ofy().save().<Object>entities(newContact, historyEntry, losingPollMessage);
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
ofy().delete().keys(existingContact.getTransferData().getServerApproveEntities());
tm().delete(existingContact.getTransferData().getServerApproveEntities());
return responseBuilder
.setResData(createTransferResponse(targetId, newContact.getTransferData()))
.build();
@@ -90,7 +90,7 @@ public final class ContactTransferRejectFlow implements TransactionalFlow {
ofy().save().<Object>entities(newContact, historyEntry, gainingPollMessage);
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
ofy().delete().keys(existingContact.getTransferData().getServerApproveEntities());
tm().delete(existingContact.getTransferData().getServerApproveEntities());
return responseBuilder
.setResData(createTransferResponse(targetId, newContact.getTransferData()))
.build();
@@ -126,13 +126,15 @@ public final class ContactTransferRequestFlow implements TransactionalFlow {
// If the transfer is server approved, this message will be sent to the gaining registrar. */
PollMessage serverApproveGainingPollMessage =
createGainingTransferPollMessage(targetId, serverApproveTransferData, historyEntry);
TransferData pendingTransferData = serverApproveTransferData.asBuilder()
.setTransferStatus(TransferStatus.PENDING)
.setServerApproveEntities(
ImmutableSet.of(
Key.create(serverApproveGainingPollMessage),
Key.create(serverApproveLosingPollMessage)))
.build();
TransferData pendingTransferData =
serverApproveTransferData
.asBuilder()
.setTransferStatus(TransferStatus.PENDING)
.setServerApproveEntities(
ImmutableSet.of(
serverApproveGainingPollMessage.createVKey(),
serverApproveLosingPollMessage.createVKey()))
.build();
// When a transfer is requested, a poll message is created to notify the losing registrar.
PollMessage requestPollMessage =
createLosingTransferPollMessage(targetId, pendingTransferData, historyEntry).asBuilder()
@@ -75,7 +75,7 @@ public class DomainCheckFlowCustomLogic extends BaseFlowCustomLogic {
/**
* The time to perform the domain check as of. This defaults to the current time, but can be
* overridden in v>=0.12 of the fee extension.
* overridden in v&gt;=0.12 of the fee extension.
*/
public abstract DateTime asOfDate();
@@ -105,7 +105,7 @@ public class DomainCheckFlowCustomLogic extends BaseFlowCustomLogic {
/**
* The time to perform the domain check as of. This defaults to the current time, but can be
* overridden in v>=0.12 of the fee extension.
* overridden in v&gt;=0.12 of the fee extension.
*/
public abstract DateTime asOfDate();
@@ -77,7 +77,6 @@ import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainCommand;
import google.registry.model.domain.DomainCommand.Create;
@@ -353,14 +352,12 @@ public class DomainCreateFlow implements TransactionalFlow {
.setLaunchNotice(hasClaimsNotice ? launchCreate.get().getNotice() : null)
.setSmdId(signedMarkId)
.setDsData(secDnsCreate.isPresent() ? secDnsCreate.get().getDsData() : null)
.setRegistrant(VKey.createOfy(ContactResource.class, command.getRegistrant()))
.setRegistrant(command.getRegistrant())
.setAuthInfo(command.getAuthInfo())
.setFullyQualifiedDomainName(targetId)
.setNameservers(
(ImmutableSet<VKey<HostResource>>)
command.getNameservers().stream()
.map(key -> VKey.createOfy(HostResource.class, key))
.collect(toImmutableSet()))
command.getNameservers().stream().collect(toImmutableSet()))
.setStatusValues(statuses.build())
.setContacts(command.getContacts())
.addGracePeriod(GracePeriod.forBillingEvent(GracePeriodStatus.ADD, createBillingEvent))
@@ -534,7 +531,7 @@ public class DomainCreateFlow implements TransactionalFlow {
.setPeriodYears(years)
.setCost(feesAndCredits.getCreateCost())
.setEventTime(now)
.setAllocationToken(allocationToken.map(Key::create).orElse(null))
.setAllocationToken(allocationToken.map(AllocationToken::createVKey).orElse(null))
.setBillingTime(
now.plus(
isAnchorTenant
@@ -307,14 +307,11 @@ public class DomainFlowUtils {
/** Verify that no linked resources have disallowed statuses. */
static void verifyNotInPendingDelete(
Set<DesignatedContact> contacts,
Key<ContactResource> registrant,
Set<Key<HostResource>> nameservers)
VKey<ContactResource> registrant,
Set<VKey<HostResource>> nameservers)
throws EppException {
ImmutableList.Builder<Key<? extends EppResource>> keysToLoad = new ImmutableList.Builder<>();
contacts.stream()
.map(DesignatedContact::getContactKey)
.map(VKey::getOfyKey)
.forEach(keysToLoad::add);
ImmutableList.Builder<VKey<? extends EppResource>> keysToLoad = new ImmutableList.Builder<>();
contacts.stream().map(DesignatedContact::getContactKey).forEach(keysToLoad::add);
Optional.ofNullable(registrant).ifPresent(keysToLoad::add);
keysToLoad.addAll(nameservers);
verifyNotInPendingDelete(EppResource.loadCached(keysToLoad.build()).values());
@@ -381,7 +378,7 @@ public class DomainFlowUtils {
}
static void validateRequiredContactsPresent(
@Nullable Key<ContactResource> registrant, Set<DesignatedContact> contacts)
@Nullable VKey<ContactResource> registrant, Set<DesignatedContact> contacts)
throws RequiredParameterMissingException {
if (registrant == null) {
throw new MissingRegistrantException();
@@ -214,7 +214,7 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
ofy().save().entities(entitiesToSave.build());
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
ofy().delete().keys(existingDomain.getTransferData().getServerApproveEntities());
tm().delete(existingDomain.getTransferData().getServerApproveEntities());
return responseBuilder
.setResData(createTransferResponse(
targetId, newDomain.getTransferData(), newDomain.getRegistrationExpirationTime()))
@@ -110,7 +110,7 @@ public final class DomainTransferCancelFlow implements TransactionalFlow {
updateAutorenewRecurrenceEndTime(existingDomain, END_OF_TIME);
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
ofy().delete().keys(existingDomain.getTransferData().getServerApproveEntities());
tm().delete(existingDomain.getTransferData().getServerApproveEntities());
return responseBuilder
.setResData(createTransferResponse(targetId, newDomain.getTransferData(), null))
.build();
@@ -112,7 +112,7 @@ public final class DomainTransferRejectFlow implements TransactionalFlow {
updateAutorenewRecurrenceEndTime(existingDomain, END_OF_TIME);
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
ofy().delete().keys(existingDomain.getTransferData().getServerApproveEntities());
tm().delete(existingDomain.getTransferData().getServerApproveEntities());
return responseBuilder
.setResData(createTransferResponse(targetId, newDomain.getTransferData(), null))
.build();
@@ -20,7 +20,6 @@ import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
@@ -37,6 +36,7 @@ import google.registry.model.transfer.TransferData;
import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
import google.registry.model.transfer.TransferResponse.DomainTransferResponse;
import google.registry.model.transfer.TransferStatus;
import google.registry.persistence.VKey;
import java.util.Optional;
import javax.annotation.Nullable;
import org.joda.money.Money;
@@ -52,37 +52,34 @@ public final class DomainTransferUtils {
TransferData.Builder transferDataBuilder,
ImmutableSet<TransferServerApproveEntity> serverApproveEntities,
Period transferPeriod) {
ImmutableSet.Builder<Key<? extends TransferServerApproveEntity>> serverApproveEntityKeys =
ImmutableSet.Builder<VKey<? extends TransferServerApproveEntity>> serverApproveEntityKeys =
new ImmutableSet.Builder<>();
for (TransferServerApproveEntity entity : serverApproveEntities) {
serverApproveEntityKeys.add(Key.create(entity));
serverApproveEntityKeys.add(entity.createVKey());
}
if (transferPeriod.getValue() != 0) {
// Unless superuser sets period to 0, add a transfer billing event.
transferDataBuilder.setServerApproveBillingEvent(
Key.create(
serverApproveEntities
.stream()
.filter(BillingEvent.OneTime.class::isInstance)
.map(BillingEvent.OneTime.class::cast)
.collect(onlyElement())));
serverApproveEntities.stream()
.filter(BillingEvent.OneTime.class::isInstance)
.map(BillingEvent.OneTime.class::cast)
.collect(onlyElement())
.createVKey());
}
return transferDataBuilder
.setTransferStatus(TransferStatus.PENDING)
.setServerApproveAutorenewEvent(
Key.create(
serverApproveEntities
.stream()
.filter(BillingEvent.Recurring.class::isInstance)
.map(BillingEvent.Recurring.class::cast)
.collect(onlyElement())))
serverApproveEntities.stream()
.filter(BillingEvent.Recurring.class::isInstance)
.map(BillingEvent.Recurring.class::cast)
.collect(onlyElement())
.createVKey())
.setServerApproveAutorenewPollMessage(
Key.create(
serverApproveEntities
.stream()
.filter(PollMessage.Autorenew.class::isInstance)
.map(PollMessage.Autorenew.class::cast)
.collect(onlyElement())))
serverApproveEntities.stream()
.filter(PollMessage.Autorenew.class::isInstance)
.map(PollMessage.Autorenew.class::cast)
.collect(onlyElement())
.createVKey())
.setServerApproveEntities(serverApproveEntityKeys.build())
.setTransferPeriod(transferPeriod)
.build();
@@ -60,7 +60,6 @@ import google.registry.flows.domain.DomainFlowUtils.MissingRegistrantException;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainCommand.Update;
import google.registry.model.domain.DomainCommand.Update.AddRemove;
@@ -73,11 +72,9 @@ import google.registry.model.eppcommon.StatusValue;
import google.registry.model.eppinput.EppInput;
import google.registry.model.eppinput.ResourceCommand;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.host.HostResource;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.persistence.VKey;
import java.util.Optional;
import javax.inject.Inject;
import org.joda.time.DateTime;
@@ -247,22 +244,11 @@ public final class DomainUpdateFlow implements TransactionalFlow {
.setLastEppUpdateClientId(clientId)
.addStatusValues(add.getStatusValues())
.removeStatusValues(remove.getStatusValues())
.addNameservers(
add.getNameservers().stream()
.map(key -> VKey.createOfy(HostResource.class, key))
.collect(toImmutableSet()))
.removeNameservers(
remove.getNameservers().stream()
.map(key -> VKey.createOfy(HostResource.class, key))
.collect(toImmutableSet()))
.addNameservers(add.getNameservers().stream().collect(toImmutableSet()))
.removeNameservers(remove.getNameservers().stream().collect(toImmutableSet()))
.addContacts(add.getContacts())
.removeContacts(remove.getContacts())
.setRegistrant(
firstNonNull(
change.getRegistrant() != null
? VKey.createOfy(ContactResource.class, change.getRegistrant())
: null,
domain.getRegistrant()))
.setRegistrant(firstNonNull(change.getRegistrant(), domain.getRegistrant()))
.setAuthInfo(firstNonNull(change.getAuthInfo(), domain.getAuthInfo()));
return domainBuilder.build();
}
@@ -275,7 +261,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
private void validateNewState(DomainBase newDomain) throws EppException {
validateNoDuplicateContacts(newDomain.getContacts());
validateRequiredContactsPresent(newDomain.getRegistrant().getOfyKey(), newDomain.getContacts());
validateRequiredContactsPresent(newDomain.getRegistrant(), newDomain.getContacts());
validateDsData(newDomain.getDsData());
validateNameserversCountForTld(
newDomain.getTld(),
@@ -128,7 +128,7 @@ public final class HostCreateFlow implements TransactionalFlow {
.setFullyQualifiedHostName(targetId)
.setInetAddresses(command.getInetAddresses())
.setRepoId(createRepoId(ObjectifyService.allocateId(), roidSuffix))
.setSuperordinateDomain(superordinateDomain.map(Key::create).orElse(null))
.setSuperordinateDomain(superordinateDomain.map(DomainBase::createVKey).orElse(null))
.build();
historyBuilder
.setType(HistoryEntry.Type.HOST_CREATE)
@@ -96,8 +96,7 @@ public final class HostDeleteFlow implements TransactionalFlow {
// the client id, needs to be read off of it.
EppResource owningResource =
existingHost.isSubordinate()
? ofy().load().key(existingHost.getSuperordinateDomain()).now()
.cloneProjectedAtTime(now)
? tm().load(existingHost.getSuperordinateDomain()).cloneProjectedAtTime(now)
: existingHost;
verifyResourceOwnership(clientId, owningResource);
}
@@ -18,7 +18,7 @@ import static google.registry.flows.FlowUtils.validateClientIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
import static google.registry.flows.host.HostFlowUtils.validateHostName;
import static google.registry.model.EppResourceUtils.isLinked;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
@@ -77,7 +77,7 @@ public final class HostInfoFlow implements Flow {
// there is no superordinate domain, the host's own values for these fields will be correct.
if (host.isSubordinate()) {
DomainBase superordinateDomain =
ofy().load().key(host.getSuperordinateDomain()).now().cloneProjectedAtTime(now);
tm().load(host.getSuperordinateDomain()).cloneProjectedAtTime(now);
hostInfoDataBuilder
.setCurrentSponsorClientId(superordinateDomain.getCurrentSponsorClientId())
.setLastTransferTime(host.computeLastTransferTime(superordinateDomain));
@@ -60,6 +60,7 @@ import google.registry.model.host.HostResource;
import google.registry.model.index.ForeignKeyIndex;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.persistence.VKey;
import java.util.Objects;
import java.util.Optional;
import javax.inject.Inject;
@@ -136,9 +137,10 @@ public final class HostUpdateFlow implements TransactionalFlow {
boolean isHostRename = suppliedNewHostName != null;
String oldHostName = targetId;
String newHostName = firstNonNull(suppliedNewHostName, oldHostName);
DomainBase oldSuperordinateDomain = existingHost.isSubordinate()
? ofy().load().key(existingHost.getSuperordinateDomain()).now().cloneProjectedAtTime(now)
: null;
DomainBase oldSuperordinateDomain =
existingHost.isSubordinate()
? tm().load(existingHost.getSuperordinateDomain()).cloneProjectedAtTime(now)
: null;
// Note that lookupSuperordinateDomain calls cloneProjectedAtTime on the domain for us.
Optional<DomainBase> newSuperordinateDomain =
lookupSuperordinateDomain(validateHostName(newHostName), now);
@@ -153,8 +155,8 @@ public final class HostUpdateFlow implements TransactionalFlow {
AddRemove remove = command.getInnerRemove();
checkSameValuesNotAddedAndRemoved(add.getStatusValues(), remove.getStatusValues());
checkSameValuesNotAddedAndRemoved(add.getInetAddresses(), remove.getInetAddresses());
Key<DomainBase> newSuperordinateDomainKey =
newSuperordinateDomain.map(Key::create).orElse(null);
VKey<DomainBase> newSuperordinateDomainKey =
newSuperordinateDomain.map(DomainBase::createVKey).orElse(null);
// If the superordinateDomain field is changing, set the lastSuperordinateChange to now.
DateTime lastSuperordinateChange =
Objects.equals(newSuperordinateDomainKey, existingHost.getSuperordinateDomain())
@@ -280,28 +282,28 @@ public final class HostUpdateFlow implements TransactionalFlow {
if (existingHost.isSubordinate()
&& newHost.isSubordinate()
&& Objects.equals(
existingHost.getSuperordinateDomain(),
newHost.getSuperordinateDomain())) {
ofy().save().entity(
ofy().load().key(existingHost.getSuperordinateDomain()).now().asBuilder()
.removeSubordinateHost(existingHost.getFullyQualifiedHostName())
.addSubordinateHost(newHost.getFullyQualifiedHostName())
.build());
existingHost.getSuperordinateDomain(), newHost.getSuperordinateDomain())) {
tm().saveNewOrUpdate(
tm().load(existingHost.getSuperordinateDomain())
.asBuilder()
.removeSubordinateHost(existingHost.getFullyQualifiedHostName())
.addSubordinateHost(newHost.getFullyQualifiedHostName())
.build());
return;
}
if (existingHost.isSubordinate()) {
ofy().save().entity(
ofy().load().key(existingHost.getSuperordinateDomain()).now()
.asBuilder()
.removeSubordinateHost(existingHost.getFullyQualifiedHostName())
.build());
tm().saveNewOrUpdate(
tm().load(existingHost.getSuperordinateDomain())
.asBuilder()
.removeSubordinateHost(existingHost.getFullyQualifiedHostName())
.build());
}
if (newHost.isSubordinate()) {
ofy().save().entity(
ofy().load().key(newHost.getSuperordinateDomain()).now()
.asBuilder()
.addSubordinateHost(newHost.getFullyQualifiedHostName())
.build());
tm().saveNewOrUpdate(
tm().load(newHost.getSuperordinateDomain())
.asBuilder()
.addSubordinateHost(newHost.getFullyQualifiedHostName())
.build());
}
}
@@ -16,12 +16,11 @@ package google.registry.model;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.ImmutableMap.toImmutableMap;
import static com.google.common.collect.Sets.difference;
import static com.google.common.collect.Sets.union;
import static google.registry.config.RegistryConfig.getEppResourceCachingDuration;
import static google.registry.config.RegistryConfig.getEppResourceMaxCachedEntries;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.nullToEmpty;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableCopy;
@@ -32,11 +31,9 @@ import com.google.common.annotations.VisibleForTesting;
import com.google.common.cache.CacheBuilder;
import com.google.common.cache.CacheLoader;
import com.google.common.cache.LoadingCache;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedMap;
import com.google.common.collect.Streams;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.Id;
import com.googlecode.objectify.annotation.Index;
@@ -44,12 +41,13 @@ import google.registry.config.RegistryConfig;
import google.registry.model.eppcommon.StatusValue;
import google.registry.model.ofy.CommitLogManifest;
import google.registry.model.transfer.TransferData;
import google.registry.persistence.VKey;
import google.registry.util.NonFinalForTesting;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Optional;
import java.util.Set;
import java.util.concurrent.ExecutionException;
import java.util.stream.StreamSupport;
import javax.persistence.Column;
import javax.persistence.MappedSuperclass;
import javax.persistence.Transient;
@@ -70,11 +68,11 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
/** The ID of the registrar that is currently sponsoring this resource. */
@Index
@Column(nullable = false)
@Column(name = "currentSponsorRegistrarId", nullable = false)
String currentSponsorClientId;
/** The ID of the registrar that created this resource. */
@Column(nullable = false)
@Column(name = "creationRegistrarId", nullable = false)
String creationClientId;
/**
@@ -84,6 +82,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
* edits; it only includes EPP-visible modifications such as {@literal <update>}. Can be null if
* the resource has never been modified.
*/
@Column(name = "lastEppUpdateRegistrarId")
String lastEppUpdateClientId;
/** The time when this resource was created. */
@@ -183,6 +182,9 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
/** Get the foreign key string for this resource. */
public abstract String getForeignKey();
/** Create the VKey for the specified EPP resource. */
public abstract VKey<? extends EppResource> createVKey();
/** Override of {@link Buildable#asBuilder} so that the extra methods are visible. */
@Override
public abstract Builder<?, ?> asBuilder();
@@ -329,18 +331,18 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
}
}
static final CacheLoader<Key<? extends EppResource>, EppResource> CACHE_LOADER =
new CacheLoader<Key<? extends EppResource>, EppResource>() {
static final CacheLoader<VKey<? extends EppResource>, EppResource> CACHE_LOADER =
new CacheLoader<VKey<? extends EppResource>, EppResource>() {
@Override
public EppResource load(Key<? extends EppResource> key) {
return tm().doTransactionless(() -> ofy().load().key(key).now());
public EppResource load(VKey<? extends EppResource> key) {
return tm().doTransactionless(() -> tm().load(key));
}
@Override
public Map<Key<? extends EppResource>, EppResource> loadAll(
Iterable<? extends Key<? extends EppResource>> keys) {
return tm().doTransactionless(() -> loadMultiple(keys));
public Map<VKey<? extends EppResource>, EppResource> loadAll(
Iterable<? extends VKey<? extends EppResource>> keys) {
return tm().doTransactionless(() -> loadAsMap(keys));
}
};
@@ -352,10 +354,10 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
* directly should of course never use the cache.
*/
@NonFinalForTesting
private static LoadingCache<Key<? extends EppResource>, EppResource> cacheEppResources =
private static LoadingCache<VKey<? extends EppResource>, EppResource> cacheEppResources =
createEppResourcesCache(getEppResourceCachingDuration());
private static LoadingCache<Key<? extends EppResource>, EppResource> createEppResourcesCache(
private static LoadingCache<VKey<? extends EppResource>, EppResource> createEppResourcesCache(
Duration expiry) {
return CacheBuilder.newBuilder()
.expireAfterWrite(expiry.getMillis(), MILLISECONDS)
@@ -369,29 +371,16 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
cacheEppResources = createEppResourcesCache(effectiveExpiry);
}
private static ImmutableMap<Key<? extends EppResource>, EppResource> loadMultiple(
Iterable<? extends Key<? extends EppResource>> keys) {
// This cast is safe because, in Objectify, Key<? extends EppResource> can also be
// treated as a Key<EppResource>.
@SuppressWarnings("unchecked")
ImmutableList<Key<EppResource>> typedKeys =
Streams.stream(keys).map(key -> (Key<EppResource>) key).collect(toImmutableList());
// Typing shenanigans are required to return the map with the correct required generic type.
return ofy().load().keys(typedKeys).entrySet().stream()
.collect(ImmutableMap.toImmutableMap(Entry::getKey, Entry::getValue));
}
/**
* Loads the given EppResources by their keys using the cache (if enabled).
*
* <p>Don't use this unless you really need it for performance reasons, and be sure that you are
* OK with the trade-offs in loss of transactional consistency.
*/
public static ImmutableMap<Key<? extends EppResource>, EppResource> loadCached(
Iterable<Key<? extends EppResource>> keys) {
public static ImmutableMap<VKey<? extends EppResource>, EppResource> loadCached(
Iterable<VKey<? extends EppResource>> keys) {
if (!RegistryConfig.isEppResourceCachingEnabled()) {
return loadMultiple(keys);
return loadAsMap(keys);
}
try {
return cacheEppResources.getAll(keys);
@@ -406,9 +395,9 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
* <p>Don't use this unless you really need it for performance reasons, and be sure that you are
* OK with the trade-offs in loss of transactional consistency.
*/
public static <T extends EppResource> T loadCached(Key<T> key) {
public static <T extends EppResource> T loadCached(VKey<T> key) {
if (!RegistryConfig.isEppResourceCachingEnabled()) {
return ofy().load().key(key).now();
return tm().load(key);
}
try {
// Safe to cast because loading a Key<T> returns an entity of type T.
@@ -419,4 +408,17 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
throw new RuntimeException("Error loading cached EppResources", e.getCause());
}
}
private static ImmutableMap<VKey<? extends EppResource>, EppResource> loadAsMap(
Iterable<? extends VKey<? extends EppResource>> keys) {
return StreamSupport.stream(keys.spliterator(), false)
// It's possible for us to receive the same key more than once which causes
// the immutable map build to break with a duplicate key, so we have to ensure key
// uniqueness.
.distinct()
// We have to use "key -> key" here instead of the identity() function, because
// the latter breaks the fairly complicated generic type checking required by the
// caching interface.
.collect(toImmutableMap(key -> key, key -> tm().load(key)));
}
}
@@ -17,6 +17,7 @@ package google.registry.model;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.isAtOrAfter;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import static google.registry.util.DateTimeUtils.latestOf;
@@ -141,7 +142,7 @@ public final class EppResourceUtils {
T resource =
useCache
? EppResource.loadCached(fki.getResourceKey())
: ofy().load().key(fki.getResourceKey()).now();
: tm().maybeLoad(fki.getResourceKey()).orElse(null);
if (resource == null || isAtOrAfter(now, resource.getDeletionTime())) {
return Optional.empty();
}
@@ -34,6 +34,7 @@ import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.Id;
import com.googlecode.objectify.annotation.Ignore;
import com.googlecode.objectify.annotation.Parent;
import google.registry.persistence.VKey;
import java.lang.reflect.Array;
import java.lang.reflect.Field;
import java.lang.reflect.Modifier;
@@ -139,10 +140,13 @@ public class ModelUtils {
// If the field's type is the same as the field's class object, then it's a non-parameterized
// type, and thus we just add it directly. We also don't bother looking at the parameterized
// types of Key objects, since they are just references to other objects and don't actually
// embed themselves in the persisted object anyway.
// types of Key and VKey objects, since they are just references to other objects and don't
// actually embed themselves in the persisted object anyway.
Class<?> fieldClazz = field.getType();
Type fieldType = field.getGenericType();
if (VKey.class.equals(fieldClazz)) {
continue;
}
builder.add(fieldClazz);
if (fieldType.equals(fieldClazz) || Key.class.equals(clazz)) {
continue;
@@ -50,7 +50,8 @@ import java.util.stream.Collectors;
public class OteStats {
/**
* Returns the statistics about the OT&E actions that have been taken by a particular registrar.
* Returns the statistics about the OT&amp;E actions that have been taken by a particular
* registrar.
*/
public static OteStats getFromRegistrar(String registrarName) {
return new OteStats().recordRegistrarHistory(registrarName);
@@ -18,6 +18,7 @@ import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
@@ -114,7 +115,7 @@ public final class ResourceTransferUtils {
R resource, R newResource, DateTime now, HistoryEntry historyEntry) {
if (resource.getStatusValues().contains(StatusValue.PENDING_TRANSFER)) {
TransferData oldTransferData = resource.getTransferData();
ofy().delete().keys(oldTransferData.getServerApproveEntities());
tm().delete(oldTransferData.getServerApproveEntities());
ofy()
.save()
.entity(
@@ -30,7 +30,7 @@ public class UpdateAutoTimestamp extends ImmutableObject {
DateTime timestamp;
/** Returns the timestamp, or {@link START_OF_TIME} if it's null. */
/** Returns the timestamp, or {@code START_OF_TIME} if it's null. */
public DateTime getTimestamp() {
return Optional.ofNullable(timestamp).orElse(START_OF_TIME);
}
@@ -30,6 +30,7 @@ import com.google.common.collect.Sets;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Id;
import com.googlecode.objectify.annotation.Ignore;
import com.googlecode.objectify.annotation.IgnoreSave;
import com.googlecode.objectify.annotation.Index;
import com.googlecode.objectify.annotation.Parent;
@@ -43,14 +44,28 @@ import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
import google.registry.persistence.VKey;
import google.registry.persistence.WithLongVKey;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import javax.annotation.Nullable;
import javax.persistence.AttributeOverride;
import javax.persistence.AttributeOverrides;
import javax.persistence.Column;
import javax.persistence.Embedded;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.MappedSuperclass;
import javax.persistence.Transient;
import org.joda.money.Money;
import org.joda.time.DateTime;
/** A billable event in a domain's lifecycle. */
@MappedSuperclass
@WithLongVKey
public abstract class BillingEvent extends ImmutableObject
implements Buildable, TransferServerApproveEntity {
@@ -93,24 +108,41 @@ public abstract class BillingEvent extends ImmutableObject
/** Entity id. */
@Id
long id;
@javax.persistence.Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
Long id;
@Parent
@DoNotHydrate
Key<HistoryEntry> parent;
@Parent @DoNotHydrate @Transient Key<HistoryEntry> parent;
/** The registrar to bill. */
@Index
@Column(name = "registrarId", nullable = false)
String clientId;
/** Revision id of the entry in DomainHistory table that ths bill belongs to. */
// TODO(shicong): Add foreign key constraint when DomainHistory table is generated
@Ignore
@Column(nullable = false)
Long domainHistoryRevisionId;
/** ID of the EPP resource that the bill is for. */
// TODO(shicong): Add foreign key constraint when we expand DatastoreHelp for Postgresql
@Ignore
@Column(nullable = false)
String domainRepoId;
/** When this event was created. For recurring events, this is also the recurrence start time. */
@Index
@Column(nullable = false)
DateTime eventTime;
/** The reason for the bill. */
@Enumerated(EnumType.STRING)
@Column(nullable = false)
Reason reason;
/** The fully qualified domain name of the domain that the bill is for. */
@Column(name = "domain_name", nullable = false)
String targetId;
@Nullable
@@ -120,6 +152,14 @@ public abstract class BillingEvent extends ImmutableObject
return clientId;
}
public long getDomainHistoryRevisionId() {
return domainHistoryRevisionId;
}
public String getDomainRepoId() {
return domainRepoId;
}
public DateTime getEventTime() {
return eventTime;
}
@@ -163,7 +203,7 @@ public abstract class BillingEvent extends ImmutableObject
return thisCastToDerived();
}
public B setId(Long id) {
public B setId(long id) {
getInstance().id = id;
return thisCastToDerived();
}
@@ -173,6 +213,16 @@ public abstract class BillingEvent extends ImmutableObject
return thisCastToDerived();
}
public B setDomainHistoryRevisionId(long domainHistoryRevisionId) {
getInstance().domainHistoryRevisionId = domainHistoryRevisionId;
return thisCastToDerived();
}
public B setDomainRepoId(String domainRepoId) {
getInstance().domainRepoId = domainRepoId;
return thisCastToDerived();
}
public B setEventTime(DateTime eventTime) {
getInstance().eventTime = eventTime;
return thisCastToDerived();
@@ -194,6 +244,7 @@ public abstract class BillingEvent extends ImmutableObject
}
public B setParent(Key<HistoryEntry> parentKey) {
// TODO(shicong): Figure out how to set domainHistoryRevisionId and domainRepoId
getInstance().parent = parentKey;
return thisCastToDerived();
}
@@ -213,9 +264,23 @@ public abstract class BillingEvent extends ImmutableObject
/** A one-time billable event. */
@ReportedOn
@Entity
@javax.persistence.Entity(name = "BillingEvent")
@javax.persistence.Table(
indexes = {
@javax.persistence.Index(columnList = "registrarId"),
@javax.persistence.Index(columnList = "eventTime"),
@javax.persistence.Index(columnList = "billingTime"),
@javax.persistence.Index(columnList = "syntheticCreationTime"),
@javax.persistence.Index(columnList = "allocation_token_id")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_event_id"))
public static class OneTime extends BillingEvent {
/** The billable value. */
@AttributeOverrides({
@AttributeOverride(name = "money.amount", column = @Column(name = "cost_amount")),
@AttributeOverride(name = "money.currency", column = @Column(name = "cost_currency"))
})
Money cost;
/** When the cost should be billed. */
@@ -223,8 +288,8 @@ public abstract class BillingEvent extends ImmutableObject
DateTime billingTime;
/**
* The period in years of the action being billed for, if applicable, otherwise null.
* Used for financial reporting.
* The period in years of the action being billed for, if applicable, otherwise null. Used for
* financial reporting.
*/
@IgnoreSave(IfNull.class)
Integer periodYears = null;
@@ -240,15 +305,21 @@ public abstract class BillingEvent extends ImmutableObject
/**
* For {@link Flag#SYNTHETIC} events, a {@link Key} to the {@link BillingEvent} from which this
* OneTime was created. This is needed in order to properly match billing events against
* {@link Cancellation}s.
* OneTime was created. This is needed in order to properly match billing events against {@link
* Cancellation}s.
*/
Key<? extends BillingEvent> cancellationMatchingBillingEvent;
@Column(name = "cancellation_matching_billing_recurrence_id")
VKey<? extends BillingEvent> cancellationMatchingBillingEvent;
/**
* The {@link AllocationToken} used in the creation of this event, or null if one was not used.
*
* <p>TODO(shicong): Add foreign key constraint when AllocationToken schema is generated
*/
@Index @Nullable Key<AllocationToken> allocationToken;
@Column(name = "allocation_token_id")
@Index
@Nullable
VKey<AllocationToken> allocationToken;
public Money getCost() {
return cost;
@@ -266,14 +337,19 @@ public abstract class BillingEvent extends ImmutableObject
return syntheticCreationTime;
}
public Key<? extends BillingEvent> getCancellationMatchingBillingEvent() {
public VKey<? extends BillingEvent> getCancellationMatchingBillingEvent() {
return cancellationMatchingBillingEvent;
}
public Optional<Key<AllocationToken>> getAllocationToken() {
public Optional<VKey<AllocationToken>> getAllocationToken() {
return Optional.ofNullable(allocationToken);
}
@Override
public VKey<OneTime> createVKey() {
return VKey.createOfy(getClass(), Key.create(this));
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
@@ -311,12 +387,12 @@ public abstract class BillingEvent extends ImmutableObject
}
public Builder setCancellationMatchingBillingEvent(
Key<? extends BillingEvent> cancellationMatchingBillingEvent) {
VKey<? extends BillingEvent> cancellationMatchingBillingEvent) {
getInstance().cancellationMatchingBillingEvent = cancellationMatchingBillingEvent;
return this;
}
public Builder setAllocationToken(@Nullable Key<AllocationToken> allocationToken) {
public Builder setAllocationToken(@Nullable VKey<AllocationToken> allocationToken) {
getInstance().allocationToken = allocationToken;
return this;
}
@@ -361,6 +437,15 @@ public abstract class BillingEvent extends ImmutableObject
*/
@ReportedOn
@Entity
@javax.persistence.Entity(name = "BillingRecurrence")
@javax.persistence.Table(
indexes = {
@javax.persistence.Index(columnList = "registrarId"),
@javax.persistence.Index(columnList = "eventTime"),
@javax.persistence.Index(columnList = "recurrenceEndTime"),
@javax.persistence.Index(columnList = "recurrence_time_of_year")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_recurrence_id"))
public static class Recurring extends BillingEvent {
/**
@@ -384,6 +469,10 @@ public abstract class BillingEvent extends ImmutableObject
* model, whereas the billing time is a fixed {@link org.joda.time.Duration} later.
*/
@Index
@Embedded
@AttributeOverrides({
@AttributeOverride(name = "timeString", column = @Column(name = "recurrence_time_of_year"))
})
TimeOfYear recurrenceTimeOfYear;
public DateTime getRecurrenceEndTime() {
@@ -394,6 +483,11 @@ public abstract class BillingEvent extends ImmutableObject
return recurrenceTimeOfYear;
}
@Override
public VKey<Recurring> createVKey() {
return VKey.createOfy(getClass(), Key.create(this));
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
@@ -434,6 +528,14 @@ public abstract class BillingEvent extends ImmutableObject
*/
@ReportedOn
@Entity
@javax.persistence.Entity(name = "BillingCancellation")
@javax.persistence.Table(
indexes = {
@javax.persistence.Index(columnList = "registrarId"),
@javax.persistence.Index(columnList = "eventTime"),
@javax.persistence.Index(columnList = "billingTime")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_cancellation_id"))
public static class Cancellation extends BillingEvent {
/** The billing time of the charge that is being cancelled. */
@@ -446,7 +548,8 @@ public abstract class BillingEvent extends ImmutableObject
* <p>Although the type is {@link Key} the name "ref" is preserved for historical reasons.
*/
@IgnoreSave(IfNull.class)
Key<BillingEvent.OneTime> refOneTime = null;
@Column(name = "billing_event_id")
VKey<BillingEvent.OneTime> refOneTime = null;
/**
* The recurring billing event to cancel, or null for non-autorenew cancellations.
@@ -454,13 +557,14 @@ public abstract class BillingEvent extends ImmutableObject
* <p>Although the type is {@link Key} the name "ref" is preserved for historical reasons.
*/
@IgnoreSave(IfNull.class)
Key<BillingEvent.Recurring> refRecurring = null;
@Column(name = "billing_recurrence_id")
VKey<BillingEvent.Recurring> refRecurring = null;
public DateTime getBillingTime() {
return billingTime;
}
public Key<? extends BillingEvent> getEventKey() {
public VKey<? extends BillingEvent> getEventKey() {
return firstNonNull(refOneTime, refRecurring);
}
@@ -492,13 +596,20 @@ public abstract class BillingEvent extends ImmutableObject
.setParent(historyEntry);
// Set the grace period's billing event using the appropriate Cancellation builder method.
if (gracePeriod.getOneTimeBillingEvent() != null) {
builder.setOneTimeEventKey(gracePeriod.getOneTimeBillingEvent());
builder.setOneTimeEventKey(
VKey.createOfy(BillingEvent.OneTime.class, gracePeriod.getOneTimeBillingEvent()));
} else if (gracePeriod.getRecurringBillingEvent() != null) {
builder.setRecurringEventKey(gracePeriod.getRecurringBillingEvent());
builder.setRecurringEventKey(
VKey.createOfy(BillingEvent.Recurring.class, gracePeriod.getRecurringBillingEvent()));
}
return builder.build();
}
@Override
public VKey<Cancellation> createVKey() {
return VKey.createOfy(getClass(), Key.create(this));
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
@@ -518,12 +629,12 @@ public abstract class BillingEvent extends ImmutableObject
return this;
}
public Builder setOneTimeEventKey(Key<BillingEvent.OneTime> eventKey) {
public Builder setOneTimeEventKey(VKey<BillingEvent.OneTime> eventKey) {
getInstance().refOneTime = eventKey;
return this;
}
public Builder setRecurringEventKey(Key<BillingEvent.Recurring> eventKey) {
public Builder setRecurringEventKey(VKey<BillingEvent.Recurring> eventKey) {
getInstance().refRecurring = eventKey;
return this;
}
@@ -540,9 +651,7 @@ public abstract class BillingEvent extends ImmutableObject
}
}
/**
* An event representing a modification of an existing one-time billing event.
*/
/** An event representing a modification of an existing one-time billing event. */
@ReportedOn
@Entity
public static class Modification extends BillingEvent {
@@ -576,6 +685,11 @@ public abstract class BillingEvent extends ImmutableObject
return new Builder(clone(this));
}
@Override
public VKey<Modification> createVKey() {
return VKey.createOfy(this.getClass(), Key.create(this));
}
/**
* Create a new Modification billing event which is a refund of the given OneTime billing event
* and that is parented off the given HistoryEntry.
@@ -29,20 +29,22 @@ import com.googlecode.objectify.annotation.Embed;
import com.googlecode.objectify.annotation.Index;
import google.registry.model.ImmutableObject;
import java.util.List;
import javax.persistence.Embeddable;
import org.joda.time.DateTime;
/**
* A time of year (month, day, millis of day) that can be stored in a sort-friendly format.
*
* <p>This is conceptually similar to {@code MonthDay} in Joda or more generally to Joda's
* {@code Partial}, but the parts we need are too simple to justify a full implementation of
* {@code Partial}.
* <p>This is conceptually similar to {@code MonthDay} in Joda or more generally to Joda's {@code
* Partial}, but the parts we need are too simple to justify a full implementation of {@code
* Partial}.
*
* <p>For simplicity, the native representation of this class's data is its stored format. This
* allows it to be embeddable with no translation needed and also delays parsing of the string on
* load until it's actually needed.
*/
@Embed
@Embeddable
public class TimeOfYear extends ImmutableObject {
/**
@@ -37,7 +37,7 @@ import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
/** A collection of {@link ContactResource} commands. */
public class ContactCommand {
/** The fields on "chgType" from {@link "http://tools.ietf.org/html/rfc5733"}. */
/** The fields on "chgType" from <a href="http://tools.ietf.org/html/rfc5733">RFC5733</a>. */
@XmlTransient
public static class ContactCreateOrChange extends ImmutableObject
implements ResourceCreateOrChange<ContactResource.Builder> {
@@ -111,8 +111,8 @@ public class ContactCommand {
}
/**
* A create command for a {@link ContactResource}, mapping "createType" from
* {@link "http://tools.ietf.org/html/rfc5733"}.
* A create command for a {@link ContactResource}, mapping "createType" from <a
* href="http://tools.ietf.org/html/rfc5733">RFC5733</a>}.
*/
@XmlType(propOrder = {"contactId", "postalInfo", "voice", "fax", "email", "authInfo", "disclose"})
@XmlRootElement
@@ -41,7 +41,6 @@ import javax.persistence.AttributeOverride;
import javax.persistence.AttributeOverrides;
import javax.persistence.Column;
import javax.persistence.Embedded;
import javax.persistence.Transient;
import javax.xml.bind.annotation.XmlElement;
import org.joda.time.DateTime;
@@ -57,7 +56,7 @@ import org.joda.time.DateTime;
name = "Contact",
indexes = {
@javax.persistence.Index(columnList = "creationTime"),
@javax.persistence.Index(columnList = "currentSponsorClientId"),
@javax.persistence.Index(columnList = "currentSponsorRegistrarId"),
@javax.persistence.Index(columnList = "deletionTime"),
@javax.persistence.Index(columnList = "contactId", unique = true),
@javax.persistence.Index(columnList = "searchName")
@@ -171,8 +170,7 @@ public class ContactResource extends EppResource
ContactAuthInfo authInfo;
/** Data about any pending or past transfers on this contact. */
// TODO(b/153363295): Figure out how to persist transfer data
@Transient TransferData transferData;
TransferData transferData;
/**
* The time that this resource was last transferred.
@@ -197,7 +195,9 @@ public class ContactResource extends EppResource
})
Disclose disclose;
@Override
public VKey<ContactResource> createVKey() {
// TODO(mmuller): create symmetric keys if we can ever reload both sides.
return VKey.createOfy(ContactResource.class, Key.create(this));
}
@@ -27,7 +27,7 @@ import javax.persistence.Embedded;
import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.bind.annotation.XmlType;
/** The "discloseType" from {@link "http://tools.ietf.org/html/rfc5733"}. */
/** The "discloseType" from <a href="http://tools.ietf.org/html/rfc5733">RFC5733</a>. */
@Embed
@Embeddable
@XmlType(propOrder = {"name", "org", "addr", "voice", "fax", "email"})
@@ -76,7 +76,7 @@ public class Disclose extends ImmutableObject {
return flag;
}
/** The "intLocType" from {@link "http://tools.ietf.org/html/rfc5733"}. */
/** The "intLocType" from <a href="http://tools.ietf.org/html/rfc5733">RFC5733</a>. */
@Embed
public static class PostalInfoChoice extends ImmutableObject {
@XmlAttribute
@@ -32,8 +32,8 @@ import javax.xml.bind.annotation.adapters.NormalizedStringAdapter;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
/**
* Implementation of both "postalInfoType" and "chgPostalInfoType" from {@link
* "http://tools.ietf.org/html/rfc5733"}.
* Implementation of both "postalInfoType" and "chgPostalInfoType" from <a href=
* "http://tools.ietf.org/html/rfc5733">RFC5733</a>.
*/
@Embed
@Embeddable
@@ -66,6 +66,7 @@ import google.registry.model.registry.Registry;
import google.registry.model.transfer.TransferData;
import google.registry.model.transfer.TransferStatus;
import google.registry.persistence.VKey;
import google.registry.persistence.WithStringVKey;
import google.registry.schema.replay.DatastoreAndSqlEntity;
import google.registry.util.CollectionUtils;
import java.util.HashSet;
@@ -101,11 +102,12 @@ import org.joda.time.Interval;
name = "Domain",
indexes = {
@javax.persistence.Index(columnList = "creationTime"),
@javax.persistence.Index(columnList = "currentSponsorClientId"),
@javax.persistence.Index(columnList = "currentSponsorRegistrarId"),
@javax.persistence.Index(columnList = "deletionTime"),
@javax.persistence.Index(columnList = "fullyQualifiedDomainName"),
@javax.persistence.Index(columnList = "tld")
})
@WithStringVKey
@ExternalMessagingName("domain")
public class DomainBase extends EppResource
implements DatastoreAndSqlEntity, ForeignKeyedEppResource, ResourceWithTransferData {
@@ -136,18 +138,11 @@ public class DomainBase extends EppResource
@Index
String tld;
/**
* References to hosts that are the nameservers for the domain.
*
* <p>This is a legacy field: we have to preserve it because it is still persisted and indexed in
* the datastore, but all external references go through nsHostVKeys.
*/
@Index @ElementCollection @Transient Set<Key<HostResource>> nsHosts;
@Ignore
/** References to hosts that are the nameservers for the domain. */
@Index
@ElementCollection
@JoinTable(name = "DomainHost")
Set<VKey<HostResource>> nsHostVKeys;
Set<VKey<HostResource>> nsHosts;
/**
* The union of the contacts visible via {@link #getContacts} and {@link #getRegistrant}.
@@ -258,7 +253,7 @@ public class DomainBase extends EppResource
String smdId;
/** Data about any pending or past transfers on this domain. */
@Transient TransferData transferData;
TransferData transferData;
/**
* The time that this resource was last transferred.
@@ -269,11 +264,6 @@ public class DomainBase extends EppResource
@OnLoad
void load() {
nsHostVKeys =
nullToEmptyImmutableCopy(nsHosts).stream()
.map(hostKey -> VKey.createOfy(HostResource.class, hostKey))
.collect(toImmutableSet());
// Reconstitute all of the contacts so that they have VKeys.
allContacts =
allContacts.stream().map(contact -> contact.reconstitute()).collect(toImmutableSet());
@@ -363,9 +353,7 @@ public class DomainBase extends EppResource
}
public ImmutableSet<VKey<HostResource>> getNameservers() {
// Since nsHostVKeys gets initialized both from setNameservers() and the OnLoad method, this
// should always be valid.
return nullToEmptyImmutableCopy(nsHostVKeys);
return nullToEmptyImmutableCopy(nsHosts);
}
public final String getCurrentSponsorClientId() {
@@ -450,8 +438,14 @@ public class DomainBase extends EppResource
.setRegistrationExpirationTime(expirationDate)
// Set the speculatively-written new autorenew events as the domain's autorenew
// events.
.setAutorenewBillingEvent(transferData.getServerApproveAutorenewEvent())
.setAutorenewPollMessage(transferData.getServerApproveAutorenewPollMessage());
.setAutorenewBillingEvent(
transferData.getServerApproveAutorenewEvent() == null
? null
: transferData.getServerApproveAutorenewEvent().getOfyKey())
.setAutorenewPollMessage(
transferData.getServerApproveAutorenewPollMessage() == null
? null
: transferData.getServerApproveAutorenewPollMessage().getOfyKey());
if (transferData.getTransferPeriod().getValue() == 1) {
// Set the grace period using a key to the prescheduled transfer billing event. Not using
// GracePeriod.forBillingEvent() here in order to avoid the actual Datastore fetch.
@@ -462,7 +456,9 @@ public class DomainBase extends EppResource
transferExpirationTime.plus(
Registry.get(getTld()).getTransferGracePeriodLength()),
transferData.getGainingClientId(),
transferData.getServerApproveBillingEvent())));
transferData.getServerApproveBillingEvent() == null
? null
: transferData.getServerApproveBillingEvent().getOfyKey())));
} else {
// There won't be a billing event, so we don't need a grace period
builder.setGracePeriods(ImmutableSet.of());
@@ -627,6 +623,12 @@ public class DomainBase extends EppResource
}
}
@Override
public VKey<DomainBase> createVKey() {
// TODO(mmuller): create symmetric keys if we can ever reload both sides.
return VKey.createOfy(DomainBase.class, Key.create(this));
}
/** Predicate to determine if a given {@link DesignatedContact} is the registrant. */
private static final Predicate<DesignatedContact> IS_REGISTRANT =
(DesignatedContact contact) -> DesignatedContact.Type.REGISTRANT.equals(contact.type);
@@ -645,14 +647,6 @@ public class DomainBase extends EppResource
Builder(DomainBase instance) {
super(instance);
// Convert nsHosts to nsHostVKeys.
if (instance.nsHosts != null) {
instance.nsHostVKeys =
instance.nsHosts.stream()
.map(key -> VKey.createOfy(HostResource.class, key))
.collect(toImmutableSet());
}
}
@Override
@@ -710,27 +704,12 @@ public class DomainBase extends EppResource
}
public Builder setNameservers(VKey<HostResource> nameserver) {
Optional<Key<HostResource>> nsKey = nameserver.maybeGetOfyKey();
if (nsKey.isPresent()) {
getInstance().nsHosts = ImmutableSet.of(nsKey.get());
} else {
getInstance().nsHosts = null;
}
getInstance().nsHostVKeys = ImmutableSet.of(nameserver);
getInstance().nsHosts = ImmutableSet.of(nameserver);
return thisCastToDerived();
}
public Builder setNameservers(ImmutableSet<VKey<HostResource>> nameservers) {
// If we have all of the ofy keys, we can set nsHosts. Otherwise, make it null.
if (nameservers != null
&& nameservers.stream().allMatch(key -> key.maybeGetOfyKey().isPresent())) {
getInstance().nsHosts =
nameservers.stream().map(key -> key.getOfyKey()).collect(toImmutableSet());
} else {
getInstance().nsHosts = null;
}
getInstance().nsHostVKeys = forceEmptyToNull(nameservers);
getInstance().nsHosts = forceEmptyToNull(nameservers);
return thisCastToDerived();
}
@@ -30,7 +30,6 @@ import com.google.common.base.MoreObjects;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.model.EppResource;
import google.registry.model.ImmutableObject;
import google.registry.model.contact.ContactResource;
@@ -72,18 +71,17 @@ public class DomainCommand {
T cloneAndLinkReferences(DateTime now) throws InvalidReferencesException;
}
/** The fields on "chgType" from {@link "http://tools.ietf.org/html/rfc5731"}. */
/** The fields on "chgType" from <a href="http://tools.ietf.org/html/rfc5731">RFC5731</a>. */
@XmlTransient
public static class DomainCreateOrChange<B extends DomainBase.Builder>
extends ImmutableObject implements ResourceCreateOrChange<B> {
public static class DomainCreateOrChange<B extends DomainBase.Builder> extends ImmutableObject
implements ResourceCreateOrChange<B> {
/** The contactId of the registrant who registered this domain. */
@XmlElement(name = "registrant")
String registrantContactId;
/** A resolved key to the registrant who registered this domain. */
@XmlTransient
Key<ContactResource> registrant;
@XmlTransient VKey<ContactResource> registrant;
/** Authorization info (aka transfer secret) of the domain. */
DomainAuthInfo authInfo;
@@ -93,7 +91,7 @@ public class DomainCommand {
}
@Nullable
public Key<ContactResource> getRegistrant() {
public VKey<ContactResource> getRegistrant() {
return registrant;
}
@@ -103,19 +101,20 @@ public class DomainCommand {
}
/**
* A create command for a {@link DomainBase}, mapping "createType" from
* {@link "http://tools.ietf.org/html/rfc5731"}.
* A create command for a {@link DomainBase}, mapping "createType" from <a
* href="http://tools.ietf.org/html/rfc5731">RFC5731</a>.
*/
@XmlRootElement
@XmlType(propOrder = {
"fullyQualifiedDomainName",
"period",
"nameserverFullyQualifiedHostNames",
"registrantContactId",
"foreignKeyedDesignatedContacts",
"authInfo"})
public static class Create
extends DomainCreateOrChange<DomainBase.Builder>
@XmlType(
propOrder = {
"fullyQualifiedDomainName",
"period",
"nameserverFullyQualifiedHostNames",
"registrantContactId",
"foreignKeyedDesignatedContacts",
"authInfo"
})
public static class Create extends DomainCreateOrChange<DomainBase.Builder>
implements CreateOrUpdate<Create> {
/** Fully qualified domain name, which serves as a unique identifier for this domain. */
@@ -128,8 +127,7 @@ public class DomainCommand {
Set<String> nameserverFullyQualifiedHostNames;
/** Resolved keys to hosts that are the nameservers for the domain. */
@XmlTransient
Set<Key<HostResource>> nameservers;
@XmlTransient Set<VKey<HostResource>> nameservers;
/** Foreign keyed associated contacts for the domain (other than registrant). */
@XmlElement(name = "contact")
@@ -159,7 +157,7 @@ public class DomainCommand {
return nullToEmptyImmutableCopy(nameserverFullyQualifiedHostNames);
}
public ImmutableSet<Key<HostResource>> getNameservers() {
public ImmutableSet<VKey<HostResource>> getNameservers() {
return nullToEmptyImmutableCopy(nameservers);
}
@@ -189,7 +187,7 @@ public class DomainCommand {
now);
for (DesignatedContact contact : contacts) {
if (DesignatedContact.Type.REGISTRANT.equals(contact.getType())) {
clone.registrant = contact.getContactKey().getOfyKey();
clone.registrant = contact.getContactKey();
clone.contacts = forceEmptyToNull(difference(contacts, contact));
break;
}
@@ -353,8 +351,7 @@ public class DomainCommand {
Set<String> nameserverFullyQualifiedHostNames;
/** Resolved keys to hosts that are the nameservers for the domain. */
@XmlTransient
Set<Key<HostResource>> nameservers;
@XmlTransient Set<VKey<HostResource>> nameservers;
/** Foreign keyed associated contacts for the domain (other than registrant). */
@XmlElement(name = "contact")
@@ -368,7 +365,7 @@ public class DomainCommand {
return nullSafeImmutableCopy(nameserverFullyQualifiedHostNames);
}
public ImmutableSet<Key<HostResource>> getNameservers() {
public ImmutableSet<VKey<HostResource>> getNameservers() {
return nullToEmptyImmutableCopy(nameservers);
}
@@ -418,7 +415,7 @@ public class DomainCommand {
}
}
private static Set<Key<HostResource>> linkHosts(
private static Set<VKey<HostResource>> linkHosts(
Set<String> fullyQualifiedHostNames, DateTime now) throws InvalidReferencesException {
if (fullyQualifiedHostNames == null) {
return null;
@@ -436,20 +433,18 @@ public class DomainCommand {
for (ForeignKeyedDesignatedContact contact : contacts) {
foreignKeys.add(contact.contactId);
}
ImmutableMap<String, Key<ContactResource>> loadedContacts =
ImmutableMap<String, VKey<ContactResource>> loadedContacts =
loadByForeignKeysCached(foreignKeys.build(), ContactResource.class, now);
ImmutableSet.Builder<DesignatedContact> linkedContacts = new ImmutableSet.Builder<>();
for (ForeignKeyedDesignatedContact contact : contacts) {
linkedContacts.add(
DesignatedContact.create(
contact.type,
VKey.createOfy(ContactResource.class, loadedContacts.get(contact.contactId))));
DesignatedContact.create(contact.type, loadedContacts.get(contact.contactId)));
}
return linkedContacts.build();
}
/** Loads keys to cached EPP resources by their foreign keys. */
private static <T extends EppResource> ImmutableMap<String, Key<T>> loadByForeignKeysCached(
private static <T extends EppResource> ImmutableMap<String, VKey<T>> loadByForeignKeysCached(
final Set<String> foreignKeys, final Class<T> clazz, final DateTime now)
throws InvalidReferencesException {
Map<String, ForeignKeyIndex<T>> fkis = ForeignKeyIndex.loadCached(clazz, foreignKeys, now);
@@ -24,6 +24,7 @@ import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingEvent;
import google.registry.model.domain.rgp.GracePeriodStatus;
import javax.annotation.Nullable;
import javax.persistence.Column;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import org.joda.time.DateTime;
@@ -51,6 +52,7 @@ public class GracePeriod extends ImmutableObject {
DateTime expirationTime;
/** The registrar to bill. */
@Column(name = "registrarId")
String clientId;
/**
@@ -16,15 +16,18 @@ package google.registry.model.domain;
import com.googlecode.objectify.annotation.Embed;
import google.registry.model.ImmutableObject;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.bind.annotation.XmlEnumValue;
import javax.xml.bind.annotation.XmlValue;
/** The "periodType" from {@link "http://tools.ietf.org/html/rfc5731"}. */
/** The "periodType" from <a href="http://tools.ietf.org/html/rfc5731">RFC5731</a>. */
@Embed
@javax.persistence.Embeddable
public class Period extends ImmutableObject {
@Enumerated(EnumType.STRING)
@XmlAttribute
Unit unit;
@@ -29,11 +29,13 @@ import org.joda.time.DateTime;
* An individual price check item in version 0.12 of the fee extension on domain check commands.
* Items look like:
*
* <pre>{@code
* <fee:command name="renew" phase="sunrise" subphase="hello">
* <fee:period unit="y">1</fee:period>
* <fee:class>premium</fee:class>
* <fee:date>2017-05-17T13:22:21.0Z</fee:date>
* </fee:command>
* }</pre>
*
* In a change from previous versions of the extension, items do not contain domain names; instead,
* the names from the non-extension check element are used.
@@ -95,9 +95,6 @@ public class LaunchNotice extends ImmutableObject {
/**
* Validate the checksum of the notice against the domain label.
*
* @throws IllegalArgumentException
* @throws InvalidChecksumException
*/
public void validate(String domainLabel) throws InvalidChecksumException {
// According to http://tools.ietf.org/html/draft-lozano-tmch-func-spec-08#section-6.3, a TCNID
@@ -49,9 +49,9 @@ public enum GracePeriodStatus implements EppEnum {
AUTO_RENEW("autoRenewPeriod"),
/**
* This status value is used to describe a domain for which a <delete> command has been received,
* but the domain has not yet been purged because an opportunity exists to restore the domain and
* abort the deletion process.
* This status value is used to describe a domain for which a &lt;delete&gt; command has been
* received, but the domain has not yet been purged because an opportunity exists to restore the
* domain and abort the deletion process.
*/
REDEMPTION("redemptionPeriod"),
@@ -45,6 +45,8 @@ import google.registry.model.common.TimedTransitionProperty;
import google.registry.model.common.TimedTransitionProperty.TimeMapper;
import google.registry.model.common.TimedTransitionProperty.TimedTransition;
import google.registry.model.reporting.HistoryEntry;
import google.registry.persistence.VKey;
import google.registry.persistence.WithStringVKey;
import java.util.Optional;
import java.util.Set;
import javax.annotation.Nullable;
@@ -53,6 +55,7 @@ import org.joda.time.DateTime;
/** An entity representing an allocation token. */
@ReportedOn
@Entity
@WithStringVKey
public class AllocationToken extends BackupGroupRoot implements Buildable {
// Promotions should only move forward, and ENDED / CANCELLED are terminal states.
@@ -179,6 +182,10 @@ public class AllocationToken extends BackupGroupRoot implements Buildable {
return tokenStatusTransitions;
}
public VKey<AllocationToken> createVKey() {
return VKey.createOfy(getClass(), Key.create(this));
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
@@ -25,7 +25,7 @@ import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
* An allocation token extension that may be present on EPP domain commands.
*
* @see <a href="https://tools.ietf.org/html/draft-ietf-regext-allocation-token-04">the IETF
* draft</a> for full details.
* draft</a>
*/
@XmlRootElement(name = "allocationToken")
public class AllocationTokenExtension extends ImmutableObject implements CommandExtension {
@@ -44,8 +44,9 @@ import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
/**
* Container for generic street address.
*
* <p>This is the "addrType" type from {@link "http://tools.ietf.org/html/rfc5733"}. It also matches
* the "addrType" type from {@link "http://tools.ietf.org/html/draft-lozano-tmch-smd"}.
* <p>This is the "addrType" type from <a href="http://tools.ietf.org/html/rfc5733">RFC5733</a>. It
* also matches the "addrType" type from <a
* href="http://tools.ietf.org/html/draft-lozano-tmch-smd">Mark and Signed Mark Objects Mapping</a>.
*
* @see google.registry.model.contact.ContactAddress
* @see google.registry.model.mark.MarkAddress
@@ -29,8 +29,9 @@ import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
/**
* Container for generic E164 phone number.
*
* <p>This is the "e164" type from {@link "http://tools.ietf.org/html/rfc5733"}. It also matches the
* "e164Type" type from {@link "http://tools.ietf.org/html/draft-lozano-tmch-smd"}.
* <p>This is the "e164" type from <a href="http://tools.ietf.org/html/rfc5733">RFC5733</a>. It also
* matches the "e164Type" type from <a href="http://tools.ietf.org/html/draft-lozano-tmch-smd">Mark
* and Signed Mark Objects Mapping</a>
*
* <blockquote>
*
@@ -24,7 +24,7 @@ import javax.xml.bind.annotation.XmlElementWrapper;
import org.joda.time.DateTime;
/**
* A greeting, defined in {@link "http://tools.ietf.org/html/rfc5730"}.
* A greeting, defined in <a href="http://tools.ietf.org/html/rfc5730">RFC5730</a>.
*
* <p>It would be nice to make this a singleton, but we need the {@link #svDate} field to stay
* current.
@@ -32,7 +32,7 @@ import javax.xml.bind.annotation.XmlType;
/** A collection of {@link HostResource} commands. */
public class HostCommand {
/** The fields on "chgType" from {@link "http://tools.ietf.org/html/rfc5732"}. */
/** The fields on "chgType" from <a href="http://tools.ietf.org/html/rfc5732">RFC5732</a>. */
@XmlTransient
abstract static class HostCreateOrChange extends AbstractSingleResourceCommand
implements ResourceCreateOrChange<HostResource.Builder> {
@@ -42,13 +42,13 @@ public class HostCommand {
}
/**
* A create command for a {@link HostResource}, mapping "createType" from
* {@link "http://tools.ietf.org/html/rfc5732"}.
* A create command for a {@link HostResource}, mapping "createType" from <a
* href="http://tools.ietf.org/html/rfc5732">RFC5732</a>.
*/
@XmlType(propOrder = {"targetId", "inetAddresses" })
@XmlType(propOrder = {"targetId", "inetAddresses"})
@XmlRootElement
public static class Create
extends HostCreateOrChange implements ResourceCreateOrChange<HostResource.Builder> {
public static class Create extends HostCreateOrChange
implements ResourceCreateOrChange<HostResource.Builder> {
/** IP Addresses for this host. Can be null if this is an external host. */
@XmlElement(name = "addr")
Set<InetAddress> inetAddresses;
@@ -40,7 +40,6 @@ import java.net.InetAddress;
import java.util.Optional;
import java.util.Set;
import javax.annotation.Nullable;
import javax.persistence.ElementCollection;
import org.joda.time.DateTime;
/**
@@ -70,13 +69,13 @@ public class HostResource extends EppResource
String fullyQualifiedHostName;
/** IP Addresses for this host. Can be null if this is an external host. */
@Index @ElementCollection Set<InetAddress> inetAddresses;
@Index Set<InetAddress> inetAddresses;
/** The superordinate domain of this host, or null if this is an external host. */
@Index
@IgnoreSave(IfNull.class)
@DoNotHydrate
Key<DomainBase> superordinateDomain;
VKey<DomainBase> superordinateDomain;
/**
* The time that this resource was last transferred.
@@ -98,7 +97,7 @@ public class HostResource extends EppResource
return fullyQualifiedHostName;
}
public Key<DomainBase> getSuperordinateDomain() {
public VKey<DomainBase> getSuperordinateDomain() {
return superordinateDomain;
}
@@ -123,7 +122,9 @@ public class HostResource extends EppResource
return fullyQualifiedHostName;
}
public VKey<HostResource> createKey() {
@Override
public VKey<HostResource> createVKey() {
// TODO(mmuller): create symmetric keys if we can ever reload both sides.
return VKey.createOfy(HostResource.class, Key.create(this));
}
@@ -143,18 +144,18 @@ public class HostResource extends EppResource
*
* <p>If the host is not subordinate the domain can be null and we just return last transfer time.
*
* @param superordinateDomain the loaded superordinate domain, which must match the key in
* the {@link #superordinateDomain} field. Passing it as a parameter allows the caller to
* control the degree of consistency used to load it.
* @param superordinateDomain the loaded superordinate domain, which must match the key in the
* {@link #superordinateDomain} field. Passing it as a parameter allows the caller to control
* the degree of consistency used to load it.
*/
public DateTime computeLastTransferTime(@Nullable DomainBase superordinateDomain) {
public DateTime computeLastTransferTime(@Nullable DomainBase superordinateDomain) {
if (!isSubordinate()) {
checkArgument(superordinateDomain == null);
return getLastTransferTime();
}
checkArgument(
superordinateDomain != null
&& Key.create(superordinateDomain).equals(getSuperordinateDomain()));
&& superordinateDomain.createVKey().equals(getSuperordinateDomain()));
DateTime lastSuperordinateChange =
Optional.ofNullable(getLastSuperordinateChange()).orElse(getCreationTime());
DateTime lastTransferOfCurrentSuperordinate =
@@ -205,7 +206,7 @@ public class HostResource extends EppResource
difference(getInstance().getInetAddresses(), inetAddresses)));
}
public Builder setSuperordinateDomain(Key<DomainBase> superordinateDomain) {
public Builder setSuperordinateDomain(VKey<DomainBase> superordinateDomain) {
getInstance().superordinateDomain = superordinateDomain;
return this;
}
@@ -43,6 +43,7 @@ import google.registry.model.annotations.ReportedOn;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.DomainBase;
import google.registry.model.host.HostResource;
import google.registry.persistence.VKey;
import google.registry.util.NonFinalForTesting;
import java.util.Map;
import java.util.Optional;
@@ -99,7 +100,7 @@ public abstract class ForeignKeyIndex<E extends EppResource> extends BackupGroup
* <p>This field holds a key to the only referenced resource. It is named "topReference" for
* historical reasons.
*/
Key<E> topReference;
VKey<E> topReference;
public String getForeignKey() {
return foreignKey;
@@ -109,7 +110,7 @@ public abstract class ForeignKeyIndex<E extends EppResource> extends BackupGroup
return deletionTime;
}
public Key<E> getResourceKey() {
public VKey<E> getResourceKey() {
return topReference;
}
@@ -125,7 +126,7 @@ public abstract class ForeignKeyIndex<E extends EppResource> extends BackupGroup
@SuppressWarnings("unchecked")
Class<E> resourceClass = (Class<E>) resource.getClass();
ForeignKeyIndex<E> instance = instantiate(mapToFkiClass(resourceClass));
instance.topReference = Key.create(resource);
instance.topReference = (VKey<E>) resource.createVKey();
instance.foreignKey = resource.getForeignKey();
instance.deletionTime = deletionTime;
return instance;
@@ -141,18 +142,18 @@ public abstract class ForeignKeyIndex<E extends EppResource> extends BackupGroup
/**
* Loads a {@link Key} to an {@link EppResource} from Datastore by foreign key.
*
* <p>Returns null if no foreign key index with this foreign key was ever created, or if the
* most recently created foreign key index was deleted before time "now". This method does not
* actually check that the referenced resource actually exists. However, for normal epp resources,
* it is safe to assume that the referenced resource exists if the foreign key index does.
* <p>Returns null if no foreign key index with this foreign key was ever created, or if the most
* recently created foreign key index was deleted before time "now". This method does not actually
* check that the referenced resource actually exists. However, for normal epp resources, it is
* safe to assume that the referenced resource exists if the foreign key index does.
*
* @param clazz the resource type to load
* @param foreignKey id to match
* @param now the current logical time to use when checking for soft deletion of the foreign key
* index
* index
*/
@Nullable
public static <E extends EppResource> Key<E> loadAndGetKey(
public static <E extends EppResource> VKey<E> loadAndGetKey(
Class<E> clazz, String foreignKey, DateTime now) {
ForeignKeyIndex<E> index = load(clazz, foreignKey, now);
return (index == null) ? null : index.getResourceKey();
@@ -14,6 +14,7 @@
package google.registry.model.ofy;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
@@ -157,7 +158,7 @@ public class DatastoreTransactionManager implements TransactionManager {
@Override
public <T> ImmutableList<T> load(Iterable<VKey<T>> keys) {
Iterator<Key<T>> iter =
StreamSupport.stream(keys.spliterator(), false).map(key -> key.getOfyKey()).iterator();
StreamSupport.stream(keys.spliterator(), false).map(VKey::getOfyKey).iterator();
// The lambda argument to keys() effectively converts Iterator -> Iterable.
return ImmutableList.copyOf(getOfy().load().keys(() -> iter).values());
@@ -170,7 +171,18 @@ public class DatastoreTransactionManager implements TransactionManager {
}
@Override
public <T> void delete(VKey<T> key) {
public void delete(VKey<?> key) {
getOfy().delete().key(key.getOfyKey()).now();
}
@Override
public void delete(Iterable<? extends VKey<?>> vKeys) {
// We have to create a list to work around the wildcard capture issue here.
// See https://docs.oracle.com/javase/tutorial/java/generics/capture.html
ImmutableList<Key<?>> list =
StreamSupport.stream(vKeys.spliterator(), false)
.map(VKey::getOfyKey)
.collect(toImmutableList());
getOfy().delete().keys(list).now();
}
}
@@ -34,10 +34,9 @@ import com.googlecode.objectify.annotation.EntitySubclass;
import com.googlecode.objectify.impl.translate.TranslatorFactory;
import com.googlecode.objectify.impl.translate.opt.joda.MoneyStringTranslatorFactory;
import google.registry.config.RegistryEnvironment;
import google.registry.model.Buildable;
import google.registry.model.EntityClasses;
import google.registry.model.ImmutableObject;
import google.registry.model.contact.ContactResource;
import google.registry.model.host.HostResource;
import google.registry.model.translators.BloomFilterOfStringTranslatorFactory;
import google.registry.model.translators.CidrAddressBlockTranslatorFactory;
import google.registry.model.translators.CommitLogRevisionsTranslatorFactory;
@@ -131,8 +130,7 @@ public class ObjectifyService {
new InetAddressTranslatorFactory(),
new MoneyStringTranslatorFactory(),
new ReadableInstantUtcTranslatorFactory(),
new VKeyTranslatorFactory<ContactResource>(ContactResource.class),
new VKeyTranslatorFactory<HostResource>(HostResource.class),
new VKeyTranslatorFactory(),
new UpdateAutoTimestampTranslatorFactory())) {
factory().getTranslators().add(translatorFactory);
}
@@ -170,10 +168,16 @@ public class ObjectifyService {
}
com.googlecode.objectify.ObjectifyService.register(clazz);
// Autogenerated ids make the commit log code very difficult since we won't always be able
// to create a key for an entity immediately when requesting a save. Disallow that here.
checkState(
!factory().getMetadata(clazz).getKeyMetadata().isIdGeneratable(),
"Can't register %s: Autogenerated ids (@Id on a Long) are not supported.", kind);
// to create a key for an entity immediately when requesting a save. So, we require such
// entities to implement google.registry.model.Buildable as its build() function allocates the
// id to the entity.
if (factory().getMetadata(clazz).getKeyMetadata().isIdGeneratable()) {
checkState(
Buildable.class.isAssignableFrom(clazz),
"Can't register %s: Entity with autogenerated ids (@Id on a Long) must implement"
+ " google.registry.model.Buildable.",
kind);
}
}
}
@@ -19,6 +19,7 @@ import com.googlecode.objectify.annotation.Embed;
import google.registry.model.ImmutableObject;
import google.registry.model.eppcommon.Trid;
import google.registry.model.eppoutput.EppResponse.ResponseData;
import javax.persistence.Embeddable;
import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
@@ -29,11 +30,12 @@ import org.joda.time.DateTime;
/** The {@link ResponseData} returned when completing a pending action on a domain. */
@XmlTransient
public abstract class PendingActionNotificationResponse
extends ImmutableObject implements ResponseData {
@Embeddable
public class PendingActionNotificationResponse extends ImmutableObject implements ResponseData {
/** The inner name type that contains a name and the result boolean. */
@Embed
@Embeddable
static class NameOrId extends ImmutableObject {
@XmlValue
String value;
@@ -25,6 +25,7 @@ import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.EntitySubclass;
import com.googlecode.objectify.annotation.Id;
import com.googlecode.objectify.annotation.Ignore;
import com.googlecode.objectify.annotation.Index;
import com.googlecode.objectify.annotation.Parent;
import google.registry.model.Buildable;
@@ -39,10 +40,24 @@ import google.registry.model.poll.PendingActionNotificationResponse.DomainPendin
import google.registry.model.poll.PendingActionNotificationResponse.HostPendingActionNotificationResponse;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
import google.registry.model.transfer.TransferResponse;
import google.registry.model.transfer.TransferResponse.ContactTransferResponse;
import google.registry.model.transfer.TransferResponse.DomainTransferResponse;
import google.registry.persistence.VKey;
import google.registry.persistence.WithLongVKey;
import java.util.List;
import java.util.Optional;
import javax.persistence.AttributeOverride;
import javax.persistence.AttributeOverrides;
import javax.persistence.Column;
import javax.persistence.DiscriminatorColumn;
import javax.persistence.DiscriminatorValue;
import javax.persistence.Embedded;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Inheritance;
import javax.persistence.InheritanceType;
import javax.persistence.Transient;
import org.joda.time.DateTime;
/**
@@ -62,34 +77,58 @@ import org.joda.time.DateTime;
* <p>Poll messages are identified externally by registrars using the format defined in {@link
* PollMessageExternalKeyConverter}.
*
* @see <a href="https://tools.ietf.org/html/rfc5730#section-2.9.2.3">RFC5730 - EPP - &ltpoll&gt
* @see <a href="https://tools.ietf.org/html/rfc5730#section-2.9.2.3">RFC5730 - EPP - &lt;poll&gt;
* Command</a>
*/
@Entity
@ReportedOn
@ExternalMessagingName("message")
@javax.persistence.Entity
@Inheritance(strategy = InheritanceType.SINGLE_TABLE)
@DiscriminatorColumn(name = "type")
@javax.persistence.Table(
indexes = {
@javax.persistence.Index(columnList = "registrar_id"),
@javax.persistence.Index(columnList = "eventTime")
})
public abstract class PollMessage extends ImmutableObject
implements Buildable, TransferServerApproveEntity {
/** Entity id. */
@Id
long id;
@javax.persistence.Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "poll_message_id")
Long id;
@Parent
@DoNotHydrate
Key<HistoryEntry> parent;
@Parent @DoNotHydrate @Transient Key<HistoryEntry> parent;
/** The registrar that this poll message will be delivered to. */
@Index
@Column(name = "registrar_id", nullable = false)
String clientId;
/** The time when the poll message should be delivered. May be in the future. */
@Index
@Column(nullable = false)
DateTime eventTime;
/** Human readable message that will be returned with this poll message. */
@Column(name = "message")
String msg;
@Ignore String domainRepoId;
@Ignore String contactRepoId;
@Ignore String hostRepoId;
@Ignore Long domainRevisionId;
@Ignore Long contactRevisionId;
@Ignore Long hostRevisionId;
public Key<HistoryEntry> getParentKey() {
return parent;
}
@@ -112,6 +151,9 @@ public abstract class PollMessage extends ImmutableObject
public abstract ImmutableList<ResponseData> getResponseData();
@Override
public abstract VKey<? extends PollMessage> createVKey();
/** Override Buildable.asBuilder() to give this method stronger typing. */
@Override
public abstract Builder<?, ?> asBuilder();
@@ -180,15 +222,83 @@ public abstract class PollMessage extends ImmutableObject
* <p>One-time poll messages are deleted from Datastore once they have been delivered and ACKed.
*/
@EntitySubclass(index = false)
@javax.persistence.Entity
@DiscriminatorValue("ONE_TIME")
@WithLongVKey
public static class OneTime extends PollMessage {
// Response data. Objectify cannot persist a base class type, so we must have a separate field
// to hold every possible derived type of ResponseData that we might store.
@Transient
List<ContactPendingActionNotificationResponse> contactPendingActionNotificationResponses;
List<ContactTransferResponse> contactTransferResponses;
@Transient List<ContactTransferResponse> contactTransferResponses;
@Transient
List<DomainPendingActionNotificationResponse> domainPendingActionNotificationResponses;
List<DomainTransferResponse> domainTransferResponses;
List<HostPendingActionNotificationResponse> hostPendingActionNotificationResponses;
@Transient List<DomainTransferResponse> domainTransferResponses;
@Transient List<HostPendingActionNotificationResponse> hostPendingActionNotificationResponses;
@Ignore
@Embedded
@AttributeOverrides({
@AttributeOverride(
name = "nameOrId.value",
column = @Column(name = "pending_action_response_name_or_id")),
@AttributeOverride(
name = "nameOrId.actionResult",
column = @Column(name = "pending_action_response_action_result")),
@AttributeOverride(
name = "trid.serverTransactionId",
column = @Column(name = "pending_action_response_server_txn_id")),
@AttributeOverride(
name = "trid.clientTransactionId",
column = @Column(name = "pending_action_response_client_txn_id")),
@AttributeOverride(
name = "processedDate",
column = @Column(name = "pending_action_response_processed_date"))
})
PendingActionNotificationResponse pendingActionNotificationResponse;
@Ignore
@Embedded
@AttributeOverrides({
@AttributeOverride(
name = "transferStatus",
column = @Column(name = "transfer_response_transfer_status")),
@AttributeOverride(
name = "gainingClientId",
column = @Column(name = "transfer_response_gaining_registrar_id")),
@AttributeOverride(
name = "transferRequestTime",
column = @Column(name = "transfer_response_transfer_request_time")),
@AttributeOverride(
name = "losingClientId",
column = @Column(name = "transfer_response_losing_registrar_id")),
@AttributeOverride(
name = "pendingTransferExpirationTime",
column = @Column(name = "transfer_response_pending_transfer_expiration_time"))
})
TransferResponse transferResponse;
@Ignore
@Column(name = "transfer_response_domain_name")
String fullyQualifiedDomainName;
@Ignore
@Column(name = "transfer_response_domain_expiration_time")
DateTime extendedRegistrationExpirationTime;
@Ignore
@Column(name = "transfer_response_contact_id")
String contactId;
@Override
public VKey<OneTime> createVKey() {
return VKey.createOfy(this.getClass(), Key.create(this));
}
@Override
public Builder asBuilder() {
@@ -265,9 +375,13 @@ public abstract class PollMessage extends ImmutableObject
* happens.
*/
@EntitySubclass(index = false)
@javax.persistence.Entity
@DiscriminatorValue("AUTORENEW")
@WithLongVKey
public static class Autorenew extends PollMessage {
/** The target id of the autorenew event. */
@Column(name = "autorenew_domain_name")
String targetId;
/** The autorenew recurs annually between {@link #eventTime} and this time. */
@@ -282,6 +396,11 @@ public abstract class PollMessage extends ImmutableObject
return autorenewEndTime;
}
@Override
public VKey<Autorenew> createVKey() {
return VKey.createOfy(this.getClass(), Key.create(this));
}
@Override
public ImmutableList<ResponseData> getResponseData() {
// Note that the event time is when the auto-renew occured, so the expiration time in the
@@ -236,7 +236,7 @@ public class Registrar extends ImmutableObject
*/
@Id
@javax.persistence.Id
@Column(name = "client_id", nullable = false)
@Column(name = "registrarId", nullable = false)
String clientIdentifier;
/**
@@ -158,8 +158,8 @@ public final class ReservedList
/**
* Gets a ReservedList by name using the caching layer.
*
* @return An Optional<ReservedList> that has a value if a reserved list exists by the given name,
* or absent if not.
* @return An Optional&lt;ReservedList&gt; that has a value if a reserved list exists by the given
* name, or absent if not.
* @throws UncheckedExecutionException if some other error occurs while trying to load the
* ReservedList from the cache or Datastore.
*/
@@ -22,8 +22,8 @@ public final class IcannReportingTypes {
* Represents the set of possible ICANN Monthly Registry Functions Activity Report fields.
*
* <p>Refer to the <a
* href="https://newgtlds.icann.org/sites/default/files/agreements/agreement-approved-09jan14-en.htm#_DV_M278>ICANN
* registry agreement Specification 3 Section 2</a> for details.
* href="https://newgtlds.icann.org/sites/default/files/agreements/agreement-approved-09jan14-en.htm#_DV_M278">
* ICANN registry agreement Specification 3 Section 2</a> for details.
*/
public enum ActivityReportField {
DOMAIN_CHECK("srs-dom-check"),
@@ -16,6 +16,9 @@ package google.registry.model.transfer;
import google.registry.model.Buildable.GenericBuilder;
import google.registry.model.ImmutableObject;
import javax.persistence.Column;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.MappedSuperclass;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlTransient;
@@ -31,10 +34,12 @@ public abstract class BaseTransferObject extends ImmutableObject {
* will always be non-null.
*/
@XmlElement(name = "trStatus")
@Enumerated(EnumType.STRING)
TransferStatus transferStatus;
/** The gaining registrar of the current or last transfer. Can be null if never transferred. */
@XmlElement(name = "reID")
@Column(name = "transfer_gaining_registrar_id")
String gainingClientId;
/** The time that the last transfer was requested. Can be null if never transferred. */
@@ -43,6 +48,7 @@ public abstract class BaseTransferObject extends ImmutableObject {
/** The losing registrar of the current or last transfer. Can be null if never transferred. */
@XmlElement(name = "acID")
@Column(name = "transfer_losing_registrar_id")
String losingClientId;
/**
@@ -51,6 +57,7 @@ public abstract class BaseTransferObject extends ImmutableObject {
* this holds the time that the last pending transfer ended. Can be null if never transferred.
*/
@XmlElement(name = "acDate")
@Column(name = "transfer_pending_expiration_time")
DateTime pendingTransferExpirationTime;
public TransferStatus getTransferStatus() {
@@ -17,8 +17,8 @@ package google.registry.model.transfer;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableCopy;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.Embed;
import com.googlecode.objectify.annotation.Ignore;
import com.googlecode.objectify.annotation.IgnoreSave;
import com.googlecode.objectify.annotation.Unindex;
import com.googlecode.objectify.condition.IfNull;
@@ -29,10 +29,14 @@ import google.registry.model.domain.Period;
import google.registry.model.domain.Period.Unit;
import google.registry.model.eppcommon.Trid;
import google.registry.model.poll.PollMessage;
import google.registry.persistence.VKey;
import java.util.Set;
import javax.annotation.Nullable;
import javax.persistence.ElementCollection;
import javax.persistence.AttributeOverride;
import javax.persistence.AttributeOverrides;
import javax.persistence.Column;
import javax.persistence.Embedded;
import javax.persistence.Transient;
import org.joda.time.DateTime;
/**
@@ -47,7 +51,16 @@ public class TransferData extends BaseTransferObject implements Buildable {
public static final TransferData EMPTY = new TransferData();
/** The transaction id of the most recent transfer request (or null if there never was one). */
@Embedded Trid transferRequestTrid;
@Embedded
@AttributeOverrides({
@AttributeOverride(
name = "serverTransactionId",
column = @Column(name = "transfer_server_txn_id")),
@AttributeOverride(
name = "clientTransactionId",
column = @Column(name = "transfer_client_txn_id"))
})
Trid transferRequestTrid;
/**
* The period to extend the registration upon completion of the transfer.
@@ -55,23 +68,29 @@ public class TransferData extends BaseTransferObject implements Buildable {
* <p>By default, domain transfers are for one year. This can be changed to zero by using the
* superuser EPP extension.
*/
@Embedded Period transferPeriod = Period.create(1, Unit.YEARS);
@Embedded
@AttributeOverrides({
@AttributeOverride(name = "unit", column = @Column(name = "transfer_renew_period_unit")),
@AttributeOverride(name = "value", column = @Column(name = "transfer_renew_period_value"))
})
Period transferPeriod = Period.create(1, Unit.YEARS);
/**
* The registration expiration time resulting from the approval - speculative or actual - of the
* most recent transfer request, applicable for domains only.
*
* <p>For pending transfers, this is the expiration time that will take effect under a projected
* server approval. For approved transfers, this is the actual expiration time of the domain as
* of the moment of transfer completion. For rejected or cancelled transfers, this field will be
* server approval. For approved transfers, this is the actual expiration time of the domain as of
* the moment of transfer completion. For rejected or cancelled transfers, this field will be
* reset to null.
*
* <p>Note that even when this field is set, it does not necessarily mean that the post-transfer
* domain has a new expiration time. Superuser transfers may not include a bundled 1 year renewal
* domain has a new expiration time. Superuser transfers may not include a bundled 1 year renewal
* at all, or even when a renewal is bundled, for a transfer during the autorenew grace period the
* bundled renewal simply subsumes the recent autorenewal, resulting in the same expiration time.
*/
// TODO(b/36405140): backfill this field for existing domains to which it should apply.
@Column(name = "transfer_registration_expiration_time")
DateTime transferredRegistrationExpirationTime;
/**
@@ -83,18 +102,35 @@ public class TransferData extends BaseTransferObject implements Buildable {
* pending transfer is explicitly approved, rejected or cancelled, the referenced entities should
* be deleted.
*/
@Transient
@IgnoreSave(IfNull.class)
@ElementCollection
Set<Key<? extends TransferServerApproveEntity>> serverApproveEntities;
Set<VKey<? extends TransferServerApproveEntity>> serverApproveEntities;
// The following 3 fields are the replacement for serverApproveEntities in Cloud SQL.
// TODO(shicong): Add getter/setter for these 3 fields and use them in the application code.
@Ignore
@Column(name = "transfer_gaining_poll_message_id")
Long gainingTransferPollMessageId;
@Ignore
@Column(name = "transfer_losing_poll_message_id")
Long losingTransferPollMessageId;
@Ignore
@Column(name = "transfer_billing_cancellation_id")
Long billingCancellationId;
/**
* The regular one-time billing event that will be charged for a server-approved transfer.
*
* <p>This field should be null if there is not currently a pending transfer or if the object
* being transferred is not a domain.
*
* <p>TODO(b/158230654) Remove unused columns for TransferData in Contact table.
*/
@IgnoreSave(IfNull.class)
Key<BillingEvent.OneTime> serverApproveBillingEvent;
@Column(name = "transfer_billing_event_id")
VKey<BillingEvent.OneTime> serverApproveBillingEvent;
/**
* The autorenew billing event that should be associated with this resource after the transfer.
@@ -103,7 +139,8 @@ public class TransferData extends BaseTransferObject implements Buildable {
* being transferred is not a domain.
*/
@IgnoreSave(IfNull.class)
Key<BillingEvent.Recurring> serverApproveAutorenewEvent;
@Column(name = "transfer_billing_recurrence_id")
VKey<BillingEvent.Recurring> serverApproveAutorenewEvent;
/**
* The autorenew poll message that should be associated with this resource after the transfer.
@@ -112,7 +149,8 @@ public class TransferData extends BaseTransferObject implements Buildable {
* being transferred is not a domain.
*/
@IgnoreSave(IfNull.class)
Key<PollMessage.Autorenew> serverApproveAutorenewPollMessage;
@Column(name = "transfer_autorenew_poll_message_id")
VKey<PollMessage.Autorenew> serverApproveAutorenewPollMessage;
@Nullable
public Trid getTransferRequestTrid() {
@@ -128,22 +166,22 @@ public class TransferData extends BaseTransferObject implements Buildable {
return transferredRegistrationExpirationTime;
}
public ImmutableSet<Key<? extends TransferServerApproveEntity>> getServerApproveEntities() {
public ImmutableSet<VKey<? extends TransferServerApproveEntity>> getServerApproveEntities() {
return nullToEmptyImmutableCopy(serverApproveEntities);
}
@Nullable
public Key<BillingEvent.OneTime> getServerApproveBillingEvent() {
public VKey<BillingEvent.OneTime> getServerApproveBillingEvent() {
return serverApproveBillingEvent;
}
@Nullable
public Key<BillingEvent.Recurring> getServerApproveAutorenewEvent() {
public VKey<BillingEvent.Recurring> getServerApproveAutorenewEvent() {
return serverApproveAutorenewEvent;
}
@Nullable
public Key<PollMessage.Autorenew> getServerApproveAutorenewPollMessage() {
public VKey<PollMessage.Autorenew> getServerApproveAutorenewPollMessage() {
return serverApproveAutorenewPollMessage;
}
@@ -202,25 +240,25 @@ public class TransferData extends BaseTransferObject implements Buildable {
}
public Builder setServerApproveEntities(
ImmutableSet<Key<? extends TransferServerApproveEntity>> serverApproveEntities) {
ImmutableSet<VKey<? extends TransferServerApproveEntity>> serverApproveEntities) {
getInstance().serverApproveEntities = serverApproveEntities;
return this;
}
public Builder setServerApproveBillingEvent(
Key<BillingEvent.OneTime> serverApproveBillingEvent) {
VKey<BillingEvent.OneTime> serverApproveBillingEvent) {
getInstance().serverApproveBillingEvent = serverApproveBillingEvent;
return this;
}
public Builder setServerApproveAutorenewEvent(
Key<BillingEvent.Recurring> serverApproveAutorenewEvent) {
VKey<BillingEvent.Recurring> serverApproveAutorenewEvent) {
getInstance().serverApproveAutorenewEvent = serverApproveAutorenewEvent;
return this;
}
public Builder setServerApproveAutorenewPollMessage(
Key<PollMessage.Autorenew> serverApproveAutorenewPollMessage) {
VKey<PollMessage.Autorenew> serverApproveAutorenewPollMessage) {
getInstance().serverApproveAutorenewPollMessage = serverApproveAutorenewPollMessage;
return this;
}
@@ -230,5 +268,7 @@ public class TransferData extends BaseTransferObject implements Buildable {
* Marker interface for objects that are written in anticipation of a server approval, and
* therefore need to be deleted under any other outcome.
*/
public interface TransferServerApproveEntity {}
public interface TransferServerApproveEntity {
VKey<? extends TransferServerApproveEntity> createVKey();
}
}
@@ -17,6 +17,7 @@ package google.registry.model.transfer;
import com.googlecode.objectify.annotation.Embed;
import google.registry.model.EppResource;
import google.registry.model.eppoutput.EppResponse.ResponseData;
import javax.persistence.Embeddable;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlTransient;
@@ -28,7 +29,8 @@ import org.joda.time.DateTime;
* are common to all transfer responses; derived classes add resource specific fields.
*/
@XmlTransient
public abstract class TransferResponse extends BaseTransferObject implements ResponseData {
@Embeddable
public class TransferResponse extends BaseTransferObject implements ResponseData {
/** An adapter to output the XML in response to a transfer command on a domain. */
@Embed
@@ -14,13 +14,14 @@
package google.registry.model.translators;
import static java.nio.charset.StandardCharsets.UTF_8;
import static com.google.common.base.Functions.identity;
import static com.google.common.collect.ImmutableMap.toImmutableMap;
import static google.registry.model.EntityClasses.ALL_CLASSES;
import com.googlecode.objectify.Key;
import com.google.appengine.api.datastore.Key;
import com.google.common.collect.ImmutableMap;
import com.googlecode.objectify.annotation.EntitySubclass;
import google.registry.persistence.VKey;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.net.URLEncoder;
/**
* Translator factory for VKey.
@@ -28,57 +29,39 @@ import java.net.URLEncoder;
* <p>These get translated to a string containing the URL safe encoding of the objectify key
* followed by a (url-unsafe) ampersand delimiter and the SQL key.
*/
public class VKeyTranslatorFactory<T> extends AbstractSimpleTranslatorFactory<VKey, String> {
private final Class<T> refClass;
public class VKeyTranslatorFactory extends AbstractSimpleTranslatorFactory<VKey, Key> {
public VKeyTranslatorFactory(Class<T> refClass) {
// Class registry allowing us to restore the original class object from the unqualified class
// name, which is all the datastore key gives us.
// Note that entities annotated with @EntitySubclass are removed because they share the same
// kind of the key with their parent class.
private static final ImmutableMap<String, Class> CLASS_REGISTRY =
ALL_CLASSES.stream()
.filter(clazz -> !clazz.isAnnotationPresent(EntitySubclass.class))
.collect(toImmutableMap(com.googlecode.objectify.Key::getKind, identity()));
;
public VKeyTranslatorFactory() {
super(VKey.class);
this.refClass = refClass;
}
@Override
public SimpleTranslator<VKey, String> createTranslator() {
return new SimpleTranslator<VKey, String>() {
public SimpleTranslator<VKey, Key> createTranslator() {
return new SimpleTranslator<VKey, Key>() {
@Override
public VKey loadValue(String datastoreValue) {
int pos = datastoreValue.indexOf('&');
Key ofyKey = null;
String sqlKey = null;
if (pos > 0) {
// We have an objectify key.
ofyKey = Key.create(datastoreValue.substring(0, pos));
}
if (pos < datastoreValue.length() - 1) {
// We have an SQL key.
sqlKey = decode(datastoreValue.substring(pos + 1));
}
return VKey.create(refClass, sqlKey, ofyKey);
public VKey loadValue(Key datastoreValue) {
// TODO(mmuller): we need to call a method on refClass to also reconstitute the SQL key.
return datastoreValue == null
? null
: VKey.createOfy(
CLASS_REGISTRY.get(datastoreValue.getKind()),
com.googlecode.objectify.Key.create(datastoreValue));
}
@Override
public String saveValue(VKey key) {
return ((key.getOfyKey() == null) ? "" : key.getOfyKey().getString())
+ "&"
+ ((key.getSqlKey() == null) ? "" : encode(key.getSqlKey().toString()));
public Key saveValue(VKey key) {
return key == null ? null : key.getOfyKey().getRaw();
}
};
}
private static String encode(String val) {
try {
return URLEncoder.encode(val, UTF_8.toString());
} catch (UnsupportedEncodingException e) {
throw new RuntimeException(e);
}
}
private static String decode(String encoded) {
try {
return URLDecoder.decode(encoded, UTF_8.toString());
} catch (UnsupportedEncodingException e) {
throw new RuntimeException(e);
}
}
}
@@ -17,6 +17,7 @@ package google.registry.persistence;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.googlecode.objectify.Key;
import google.registry.model.ImmutableObject;
import java.util.Optional;
@@ -57,6 +58,23 @@ public class VKey<T> extends ImmutableObject {
return new VKey(kind, ofyKey, null);
}
/**
* Creates a {@link VKey} which only contains the ofy primary key by specifying the id of the
* {@link Key}.
*/
public static <T> VKey<T> createOfy(Class<? extends T> kind, long id) {
return createOfy(kind, Key.create(kind, id));
}
/**
* Creates a {@link VKey} which only contains the ofy primary key by specifying the name of the
* {@link Key}.
*/
public static <T> VKey<T> createOfy(Class<? extends T> kind, String name) {
checkArgumentNotNull(kind, "name must not be null");
return createOfy(kind, Key.create(kind, name));
}
/** Creates a {@link VKey} which only contains both sql and ofy primary key. */
public static <T> VKey<T> create(
Class<? extends T> kind, Object sqlKey, com.googlecode.objectify.Key ofyKey) {
@@ -0,0 +1,47 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.persistence.converter;
import avro.shaded.com.google.common.collect.Maps;
import google.registry.model.registry.Registry.BillingCostTransition;
import java.util.Map;
import javax.persistence.Converter;
import org.joda.money.Money;
import org.joda.time.DateTime;
/**
* JPA converter for storing/retrieving {@code TimedTransitionProperty<Money,BillingCostTransition>}
* objects.
*/
@Converter(autoApply = true)
public class BillingCostTransitionConverter
extends TimedTransitionPropertyConverterBase<Money, BillingCostTransition> {
@Override
Map.Entry<String, String> convertToDatabaseMapEntry(
Map.Entry<DateTime, BillingCostTransition> entry) {
return Maps.immutableEntry(entry.getKey().toString(), entry.getValue().getValue().toString());
}
@Override
Map.Entry<DateTime, Money> convertToEntityMapEntry(Map.Entry<String, String> entry) {
return Maps.immutableEntry(DateTime.parse(entry.getKey()), Money.parse(entry.getValue()));
}
@Override
Class<BillingCostTransition> getTimedTransitionSubclass() {
return BillingCostTransition.class;
}
}
@@ -0,0 +1,35 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.persistence.converter;
import google.registry.model.billing.BillingEvent.Flag;
import java.util.Set;
import javax.persistence.AttributeConverter;
import javax.persistence.Converter;
/** JPA {@link AttributeConverter} for storing/retrieving {@link Set}. */
@Converter(autoApply = true)
public class BillingEventFlagSetConverter extends StringSetConverterBase<Flag> {
@Override
String toString(Flag element) {
return element.name();
}
@Override
Flag fromString(String value) {
return Flag.valueOf(value);
}
}
@@ -15,12 +15,11 @@
package google.registry.persistence.converter;
import google.registry.util.CidrAddressBlock;
import java.util.List;
import javax.persistence.AttributeConverter;
import javax.persistence.Converter;
/**
* JPA {@link AttributeConverter} for storing/retrieving {@link List<CidrAddressBlock>} objects.
* JPA {@link AttributeConverter} for storing/retrieving {@code List<CidrAddressBlock>} objects.
* TODO(shicong): Investigate if we can have one converter for any List type
*/
@Converter(autoApply = true)
@@ -21,7 +21,7 @@ import java.util.Map;
import javax.persistence.Converter;
import org.joda.money.CurrencyUnit;
/** JPA converter for storing/retrieving {@link Map <CurrencyUnit, BillingAccountEntry>} objects. */
/** JPA converter for storing/retrieving {@code Map<CurrencyUnit, BillingAccountEntry>} objects. */
@Converter(autoApply = true)
public class CurrencyToBillingConverter
extends StringMapConverterBase<CurrencyUnit, BillingAccountEntry> {
@@ -0,0 +1,33 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.persistence.converter;
import com.google.common.net.InetAddresses;
import java.net.InetAddress;
import javax.persistence.Converter;
@Converter(autoApply = true)
public class InetAddressSetConverter extends StringSetConverterBase<InetAddress> {
@Override
String toString(InetAddress element) {
return InetAddresses.toAddrString(element);
}
@Override
InetAddress fromString(String value) {
return InetAddresses.forString(value);
}
}
@@ -16,11 +16,10 @@ package google.registry.persistence.converter;
import google.registry.model.contact.Disclose.PostalInfoChoice;
import google.registry.model.contact.PostalInfo;
import java.util.List;
import javax.persistence.AttributeConverter;
import javax.persistence.Converter;
/** JPA {@link AttributeConverter} for storing/retrieving {@link List < PostalInfoChoice >}. */
/** JPA {@link AttributeConverter} for storing/retrieving {@code List<PostalInfoChoice>}. */
@Converter(autoApply = true)
public class PostalInfoChoiceListConverter extends StringListConverterBase<PostalInfoChoice> {
@@ -15,11 +15,10 @@
package google.registry.persistence.converter;
import google.registry.model.registrar.RegistrarContact.Type;
import java.util.Set;
import javax.persistence.AttributeConverter;
import javax.persistence.Converter;
/** JPA {@link AttributeConverter} for storing/retrieving {@link Set<Type>}. */
/** JPA {@link AttributeConverter} for storing/retrieving {@code Set<Type>}. */
@Converter(autoApply = true)
public class RegistrarPocSetConverter extends StringSetConverterBase<Type> {
@Override
@@ -15,11 +15,10 @@
package google.registry.persistence.converter;
import google.registry.model.eppcommon.StatusValue;
import java.util.Set;
import javax.persistence.AttributeConverter;
import javax.persistence.Converter;
/** JPA {@link AttributeConverter} for storing/retrieving {@link Set<StatusValue>}. */
/** JPA {@link AttributeConverter} for storing/retrieving {@code Set<StatusValue>}. */
@Converter(autoApply = true)
public class StatusValueSetConverter extends StringSetConverterBase<StatusValue> {
@@ -37,7 +37,7 @@ import org.hibernate.type.descriptor.sql.SqlTypeDescriptor;
/**
* The {@link JavaTypeDescriptor} and {@link SqlTypeDescriptor} for {@link StringCollection}.
*
* <p>A {@link StringCollection} object is a simple wrapper for a {@link Collection<String>} which
* <p>A {@link StringCollection} object is a simple wrapper for a {@code Collection<String>} which
* can be stored as a string array in the database. The {@link JavaTypeDescriptor} and {@link
* SqlTypeDescriptor} is used by JPA/Hibernate to map between the collection and {@link Array} which
* is the actual type that JDBC uses to read from and write to the database.
@@ -14,11 +14,10 @@
package google.registry.persistence.converter;
import java.util.List;
import javax.persistence.AttributeConverter;
import javax.persistence.Converter;
/** JPA {@link AttributeConverter} for storing/retrieving {@link List<String>}. */
/** JPA {@link AttributeConverter} for storing/retrieving {@code List<String>}. */
@Converter(autoApply = true)
public class StringListConverter extends StringListConverterBase<String> {
@@ -36,7 +36,7 @@ import org.hibernate.type.descriptor.sql.SqlTypeDescriptor;
/**
* The {@link JavaTypeDescriptor} and {@link SqlTypeDescriptor} for {@link StringMap}.
*
* <p>A {@link StringMap} object is a simple wrapper for a {@link Map <String, String>} which can be
* <p>A {@link StringMap} object is a simple wrapper for a {@code Map<String, String>} which can be
* stored in a column with data type of hstore in the database. The {@link JavaTypeDescriptor} and
* {@link SqlTypeDescriptor} is used by JPA/Hibernate to map between the map and hstore which is the
* actual type that JDBC uses to read from and write to the database.
@@ -156,7 +156,7 @@ public class StringMapDescriptor extends AbstractTypeDescriptor<StringMap>
};
}
/** A simple wrapper class for {@link Map<String, String>}. */
/** A simple wrapper class for {@code Map<String, String>}. */
public static class StringMap {
private Map<String, String> map;
@@ -169,7 +169,7 @@ public class StringMapDescriptor extends AbstractTypeDescriptor<StringMap>
return new StringMap(ImmutableMap.copyOf(map));
}
/** Returns the underlying {@link Map<String, String>} object. */
/** Returns the underlying {@code Map<String, String>} object. */
public Map<String, String> getMap() {
return map;
}
@@ -14,11 +14,10 @@
package google.registry.persistence.converter;
import java.util.Set;
import javax.persistence.AttributeConverter;
import javax.persistence.Converter;
/** JPA {@link AttributeConverter} for storing/retrieving {@link Set<String>}. */
/** JPA {@link AttributeConverter} for storing/retrieving {@code Set<String>}. */
@Converter(autoApply = true)
public class StringSetConverter extends StringSetConverterBase<String> {
@@ -0,0 +1,63 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.persistence.converter;
import static com.google.common.collect.ImmutableMap.toImmutableMap;
import com.google.common.collect.ImmutableSortedMap;
import google.registry.model.common.TimedTransitionProperty;
import google.registry.model.common.TimedTransitionProperty.TimedTransition;
import google.registry.persistence.converter.StringMapDescriptor.StringMap;
import java.util.Map;
import javax.annotation.Nullable;
import javax.persistence.AttributeConverter;
import org.joda.time.DateTime;
/**
* Base JPA converter for {@link TimedTransitionProperty} objects that are stored in a column with
* data type of hstore in the database.
*/
public abstract class TimedTransitionPropertyConverterBase<K, V extends TimedTransition<K>>
implements AttributeConverter<TimedTransitionProperty<K, V>, StringMap> {
abstract Map.Entry<String, String> convertToDatabaseMapEntry(Map.Entry<DateTime, V> entry);
abstract Map.Entry<DateTime, K> convertToEntityMapEntry(Map.Entry<String, String> entry);
abstract Class<V> getTimedTransitionSubclass();
@Override
public StringMap convertToDatabaseColumn(@Nullable TimedTransitionProperty<K, V> attribute) {
return attribute == null
? null
: StringMap.create(
attribute.entrySet().stream()
.map(this::convertToDatabaseMapEntry)
.collect(toImmutableMap(Map.Entry::getKey, Map.Entry::getValue)));
}
@Override
public TimedTransitionProperty<K, V> convertToEntityAttribute(@Nullable StringMap dbData) {
if (dbData == null) {
return null;
}
Map<DateTime, K> map =
dbData.getMap().entrySet().stream()
.map(this::convertToEntityMapEntry)
.collect(toImmutableMap(Map.Entry::getKey, Map.Entry::getValue));
return TimedTransitionProperty.fromValueMap(
ImmutableSortedMap.copyOf(map), getTimedTransitionSubclass());
}
}
@@ -0,0 +1,47 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.persistence.converter;
import com.google.common.collect.Maps;
import google.registry.model.registry.Registry.TldState;
import google.registry.model.registry.Registry.TldStateTransition;
import java.util.Map;
import javax.persistence.Converter;
import org.joda.time.DateTime;
/**
* JPA converter for storing/retrieving {@code TimedTransitionProperty<TldState,
* TldStateTransition>} objects.
*/
@Converter(autoApply = true)
public class TldStateTransitionConverter
extends TimedTransitionPropertyConverterBase<TldState, TldStateTransition> {
@Override
Map.Entry<String, String> convertToDatabaseMapEntry(
Map.Entry<DateTime, TldStateTransition> entry) {
return Maps.immutableEntry(entry.getKey().toString(), entry.getValue().getValue().name());
}
@Override
Map.Entry<DateTime, TldState> convertToEntityMapEntry(Map.Entry<String, String> entry) {
return Maps.immutableEntry(DateTime.parse(entry.getKey()), TldState.valueOf(entry.getValue()));
}
@Override
Class<TldStateTransition> getTimedTransitionSubclass() {
return TldStateTransition.class;
}
}
@@ -0,0 +1,37 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.persistence.converter;
import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
import google.registry.persistence.VKey;
import java.util.Set;
import javax.persistence.AttributeConverter;
import javax.persistence.Converter;
/** {@link AttributeConverter} for {@link Set}. */
@Converter(autoApply = true)
public class TransferServerApproveEntitySetConverter
extends StringSetConverterBase<VKey<? extends TransferServerApproveEntity>> {
@Override
String toString(VKey<? extends TransferServerApproveEntity> element) {
return String.valueOf(element.getSqlKey());
}
@Override
VKey<? extends TransferServerApproveEntity> fromString(String value) {
return VKey.createSql(TransferServerApproveEntity.class, Long.parseLong(value));
}
}
@@ -278,7 +278,7 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
.getResultList());
}
private <T> int internalDelete(VKey<T> key) {
private int internalDelete(VKey<?> key) {
checkArgumentNotNull(key, "key must be specified");
assertInTransaction();
EntityType<?> entityType = getEntityType(key.getKind());
@@ -291,10 +291,16 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
}
@Override
public <T> void delete(VKey<T> key) {
public void delete(VKey<?> key) {
internalDelete(key);
}
@Override
public void delete(Iterable<? extends VKey<?>> vKeys) {
checkArgumentNotNull(vKeys, "vKeys must be specified");
vKeys.forEach(this::internalDelete);
}
@Override
public <T> void assertDelete(VKey<T> key) {
if (internalDelete(key) != 1) {
@@ -17,6 +17,7 @@ package google.registry.persistence.transaction;
import com.google.common.collect.ImmutableCollection;
import com.google.common.collect.ImmutableList;
import google.registry.persistence.VKey;
import java.util.NoSuchElementException;
import java.util.Optional;
import java.util.function.Supplier;
import org.joda.time.DateTime;
@@ -114,7 +115,7 @@ public interface TransactionManager {
<T> T load(VKey<T> key);
/**
* Leads the set of entities by their key id.
* Loads the set of entities by their key id.
*
* @throws NoSuchElementException if any of the keys are not found.
*/
@@ -124,5 +125,8 @@ public interface TransactionManager {
<T> ImmutableList<T> loadAll(Class<T> clazz);
/** Deletes the entity by its id. */
<T> void delete(VKey<T> key);
void delete(VKey<?> key);
/** Deletes the set of entities by their key id. */
void delete(Iterable<? extends VKey<?>> keys);
}
@@ -16,6 +16,7 @@ package google.registry.persistence.transaction;
import com.google.appengine.api.utils.SystemProperty;
import com.google.appengine.api.utils.SystemProperty.Environment.Value;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Suppliers;
import google.registry.model.ofy.DatastoreTransactionManager;
import google.registry.persistence.DaggerPersistenceComponent;
@@ -26,7 +27,9 @@ import java.util.function.Supplier;
// TODO: Rename this to PersistenceFactory and move to persistence package.
public class TransactionManagerFactory {
private static final TransactionManager TM = createTransactionManager();
private static final DatastoreTransactionManager ofyTm = createTransactionManager();
@NonFinalForTesting private static TransactionManager tm = ofyTm;
/** Supplier for jpaTm so that it is initialized only once, upon first usage. */
@NonFinalForTesting
@@ -45,10 +48,7 @@ public class TransactionManagerFactory {
}
}
private static TransactionManager createTransactionManager() {
// TODO: Determine how to provision TransactionManager after the dual-write. During the
// dual-write transitional phase, we need the TransactionManager for both Datastore and Cloud
// SQL, and this method returns the one for Datastore.
private static DatastoreTransactionManager createTransactionManager() {
return new DatastoreTransactionManager(null);
}
@@ -67,7 +67,7 @@ public class TransactionManagerFactory {
/** Returns {@link TransactionManager} instance. */
public static TransactionManager tm() {
return TM;
return tm;
}
/** Returns {@link JpaTransactionManager} instance. */
@@ -75,8 +75,20 @@ public class TransactionManagerFactory {
return jpaTm.get();
}
/** Returns {@link DatastoreTransactionManager} instance. */
@VisibleForTesting
public static DatastoreTransactionManager ofyTm() {
return ofyTm;
}
/** Sets the return of {@link #jpaTm()} to the given instance of {@link JpaTransactionManager}. */
public static void setJpaTm(JpaTransactionManager newJpaTm) {
jpaTm = Suppliers.ofInstance(newJpaTm);
}
/** Sets the return of {@link #tm()} to the given instance of {@link TransactionManager}. */
@VisibleForTesting
public static void setTm(TransactionManager newTm) {
tm = newTm;
}
}
@@ -14,6 +14,7 @@
package google.registry.rdap;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.model.EppResourceUtils.loadByForeignKey;
import static google.registry.model.index.ForeignKeyIndex.loadAndGetKey;
import static google.registry.model.ofy.ObjectifyService.ofy;
@@ -28,10 +29,10 @@ import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import com.google.common.net.InetAddresses;
import com.google.common.primitives.Booleans;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.cmd.Query;
import google.registry.model.domain.DomainBase;
import google.registry.model.host.HostResource;
import google.registry.persistence.VKey;
import google.registry.rdap.RdapJsonFormatter.OutputDataType;
import google.registry.rdap.RdapMetrics.EndpointType;
import google.registry.rdap.RdapMetrics.SearchType;
@@ -50,6 +51,7 @@ import java.util.Comparator;
import java.util.List;
import java.util.Optional;
import java.util.stream.Stream;
import java.util.stream.StreamSupport;
import javax.inject.Inject;
/**
@@ -61,10 +63,9 @@ import javax.inject.Inject;
* (RDAP) Query Format</a>
* @see <a href="http://tools.ietf.org/html/rfc7483">RFC 7483: JSON Responses for the Registration
* Data Access Protocol (RDAP)</a>
*
* TODO(guyben):This isn't required by the RDAP Technical Implementation Guide, and hence should be
* deleted, at least until it's actually required.
*/
// TODO: This isn't required by the RDAP Technical Implementation Guide, and hence should be
// deleted, at least until it's actually required.
@Action(
service = Action.Service.PUBAPI,
path = "/rdap/domains",
@@ -244,7 +245,7 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
*/
private DomainSearchResponse searchByNameserverLdhName(
final RdapSearchPattern partialStringQuery) {
Iterable<Key<HostResource>> hostKeys = getNameserverRefsByLdhName(partialStringQuery);
Iterable<VKey<HostResource>> hostKeys = getNameserverRefsByLdhName(partialStringQuery);
if (Iterables.isEmpty(hostKeys)) {
metricInformationBuilder.setNumHostsRetrieved(0);
throw new NotFoundException("No matching nameservers found");
@@ -262,7 +263,7 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
* initial string is not required (e.g. "*.example.tld" is valid), because we can look up the
* domain and just list all of its subordinate hosts.
*/
private Iterable<Key<HostResource>> getNameserverRefsByLdhName(
private Iterable<VKey<HostResource>> getNameserverRefsByLdhName(
final RdapSearchPattern partialStringQuery) {
// Handle queries without a wildcard.
if (!partialStringQuery.getHasWildcard()) {
@@ -293,11 +294,13 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
if (desiredRegistrar.isPresent()) {
query = query.filter("currentSponsorClientId", desiredRegistrar.get());
}
return query.keys();
return StreamSupport.stream(query.keys().spliterator(), false)
.map(key -> VKey.createOfy(HostResource.class, key))
.collect(toImmutableSet());
}
/** Assembles a list of {@link HostResource} keys by name when the pattern has no wildcard. */
private Iterable<Key<HostResource>> getNameserverRefsByLdhNameWithoutWildcard(
private Iterable<VKey<HostResource>> getNameserverRefsByLdhNameWithoutWildcard(
final RdapSearchPattern partialStringQuery) {
// If we need to check the sponsoring registrar, we need to load the resource rather than just
// the key.
@@ -311,9 +314,9 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
return (!host.isPresent()
|| !desiredRegistrar.get().equals(host.get().getPersistedCurrentSponsorClientId()))
? ImmutableList.of()
: ImmutableList.of(Key.create(host.get()));
: ImmutableList.of(host.get().createVKey());
} else {
Key<HostResource> hostKey =
VKey<HostResource> hostKey =
loadAndGetKey(
HostResource.class,
partialStringQuery.getInitialString(),
@@ -323,7 +326,7 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
}
/** Assembles a list of {@link HostResource} keys by name using a superordinate domain suffix. */
private Iterable<Key<HostResource>> getNameserverRefsByLdhNameWithSuffix(
private Iterable<VKey<HostResource>> getNameserverRefsByLdhNameWithSuffix(
final RdapSearchPattern partialStringQuery) {
// The suffix must be a domain that we manage. That way, we can look up the domain and search
// through the subordinate hosts. This is more efficient, and lets us permit wildcard searches
@@ -339,7 +342,7 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
"A suffix in a lookup by nameserver name "
+ "must be a domain defined in the system"));
Optional<String> desiredRegistrar = getDesiredRegistrar();
ImmutableList.Builder<Key<HostResource>> builder = new ImmutableList.Builder<>();
ImmutableList.Builder<VKey<HostResource>> builder = new ImmutableList.Builder<>();
for (String fqhn : ImmutableSortedSet.copyOf(domainBase.getSubordinateHosts())) {
// We can't just check that the host name starts with the initial query string, because
// then the query ns.exam*.example.com would match against nameserver ns.example.com.
@@ -352,10 +355,10 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
shouldIncludeDeleted() ? START_OF_TIME : getRequestTime());
if (host.isPresent()
&& desiredRegistrar.get().equals(host.get().getPersistedCurrentSponsorClientId())) {
builder.add(Key.create(host.get()));
builder.add(host.get().createVKey());
}
} else {
Key<HostResource> hostKey =
VKey<HostResource> hostKey =
loadAndGetKey(
HostResource.class,
fqhn,
@@ -401,7 +404,10 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
if (desiredRegistrar.isPresent()) {
query = query.filter("currentSponsorClientId", desiredRegistrar.get());
}
return searchByNameserverRefs(query.keys());
return searchByNameserverRefs(
StreamSupport.stream(query.keys().spliterator(), false)
.map(key -> VKey.createOfy(HostResource.class, key))
.collect(toImmutableSet()));
}
/**
@@ -410,7 +416,7 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
* <p>This method is called by {@link #searchByNameserverLdhName} and {@link
* #searchByNameserverIp} after they assemble the relevant host keys.
*/
private DomainSearchResponse searchByNameserverRefs(final Iterable<Key<HostResource>> hostKeys) {
private DomainSearchResponse searchByNameserverRefs(final Iterable<VKey<HostResource>> hostKeys) {
// We must break the query up into chunks, because the in operator is limited to 30 subqueries.
// Since it is possible for the same domain to show up more than once in our result list (if
// we do a wildcard nameserver search that returns multiple nameservers used by the same
@@ -421,11 +427,13 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
ImmutableSortedSet.orderedBy(
Comparator.comparing(DomainBase::getFullyQualifiedDomainName));
int numHostKeysSearched = 0;
for (List<Key<HostResource>> chunk : Iterables.partition(hostKeys, 30)) {
for (List<VKey<HostResource>> chunk : Iterables.partition(hostKeys, 30)) {
numHostKeysSearched += chunk.size();
Query<DomainBase> query = ofy().load()
.type(DomainBase.class)
.filter("nsHosts in", chunk);
Query<DomainBase> query =
ofy()
.load()
.type(DomainBase.class)
.filter("nsHosts in", chunk.stream().map(VKey::getOfyKey).collect(toImmutableSet()));
if (!shouldIncludeDeleted()) {
query = query.filter("deletionTime >", getRequestTime());
// If we are not performing an inequality query, we can filter on the cursor in the query.
@@ -72,10 +72,9 @@ import javax.inject.Inject;
* (RDAP) Query Format</a>
* @see <a href="http://tools.ietf.org/html/rfc7483">RFC 7483: JSON Responses for the Registration
* Data Access Protocol (RDAP)</a>
*
* TODO(guyben):This isn't required by the RDAP Technical Implementation Guide, and hence should be
* deleted, at least until it's actually required.
*/
// TODO: This isn't required by the RDAP Technical Implementation Guide, and hence should be
// deleted, at least until it's actually required.
@Action(
service = Action.Service.PUBAPI,
path = "/rdap/entities",
@@ -433,10 +433,7 @@ public class RdapJsonFormatter {
statuses.add(StatusValue.LINKED);
}
if (hostResource.isSubordinate()
&& ofy()
.load()
.key(hostResource.getSuperordinateDomain())
.now()
&& tm().load(hostResource.getSuperordinateDomain())
.cloneProjectedAtTime(getRequestTime())
.getStatusValues()
.contains(StatusValue.PENDING_TRANSFER)) {
@@ -221,10 +221,9 @@ public abstract class RdapSearchActionBase extends RdapActionBase {
*
* <p>This version handles a list of parameter values, all associated with the same name.
*
* <p>Example: If the original parameters were "a=w&a=x&b=y&c=z", and this method is called with
* parameterName = "b" and parameterValues of "p" and "q", the result will be
* "a=w&a=x&c=z&b=p&b=q". The new values of parameter "b" replace the old ones.
*
* <p>Example: If the original parameters were "a=w&amp;a=x&amp;b=y&amp;c=z", and this method is
* called with parameterName = "b" and parameterValues of "p" and "q", the result will be
* "a=w&amp;a=x&amp;c=z&amp;b=p&amp;b=q". The new values of parameter "b" replace the old ones.
*/
protected String getRequestUrlWithExtraParameter(
String parameterName, List<String> parameterValues) {
@@ -120,9 +120,6 @@ public final class Ghostryde {
/**
* Creates a ghostryde file from an in-memory byte array.
*
* @throws PGPException
* @throws IOException
*/
public static byte[] encode(byte[] data, PGPPublicKey key)
throws IOException, PGPException {
@@ -137,9 +134,6 @@ public final class Ghostryde {
/**
* Deciphers a ghostryde file from an in-memory byte array.
*
* @throws PGPException
* @throws IOException
*/
public static byte[] decode(byte[] data, PGPPrivateKey key)
throws IOException, PGPException {
@@ -17,7 +17,6 @@ package google.registry.rde;
import static com.google.common.base.Preconditions.checkArgument;
import com.google.common.net.InetAddresses;
import com.googlecode.objectify.Key;
import google.registry.model.domain.DomainBase;
import google.registry.model.eppcommon.StatusValue;
import google.registry.model.host.HostResource;
@@ -35,9 +34,8 @@ import org.joda.time.DateTime;
final class HostResourceToXjcConverter {
/** Converts a subordinate {@link HostResource} to {@link XjcRdeHostElement}. */
static XjcRdeHostElement convertSubordinate(
HostResource host, DomainBase superordinateDomain) {
checkArgument(Key.create(superordinateDomain).equals(host.getSuperordinateDomain()));
static XjcRdeHostElement convertSubordinate(HostResource host, DomainBase superordinateDomain) {
checkArgument(superordinateDomain.createVKey().equals(host.getSuperordinateDomain()));
return new XjcRdeHostElement(convertSubordinateHost(host, superordinateDomain));
}

Some files were not shown because too many files have changed in this diff Show More