1
0
mirror of https://github.com/google/nomulus synced 2026-07-06 00:04:50 +00:00

Compare commits

...

11 Commits

Author SHA1 Message Date
gbrodman 0777be3d6c Allow superuser ext to override client/server transfer prohibited (#2890)
The superuser can remove/add those statuses anyway, so there's not
really any point. This also saves us trouble if we need to do a BTAPPA
transfer.
2025-12-05 20:22:15 +00:00
Weimin Yu f9cd167ae4 Copy artifacts for schema tests after deployment (#2895)
After each deployment in sandbox or production, move the artifacts from
the corresponding release to a well-known location so that they can be
mapped to Kokoro in presubmit tests. The Kokoro-mapping does not need
public access to the GCS bucket.

The artifacts include the  postgresql schema jar, the nomulus release
jar, and the uber jar of the nomulus schema integration test classes.

Every jar name consists of a fixed prefix and the environment. Each jar
of a new deployment overrides the previous copy.
2025-12-04 20:55:19 +00:00
sharma1210 eed1886121 Implement rdap_query command (#2886)
* Implement rdap_query command

* modifying and correcting issues

* modifying and correcting issues

* modifying and correcting issues

* resolving comments

* resolving comments

* resolving comments

* resolving comments

* resolving comments

* modifying and correcting issues

* resolving comments

* resolving comments

* resolving comments

* modifying and correcting issues

* modifying and correcting issues

* modifying and correcting issues

* resolving comments

* modifying and correcting issues

* resolving comments

* Fixing Deduplication in test

* Fixing Deduplication in test

* resolving comments
2025-12-01 20:45:57 +00:00
gbrodman 7149fd3307 Remove more references to GAE (#2894)
These are old/pointless now that we've migrated to GKE. Note that this
doesn't update anything in the docs/ folder, as that's a much larger
project that should be done on its own.
2025-12-01 16:43:49 +00:00
Weimin Yu 0dc7ab99d7 Update CreateCdnsTld command for RST Tests (#2891)
Add a flag indicating that a Sandbox TLD should use the production
servers.

No additional TLD name pattern checks. Cloud DNS has an allowlist for
names that may use production servers.

Also updated default descriptive name generation: dropping the trailing
'.', and replacing remaining dots with '_'.
2025-11-25 19:41:44 +00:00
Ben McIlwain 76d4dfbb04 Add "augmented_latin.txt" IDN table in existing txt table format (#2884)
This contains the same codepoints from the
core/src/main/java/google/registry/idn/Latin-IDN.xml file, just in the old .txt
IDN format that Nomulus actually ingests.
2025-11-24 21:26:05 +00:00
gbrodman 8547ad7941 Remove the concept of a GAE service endpoint (#2869)
We don't need to support the mix of GAE and GKE any more so we can get
rid of the GaeService bits and unify everything under one constant
service. This also allows us to reduce the number of services down to
four (FE, BE, PUBAPI, console) which is nice.
2025-11-18 19:31:40 +00:00
gbrodman b1266c95e8 Add and default to Argon2 hashing (#2877)
We've previously been using Scrypt since PR #2191 which, while being a
memory-hard slow function, isn't the optimal solution according to the
OWASP recommendations. While we could get away with increasing the
parallelization parameter to 3, it's better to just switch to the
most-recommended solution if we're switching things up anyway.

For the transition, we do something similar to PR #2191 where if the
previous-algorithm's hash is successful, we re-hash with Argo2id and
store that version. By doing this, we should not need any intervention
for registrars who log in at any point during the transition period.

Much of this PR, especially the parts where we re-hash the passwords in
Argon2 instead of Scrypt upon login, is based on the code that was
eventually removed in #2310.
2025-11-17 20:11:22 +00:00
Weimin Yu bc9aab6790 Reformat Fee extension v1.0 schema (#2888)
Reformat the current schema file for RFC 8748 final version. This was
adapted from v0.12 is not fully consistent with the final schema

This helps highlight the differences we missed in PR 2855 when we check
in the official schema.
2025-11-17 15:58:56 +00:00
Ben McIlwain 6cb669a5a7 Remove Tld table field allowed_registrant_contact_ids (#2871)
This is a follow-up to PR #2867, requiring merging/deployment in a subsequent release.

BUG= http://b/448619572
2025-11-14 21:32:36 +00:00
Weimin Yu 0f92e98028 Disable Fee version 1.0 (#2887)
The v1.0 support added in PR 2855 is buggy. Disable it for now.
2025-11-14 20:32:45 +00:00
202 changed files with 1186 additions and 3224 deletions
+2 -2
View File
@@ -84,10 +84,10 @@ tasks.build.dependsOn(tasks.checkLicense)
// Paths to main and test sources.
ext.projectRootDir = "${rootDir}"
// Tasks to deploy/stage all App Engine services
// Tasks to deploy/stage all services
task deploy {
group = 'deployment'
description = 'Deploys all services to App Engine.'
description = 'Deploys all services.'
}
task stage {
@@ -33,8 +33,8 @@ public abstract class DateTimeUtils {
/**
* A date in the far future that we can treat as infinity.
*
* <p>This value is (2^63-1)/1000 rounded down. AppEngine stores dates as 64 bit microseconds, but
* Java uses milliseconds, so this is the largest representable date that will survive a
* <p>This value is (2^63-1)/1000 rounded down. Postgres can store dates as 64 bit microseconds,
* but Java uses milliseconds, so this is the largest representable date that will survive a
* round-trip through the database.
*/
public static final DateTime END_OF_TIME = new DateTime(Long.MAX_VALUE / 1000, DateTimeZone.UTC);
+1 -1
View File
@@ -104,7 +104,7 @@ PROPERTIES = [
Property('testFilter',
'Comma separated list of test patterns, if specified run only '
'these.'),
Property('environment', 'GAE Environment for deployment and staging.'),
Property('environment', 'Environment for deployment and staging.'),
# Cloud SQL properties
Property('dbServer',
+1 -1
View File
@@ -9,7 +9,7 @@ expected to change.
## Deployment
Webapp is deployed with the nomulus default service war to Google App Engine.
The webapp is deployed with the nomulus default service war to GKE.
During nomulus default service war build task, gradle script triggers the
following:
+1 -1
View File
@@ -110,7 +110,7 @@ configurations {
// for details.
nomulus_test
// Exclude non-canonical servlet-api jars. Our AppEngine deployment uses
// Exclude non-canonical servlet-api jars. Our deployment uses
// javax.servlet:servlet-api:2.5
// For reasons we do not understand, marking the following dependencies as
// compileOnly instead of compile does not exclude them from runtimeClasspath.
@@ -20,7 +20,6 @@ import static java.nio.charset.StandardCharsets.UTF_8;
import com.google.common.flogger.FluentLogger;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.UrlConnectionService;
@@ -43,7 +42,7 @@ import javax.net.ssl.HttpsURLConnection;
* --service BACKEND -X POST -u '/_dr/task/executeCannedScript}'}
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = "/_dr/task/executeCannedScript",
method = {POST, GET},
automaticallyPrintOk = true,
@@ -27,7 +27,6 @@ import google.registry.model.domain.token.AllocationToken;
import google.registry.model.domain.token.BulkPricingPackage;
import google.registry.model.registrar.Registrar;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.auth.Auth;
import google.registry.ui.server.SendEmailUtils;
import google.registry.util.Clock;
@@ -39,7 +38,10 @@ import org.joda.time.Days;
* An action that checks all {@link BulkPricingPackage} objects for compliance with their max create
* limit.
*/
@Action(service = GaeService.BACKEND, path = CheckBulkComplianceAction.PATH, auth = Auth.AUTH_ADMIN)
@Action(
service = Action.Service.BACKEND,
path = CheckBulkComplianceAction.PATH,
auth = Auth.AUTH_ADMIN)
public class CheckBulkComplianceAction implements Runnable {
public static final String PATH = "/_dr/task/checkBulkCompliance";
@@ -43,7 +43,6 @@ import google.registry.config.CredentialModule.ApplicationDefaultCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.request.Action;
import google.registry.request.Action.Method;
import google.registry.request.Action.Service;
import google.registry.util.Clock;
import google.registry.util.CollectionUtils;
import google.registry.util.GoogleCredentialsBundle;
@@ -56,8 +55,6 @@ import java.nio.charset.StandardCharsets;
import java.util.Arrays;
import java.util.Optional;
import java.util.Random;
import java.util.function.BiConsumer;
import java.util.function.Consumer;
import java.util.function.Supplier;
import org.joda.time.Duration;
@@ -119,19 +116,13 @@ public class CloudTasksUtils implements Serializable {
* <p>For GET requests we add them on to the URL, and for POST requests we add them in the body of
* the request.
*
* <p>The parameters {@code putHeadersFunction} and {@code setBodyFunction} are used so that this
* method can be called with either an AppEngine HTTP request or a standard non-AppEngine HTTP
* request. The two objects do not have the same methods, but both have ways of setting headers /
* body.
*
* @return the resulting path (unchanged for POST requests, with params added for GET requests)
*/
private static String processRequestParameters(
String path,
Method method,
Multimap<String, String> params,
BiConsumer<String, String> putHeadersFunction,
Consumer<ByteString> setBodyFunction) {
HttpRequest.Builder requestBuilder) {
if (CollectionUtils.isNullOrEmpty(params)) {
return path;
}
@@ -149,8 +140,8 @@ public class CloudTasksUtils implements Serializable {
if (method.equals(Method.GET)) {
return String.format("%s?%s", path, encodedParams);
}
putHeadersFunction.accept(HttpHeaders.CONTENT_TYPE, MediaType.FORM_DATA.toString());
setBodyFunction.accept(ByteString.copyFrom(encodedParams, StandardCharsets.UTF_8));
requestBuilder.putHeaders(HttpHeaders.CONTENT_TYPE, MediaType.FORM_DATA.toString());
requestBuilder.setBody(ByteString.copyFrom(encodedParams, StandardCharsets.UTF_8));
return path;
}
@@ -161,29 +152,26 @@ public class CloudTasksUtils implements Serializable {
* default service account as the principal. That account must have permission to submit tasks to
* Cloud Tasks.
*
* <p>The caller of this method is responsible for passing in the appropriate service based on the
* runtime (GAE/GKE). Use the overload that takes an action class if possible.
* <p>The caller of this method is responsible for passing in the appropriate service. Use the
* overload that takes an action class if possible.
*
* @param path the relative URI (staring with a slash and ending without one).
* @param method the HTTP method to be used for the request.
* @param service the GAE/GKE service to route the request to.
* @param service the service to route the request to.
* @param params a multimap of URL query parameters. Duplicate keys are saved as is, and it is up
* to the server to process the duplicate keys.
* @return the enqueued task.
* @see <a
* href=ttps://cloud.google.com/appengine/docs/standard/java/taskqueue/push/creating-tasks#target>Specifyinig
* the worker service</a>
* @see <a href=https://docs.cloud.google.com/tasks/docs/creating-http-target-tasks#java>Creating
* HTTP target tasks</a>
*/
protected Task createTask(
String path, Method method, Service service, Multimap<String, String> params) {
String path, Method method, Action.Service service, Multimap<String, String> params) {
checkArgument(
path != null && !path.isEmpty() && path.charAt(0) == '/',
"The path must start with a '/'.");
HttpRequest.Builder requestBuilder =
HttpRequest.newBuilder().setHttpMethod(HttpMethod.valueOf(method.name()));
path =
processRequestParameters(
path, method, params, requestBuilder::putHeaders, requestBuilder::setBody);
path = processRequestParameters(path, method, params, requestBuilder);
OidcToken.Builder oidcTokenBuilder =
OidcToken.newBuilder()
.setServiceAccountEmail(credential.serviceAccount())
@@ -205,16 +193,15 @@ public class CloudTasksUtils implements Serializable {
* Cloud Tasks.
*
* <p>Prefer this overload over the one where the path and service are explicitly defined, as this
* class will automatically determine the service to use based on the action and the runtime.
* class will automatically determine the service to use based on the action.
*
* @param actionClazz the action class to run, must be annotated with {@link Action}.
* @param method the HTTP method to be used for the request.
* @param params a multimap of URL query parameters. Duplicate keys are saved as is, and it is up
* to the server to process the duplicate keys.
* @return the enqueued task.
* @see <a
* href=ttps://cloud.google.com/appengine/docs/standard/java/taskqueue/push/creating-tasks#target>Specifyinig
* the worker service</a>
* @see <a href=https://docs.cloud.google.com/tasks/docs/creating-http-target-tasks#java>Creating
* HTTP target tasks</a>
*/
public Task createTask(
Class<? extends Runnable> actionClazz, Method method, Multimap<String, String> params) {
@@ -231,32 +218,29 @@ public class CloudTasksUtils implements Serializable {
method,
actionClazz.getSimpleName(),
allowedMethods);
Service service =
RegistryEnvironment.isOnJetty() ? Action.ServiceGetter.get(action) : action.service();
return createTask(path, method, service, params);
return createTask(path, method, action.service(), params);
}
/**
* Create a {@link Task} to be enqueued with a random delay up to {@code jitterSeconds}.
*
* <p>The caller of this method is responsible for passing in the appropriate service based on the
* runtime (GAE/GKE). Use the overload that takes an action class if possible.
* <p>The caller of this method is responsible for passing in the appropriate service. Use the
* overload that takes an action class if possible.
*
* @param path the relative URI (staring with a slash and ending without one).
* @param method the HTTP method to be used for the request.
* @param service the GAE/GKE service to route the request to.
* @param service the service to route the request to.
* @param params a multimap of URL query parameters. Duplicate keys are saved as is, and it is up
* to the server to process the duplicate keys.
* @param jitterSeconds the number of seconds that a task is randomly delayed up to.
* @return the enqueued task.
* @see <a
* href=ttps://cloud.google.com/appengine/docs/standard/java/taskqueue/push/creating-tasks#target>Specifyinig
* the worker service</a>
* @see <a href=https://docs.cloud.google.com/tasks/docs/creating-http-target-tasks#java>Creating
* HTTP target tasks</a>
*/
public Task createTaskWithJitter(
String path,
Method method,
Service service,
Action.Service service,
Multimap<String, String> params,
Optional<Integer> jitterSeconds) {
if (jitterSeconds.isEmpty() || jitterSeconds.get() <= 0) {
@@ -274,7 +258,7 @@ public class CloudTasksUtils implements Serializable {
* Create a {@link Task} to be enqueued with a random delay up to {@code jitterSeconds}.
*
* <p>Prefer this overload over the one where the path and service are explicitly defined, as this
* class will automatically determine the service to use based on the action and the runtime.
* class will automatically determine the service to use based on the action.
*
* @param actionClazz the action class to run, must be annotated with {@link Action}.
* @param method the HTTP method to be used for the request.
@@ -282,9 +266,8 @@ public class CloudTasksUtils implements Serializable {
* to the server to process the duplicate keys.
* @param jitterSeconds the number of seconds that a task is randomly delayed up to.
* @return the enqueued task.
* @see <a
* href=ttps://cloud.google.com/appengine/docs/standard/java/taskqueue/push/creating-tasks#target>Specifyinig
* the worker service</a>
* @see <a href=https://docs.cloud.google.com/tasks/docs/creating-http-target-tasks#java>Creating
* HTTP target tasks</a>
*/
public Task createTaskWithJitter(
Class<? extends Runnable> actionClazz,
@@ -297,9 +280,7 @@ public class CloudTasksUtils implements Serializable {
"Action class %s is not annotated with @Action",
actionClazz.getSimpleName());
String path = action.path();
Service service =
RegistryEnvironment.isOnJetty() ? Action.ServiceGetter.get(action) : action.service();
return createTaskWithJitter(path, method, service, params, jitterSeconds);
return createTaskWithJitter(path, method, action.service(), params, jitterSeconds);
}
/**
@@ -307,19 +288,18 @@ public class CloudTasksUtils implements Serializable {
*
* @param path the relative URI (staring with a slash and ending without one).
* @param method the HTTP method to be used for the request.
* @param service the GAE/GKE service to route the request to.
* @param service the service to route the request to.
* @param params a multimap of URL query parameters. Duplicate keys are saved as is, and it is up
* to the server to process the duplicate keys.
* @param delay the amount of time that a task needs to be delayed for.
* @return the enqueued task.
* @see <a
* href=ttps://cloud.google.com/appengine/docs/standard/java/taskqueue/push/creating-tasks#target>Specifyinig
* the worker service</a>
* @see <a href=https://docs.cloud.google.com/tasks/docs/creating-http-target-tasks#java>Creating
* HTTP target tasks</a>
*/
private Task createTaskWithDelay(
String path,
Method method,
Service service,
Action.Service service,
Multimap<String, String> params,
Duration delay) {
if (delay.isEqual(Duration.ZERO)) {
@@ -335,7 +315,7 @@ public class CloudTasksUtils implements Serializable {
* Create a {@link Task} to be enqueued with delay of {@code duration}.
*
* <p>Prefer this overload over the one where the path and service are explicitly defined, as this
* class will automatically determine the service to use based on the action and the runtime.
* class will automatically determine the service to use based on the action.
*
* @param actionClazz the action class to run, must be annotated with {@link Action}.
* @param method the HTTP method to be used for the request.
@@ -343,9 +323,8 @@ public class CloudTasksUtils implements Serializable {
* to the server to process the duplicate keys.
* @param delay the amount of time that a task needs to be delayed for.
* @return the enqueued task.
* @see <a
* href=ttps://cloud.google.com/appengine/docs/standard/java/taskqueue/push/creating-tasks#target>Specifyinig
* the worker service</a>
* @see <a href=https://docs.cloud.google.com/tasks/docs/creating-http-target-tasks#java>Creating
* HTTP target tasks</a>
*/
public Task createTaskWithDelay(
Class<? extends Runnable> actionClazz,
@@ -354,9 +333,7 @@ public class CloudTasksUtils implements Serializable {
Duration delay) {
Action action = getAction(actionClazz);
String path = action.path();
Service service =
RegistryEnvironment.isOnJetty() ? Action.ServiceGetter.get(action) : action.service();
return createTaskWithDelay(path, method, service, params, delay);
return createTaskWithDelay(path, method, action.service(), params, delay);
}
private static Action getAction(Class<? extends Runnable> actionClazz) {
@@ -37,7 +37,6 @@ import google.registry.model.eppcommon.ProtocolDefinition;
import google.registry.model.eppoutput.EppOutput;
import google.registry.persistence.transaction.QueryComposer.Comparator;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.request.lock.LockHandler;
@@ -68,7 +67,7 @@ import org.joda.time.Duration;
* this action runs, thus alerting us that human action is needed to correctly process the delete.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = DeleteExpiredDomainsAction.PATH,
auth = Auth.AUTH_ADMIN)
public class DeleteExpiredDomainsAction implements Runnable {
@@ -37,7 +37,6 @@ import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.HistoryEntryDao;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
@@ -55,7 +54,7 @@ import jakarta.inject.Inject;
* production.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = "/_dr/task/deleteLoadTestData",
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -42,7 +42,6 @@ import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.tld.Tld.TldType;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
import google.registry.util.RegistryEnvironment;
@@ -59,7 +58,7 @@ import org.joda.time.Duration;
* billing events, along with their ForeignKeyDomainIndex entities.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = "/_dr/task/deleteProberData",
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -35,7 +35,6 @@ import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.common.Cursor;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
@@ -51,7 +50,7 @@ import org.joda.time.DateTime;
* BillingRecurrence} billing events into synthetic {@link BillingEvent} events.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = "/_dr/task/expandBillingRecurrences",
auth = Auth.AUTH_ADMIN)
public class ExpandBillingRecurrencesAction implements Runnable {
@@ -32,7 +32,6 @@ import google.registry.model.eppcommon.StatusValue;
import google.registry.model.tld.RegistryLockDao;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
@@ -47,7 +46,7 @@ import org.joda.time.Duration;
/** Task that re-locks a previously-Registry-Locked domain after a predetermined period of time. */
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = RelockDomainAction.PATH,
method = POST,
automaticallyPrintOk = true,
@@ -113,11 +112,11 @@ public class RelockDomainAction implements Runnable {
public void run() {
/* We wish to manually control our retry behavior, in order to limit the number of retries
* and/or notify registrars / support only after a certain number of retries, or only
* with a certain type of failure. AppEngine will automatically retry on any non-2xx status
* with a certain type of failure. Cloud Tasks will automatically retry on any non-2xx status
* code, so return SC_NO_CONTENT (204) by default to avoid this auto-retry.
*
* See https://cloud.google.com/appengine/docs/standard/java/taskqueue/push/retrying-tasks
* for more details on retry behavior. */
* See https://docs.cloud.google.com/tasks/docs/configuring-queues#retry for more details on
* retry behavior. */
response.setStatus(SC_NO_CONTENT);
response.setContentType(MediaType.PLAIN_TEXT_UTF_8);
tm().transact(this::relockDomain);
@@ -44,7 +44,6 @@ import google.registry.model.eppcommon.ProtocolDefinition;
import google.registry.model.eppoutput.EppOutput;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.request.lock.LockHandler;
@@ -67,7 +66,7 @@ import org.joda.time.Duration;
* leaving behind a record recording that update.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = RemoveAllDomainContactsAction.PATH,
method = Action.Method.POST,
auth = Auth.AUTH_ADMIN)
@@ -28,7 +28,6 @@ import com.google.common.flogger.FluentLogger;
import com.google.common.net.MediaType;
import google.registry.config.RegistryConfig.Config;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
@@ -54,7 +53,7 @@ import jakarta.inject.Inject;
* <p>This runs the {@link google.registry.beam.resave.ResaveAllEppResourcesPipeline}.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = ResaveAllEppResourcesPipelineAction.PATH,
auth = Auth.AUTH_ADMIN)
public class ResaveAllEppResourcesPipelineAction implements Runnable {
@@ -25,7 +25,6 @@ import com.google.common.flogger.FluentLogger;
import google.registry.model.EppResource;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Action.Method;
import google.registry.request.Parameter;
import google.registry.request.Response;
@@ -40,7 +39,7 @@ import org.joda.time.DateTime;
* <p>{@link EppResource}s will be projected forward to the current time.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = ResaveEntityAction.PATH,
auth = Auth.AUTH_ADMIN,
method = Method.POST)
@@ -35,7 +35,6 @@ import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.model.registrar.RegistrarPoc.Type;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.EmailMessage;
@@ -50,7 +49,7 @@ import org.joda.time.format.DateTimeFormatter;
/** An action that sends notification emails to registrars whose certificates are expiring soon. */
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = SendExpiringCertificateNotificationEmailAction.PATH,
auth = Auth.AUTH_ADMIN)
public class SendExpiringCertificateNotificationEmailAction implements Runnable {
@@ -30,7 +30,6 @@ import google.registry.beam.wipeout.WipeOutContactHistoryPiiPipeline;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.contact.ContactHistory;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
@@ -49,7 +48,7 @@ import org.joda.time.DateTime;
* time.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = WipeOutContactHistoryPiiAction.PATH,
auth = Auth.AUTH_ADMIN)
public class WipeOutContactHistoryPiiAction implements Runnable {
@@ -40,8 +40,6 @@ public class RegistryPipelineWorkerInitializer implements JvmInitializer {
@Override
public void beforeProcessing(PipelineOptions options) {
// TODO(b/416299900): remove next line after GAE is removed.
System.setProperty("google.registry.jetty", "true");
RegistryPipelineOptions registryOptions = options.as(RegistryPipelineOptions.class);
RegistryEnvironment environment = registryOptions.getRegistryEnvironment();
if (environment == null || environment.equals(RegistryEnvironment.UNITTEST)) {
@@ -279,20 +279,6 @@ public class BigqueryConnection implements AutoCloseable {
private TableReference getTableReference() {
return table.getTableReference().clone();
}
/** Returns a string representation of the TableReference for the wrapped table. */
public String getStringReference() {
return tableReferenceToString(table.getTableReference());
}
/** Returns a string representation of the given TableReference. */
private static String tableReferenceToString(TableReference tableRef) {
return String.format(
"%s:%s.%s",
tableRef.getProjectId(),
tableRef.getDatasetId(),
tableRef.getTableId());
}
}
/**
@@ -398,29 +384,12 @@ public class BigqueryConnection implements AutoCloseable {
}
/**
* Starts an asynchronous query job to dump the results of the specified query into a local
* ImmutableTable object, row-keyed by the row number (indexed from 1), column-keyed by the
* TableFieldSchema for that column, and with the value object as the cell value. Note that null
* values will not actually be null, but they can be checked for using Data.isNull().
* Dumps the results of the specified query into a local ImmutableTable object, row-keyed by the
* row number (indexed from 1), column-keyed by the TableFieldSchema for that column, and with the
* value object as the cell value.
*
* <p>Returns a ListenableFuture that holds the ImmutableTable on success.
*/
public ListenableFuture<ImmutableTable<Integer, TableFieldSchema, Object>>
queryToLocalTable(String querySql) {
Job job = new Job()
.setConfiguration(new JobConfiguration()
.setQuery(new JobConfigurationQuery()
.setQuery(querySql)
.setDefaultDataset(getDataset())));
return transform(runJobToCompletion(job), this::getQueryResults, directExecutor());
}
/**
* Returns the result of calling queryToLocalTable, but synchronously to avoid spawning new
* background threads, which App Engine doesn't support.
*
* @see <a href="https://cloud.google.com/appengine/docs/standard/java/runtime#Threads">App Engine
* Runtime</a>
* <p>Note that null values will not actually be null, but they can be checked for using
* Data.isNull()
*/
public ImmutableTable<Integer, TableFieldSchema, Object> queryToLocalTableSync(String querySql) {
Job job = new Job()
@@ -634,10 +603,6 @@ public class BigqueryConnection implements AutoCloseable {
});
}
private ListenableFuture<Job> runJobToCompletion(final Job job) {
return service.submit(() -> runJob(job, null));
}
/** Helper that returns true if a dataset with this name exists. */
public boolean checkDatasetExists(String datasetName) throws IOException {
try {
@@ -676,14 +641,6 @@ public class BigqueryConnection implements AutoCloseable {
.setDatasetId(getDatasetId());
}
/** Returns table reference with the projectId and datasetId filled out for you. */
public TableReference getTable(String tableName) {
return new TableReference()
.setProjectId(getProjectId())
.setDatasetId(getDatasetId())
.setTableId(tableName);
}
/**
* Helper that creates a dataset with this name if it doesn't already exist, and returns true if
* creation took place.
@@ -71,9 +71,7 @@ class BsaDiffCreator {
Optional<String> previousJobName = schedule.latestCompleted().map(CompletedJob::jobName);
/*
* Memory usage is a concern when creating a diff, when the newest download needs to be held in
* memory in its entirety. The top-grade AppEngine VM has 3GB of memory, leaving less than 1.5GB
* to application memory footprint after subtracting overheads due to copying garbage collection
* and non-heap data etc. Assuming 400K labels, each of which on average included in 5 orders,
* memory in its entirety. Assuming 400K labels, each of which on average included in 5 orders,
* the memory footprint is at least 300MB when loaded into a Hashset-backed Multimap (64-bit
* JVM, with 12-byte object header, 16-byte array header, and 16-byte alignment).
*
@@ -41,7 +41,6 @@ import google.registry.bsa.persistence.DownloadScheduler;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.tld.Tlds;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
@@ -51,7 +50,7 @@ import java.util.Optional;
import java.util.stream.Stream;
@Action(
service = GaeService.BSA,
service = Action.Service.BACKEND,
path = BsaDownloadAction.PATH,
method = {GET, POST},
auth = Auth.AUTH_ADMIN)
@@ -31,7 +31,6 @@ import google.registry.bsa.persistence.RefreshScheduler;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.tld.Tlds;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.BatchedStreams;
@@ -42,7 +41,7 @@ import java.util.stream.Stream;
import org.joda.time.Duration;
@Action(
service = GaeService.BSA,
service = Action.Service.BACKEND,
path = BsaRefreshAction.PATH,
method = {GET, POST},
auth = Auth.AUTH_ADMIN)
@@ -53,7 +53,6 @@ import google.registry.model.domain.Domain;
import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldType;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
@@ -66,7 +65,7 @@ import org.joda.time.Duration;
/** Validates the BSA data in the database against the most recent block lists. */
@Action(
service = GaeService.BSA,
service = Action.Service.BACKEND,
path = BsaValidateAction.PATH,
method = {GET, POST},
auth = Auth.AUTH_ADMIN)
@@ -42,7 +42,6 @@ import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldType;
import google.registry.model.tld.label.ReservedList;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import jakarta.inject.Inject;
@@ -78,7 +77,7 @@ import org.joda.time.DateTime;
* <p>The file is also uploaded to GCS to preserve it as a record for ourselves.
*/
@Action(
service = GaeService.BSA,
service = Action.Service.BACKEND,
path = "/_dr/task/uploadBsaUnavailableNames",
method = {GET, POST},
auth = Auth.AUTH_ADMIN)
@@ -44,8 +44,6 @@ public abstract class CredentialModule {
* <p>The credential returned by the Cloud Runtime depends on the runtime environment:
*
* <ul>
* <li>On App Engine, returns a scope-less {@code ComputeEngineCredentials} for
* PROJECT_ID@appspot.gserviceaccount.com
* <li>On Compute Engine, returns a scope-less {@code ComputeEngineCredentials} for
* PROJECT_NUMBER-compute@developer.gserviceaccount.com
* <li>On end user host, this returns the credential downloaded by gcloud. Please refer to <a
@@ -87,8 +85,8 @@ public abstract class CredentialModule {
* the application default credential user.
*
* <p>The Workspace domain must grant delegated admin access to the default service account user
* (project-id@appspot.gserviceaccount.com on AppEngine) with all scopes in {@code defaultScopes}
* and {@code delegationScopes}.
* (nomulus-service-account@{project-id}.iam.gserviceaccount.com on GCP) with all scopes in {@code
* defaultScopes} and {@code delegationScopes}.
*/
@AdcDelegatedCredential
@Provides
@@ -113,9 +111,9 @@ public abstract class CredentialModule {
* Provides a {@link GoogleCredentialsBundle} for sending emails through Google Workspace.
*
* <p>The Workspace domain must grant delegated admin access to the default service account user
* (project-id@appspot.gserviceaccount.com on AppEngine) with all scopes in {@code defaultScopes}
* and {@code delegationScopes}. In addition, the user {@code gSuiteOutgoingEmailAddress} must
* have the permission to send emails.
* (nomulus-service-account@{project-id}.iam.gserviceaccount.com on GCP) with all scopes in {@code
* defaultScopes} and {@code delegationScopes}. In addition, the user {@code
* gSuiteOutgoingEmailAddress} must have the permission to send emails.
*/
@GmailDelegatedCredential
@Provides
@@ -55,8 +55,9 @@ import org.apache.commons.codec.binary.Base64;
*
* <p>This class accepts the application-default-credential as {@code ServiceAccountSigner},
* avoiding the need for exported private keys. In this case, the default credential user itself
* (project-id@appspot.gserviceaccount.com on AppEngine) must have domain-wide delegation to the
* Workspace APIs. The default credential user also must have the Token Creator role to itself.
* (nomulus-service-account@{project-id}.iam.gserviceaccount.com on GCP) must have domain-wide
* delegation to the Workspace APIs. The default credential user also must have the Token Creator
* role to itself.
*
* <p>If the user provides a credential {@code S} that carries its own private key, such as {@link
* com.google.auth.oauth2.ServiceAccountCredentials}, this class can use {@code S} to impersonate
@@ -37,7 +37,7 @@ import google.registry.bsa.UploadBsaUnavailableDomainsAction;
import google.registry.dns.ReadDnsRefreshRequestsAction;
import google.registry.model.common.DnsRefreshRequest;
import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.request.Action.GkeService;
import google.registry.request.Action.Service;
import google.registry.util.RegistryEnvironment;
import google.registry.util.YamlUtils;
import jakarta.inject.Named;
@@ -961,7 +961,7 @@ public final class RegistryConfig {
}
/**
* Number of times to retry a GAE operation when {@code TransientFailureException} is thrown.
* Number of times to retry an operation when {@code TransientFailureException} is thrown.
*
* <p>The number of milliseconds it'll sleep before giving up is {@code (2^n - 2) * 100}.
*
@@ -1422,7 +1422,7 @@ public final class RegistryConfig {
}
}
/** Returns the App Engine project ID, which is based off the environment name. */
/** Returns the project ID, which is based off the environment name. */
public static String getProjectId() {
return CONFIG_SETTINGS.get().gcpProject.projectId;
}
@@ -1444,55 +1444,10 @@ public final class RegistryConfig {
return CONFIG_SETTINGS.get().gcpProject.baseDomain;
}
public static URL getServiceUrl(GkeService service) {
public static URL getServiceUrl(Service service) {
return makeUrl(String.format("https://%s.%s", service.getServiceId(), getBaseDomain()));
}
/**
* Returns the address of the Nomulus app default HTTP server.
*
* <p>This is used by the {@code nomulus} tool to connect to the App Engine remote API.
*/
public static URL getDefaultServer() {
return makeUrl(CONFIG_SETTINGS.get().gcpProject.defaultServiceUrl);
}
/**
* Returns the address of the Nomulus app backend HTTP server.
*
* <p>This is used by the {@code nomulus} tool to connect to the App Engine remote API.
*/
public static URL getBackendServer() {
return makeUrl(CONFIG_SETTINGS.get().gcpProject.backendServiceUrl);
}
/**
* Returns the address of the Nomulus app bsa HTTP server.
*
* <p>This is used by the {@code nomulus} tool to connect to the App Engine remote API.
*/
public static URL getBsaServer() {
return makeUrl(CONFIG_SETTINGS.get().gcpProject.bsaServiceUrl);
}
/**
* Returns the address of the Nomulus app tools HTTP server.
*
* <p>This is used by the {@code nomulus} tool to connect to the App Engine remote API.
*/
public static URL getToolsServer() {
return makeUrl(CONFIG_SETTINGS.get().gcpProject.toolsServiceUrl);
}
/**
* Returns the address of the Nomulus app pubapi HTTP server.
*
* <p>This is used by the {@code nomulus} tool to connect to the App Engine remote API.
*/
public static URL getPubapiServer() {
return makeUrl(CONFIG_SETTINGS.get().gcpProject.pubapiServiceUrl);
}
/** Returns the amount of time a singleton should be cached, before expiring. */
public static java.time.Duration getSingletonCacheRefreshDuration() {
return java.time.Duration.ofSeconds(CONFIG_SETTINGS.get().caching.singletonCacheRefreshSeconds);
@@ -50,11 +50,6 @@ public class RegistryConfigSettings {
public long projectIdNumber;
public String locationId;
public boolean isLocal;
public String defaultServiceUrl;
public String backendServiceUrl;
public String bsaServiceUrl;
public String toolsServiceUrl;
public String pubapiServiceUrl;
public String baseDomain;
}
@@ -12,17 +12,11 @@ gcpProject:
projectIdNumber: 123456789012
# Location of the GCP project, note that us-central1 and europe-west1 are special in that
# they are used without the trailing number in GCP commands and Google Cloud Console.
# See: https://cloud.google.com/appengine/docs/locations as an example
# See: https://docs.cloud.google.com/compute/docs/regions-zones as an example
locationId: registry-location-id
# whether to use local/test credentials when connecting to the servers
isLocal: true
# URLs of the services for the project.
defaultServiceUrl: https://default.example.com
backendServiceUrl: https://backend.example.com
bsaServiceUrl: https://bsa.example.com
toolsServiceUrl: https://tools.example.com
pubapiServiceUrl: https://pubapi.example.com
# The base domain name of the registry service. Services are reachable at [service].baseDomain.
baseDomain: registry.test
@@ -32,9 +26,9 @@ gSuite:
domainName: domain-registry.example
# Display name and email address used on outgoing emails through G Suite.
# The email address must be valid and have permission in the GAE app to send
# emails. For more info see:
# https://cloud.google.com/appengine/docs/standard/java/mail/#who_can_send_mail
# The email address must be valid and the domain must be set up to send emails.
# For more info see
# https://docs.cloud.google.com/compute/docs/tutorials/sending-mail
outgoingEmailDisplayName: Example Registry
outgoingEmailAddress: noreply@project-id.appspotmail.com
# TODO(b/279671974): reuse `outgoingEmailAddress` after migration
@@ -201,18 +195,16 @@ hibernate:
# but lock tables explicitly, either using framework-dependent API, or execute
# "select table for update" statements directly.
connectionIsolation: TRANSACTION_SERIALIZABLE
# Whether to log all SQL queries to App Engine logs. Overridable at runtime.
# Whether to log all SQL queries. Overridable at runtime.
logSqlQueries: false
# Connection pool configurations.
hikariConnectionTimeout: 20000
# Cloud SQL connections are a relatively scarce resource (maximum is 1000 as
# of March 2021). The minimumIdle should be a small value so that machines may
# release connections after a demand spike. The maximumPoolSize is set to 10
# because that is the maximum number of concurrent requests a Nomulus server
# instance can handle (as limited by AppEngine for basic/manual scaling). Note
# that BEAM pipelines are not subject to the maximumPoolSize value defined
# here. See PersistenceModule.java for more information.
# release connections after a demand spike. Note that BEAM pipelines are not
# subject to the maximumPoolSize value defined here. See PersistenceModule.java
# for more information.
hikariMinimumIdle: 1
hikariMaximumPoolSize: 40
hikariIdleTimeout: 300000
@@ -264,8 +256,8 @@ caching:
# Maximum total number of static premium list entry entities to cache in
# memory, across all premium lists for all TLDs. Tuning this up will use more
# memory (and might require using larger App Engine instances). Note that
# premium list entries that are absent are cached in addition to ones that are
# memory (and might require using larger instances). Note that premium list
# entries that are absent are cached in addition to ones that are
# present, so the total cache size is not bounded by the total number of
# premium price entries that exist.
staticPremiumListMaxCachedEntries: 200000
@@ -346,12 +338,8 @@ credentialOAuth:
localCredentialOauthScopes:
# View and manage data in all Google Cloud APIs.
- https://www.googleapis.com/auth/cloud-platform
# Call App Engine APIs locally.
- https://www.googleapis.com/auth/appengine.apis
# View your email address.
- https://www.googleapis.com/auth/userinfo.email
# View and manage your applications deployed on Google App Engine
- https://www.googleapis.com/auth/appengine.admin
# The lifetime of an access token generated by our custom credentials classes
# Must be shorter than one hour.
tokenRefreshDelaySeconds: 1800
@@ -433,7 +421,7 @@ misc:
spec11BccEmailAddresses:
- abuse@example.com
# Number of times to retry a GAE operation when a transient exception is thrown.
# Number of times to retry an operation when a transient exception is thrown.
# The number of milliseconds it'll sleep before giving up is (2^n - 2) * 100.
transientFailureRetries: 12
@@ -40,14 +40,12 @@ import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import google.registry.batch.CloudTasksUtils;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Action.GkeService;
import google.registry.request.Action.Service;
import google.registry.request.Parameter;
import google.registry.request.ParameterMap;
import google.registry.request.RequestParameters;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.RegistryEnvironment;
import jakarta.inject.Inject;
import java.util.Optional;
import java.util.stream.Stream;
@@ -59,8 +57,7 @@ import java.util.stream.Stream;
*
* <ul>
* <li>{@code endpoint} (Required) URL path of servlet to launch. This may contain pathargs.
* <li>{@code queue} (Required) Name of the App Engine push queue to which this task should be
* sent.
* <li>{@code queue} (Required) Name of the queue to which this task should be sent.
* <li>{@code forEachRealTld} Launch the task in each real TLD namespace.
* <li>{@code forEachTestTld} Launch the task in each test TLD namespace.
* <li>{@code runInEmpty} Launch the task once, without the TLD argument.
@@ -80,7 +77,7 @@ import java.util.stream.Stream;
* </ul>
*/
@Action(
service = GaeService.BACKEND,
service = Service.BACKEND,
path = "/_dr/cron/fanout",
automaticallyPrintOk = true,
auth = Auth.AUTH_ADMIN)
@@ -160,10 +157,6 @@ public final class TldFanoutAction implements Runnable {
params.put(RequestParameters.PARAM_TLD, tld);
}
return cloudTasksUtils.createTaskWithJitter(
endpoint,
Action.Method.POST,
RegistryEnvironment.isOnJetty() ? GkeService.BACKEND : GaeService.BACKEND,
params,
jitterSeconds);
endpoint, Action.Method.POST, Service.BACKEND, params, jitterSeconds);
}
}
@@ -52,7 +52,6 @@ import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.model.tld.Tld;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Header;
import google.registry.request.HttpException.ServiceUnavailableException;
import google.registry.request.Parameter;
@@ -72,7 +71,7 @@ import org.joda.time.Duration;
/** Task that sends domain and host updates to the DNS server. */
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = PublishDnsUpdatesAction.PATH,
method = POST,
automaticallyPrintOk = true,
@@ -45,7 +45,6 @@ import google.registry.dns.DnsUtils.TargetType;
import google.registry.model.common.DnsRefreshRequest;
import google.registry.model.tld.Tld;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
@@ -60,7 +59,7 @@ import org.joda.time.Duration;
* table.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = "/_dr/task/readDnsRefreshRequests",
automaticallyPrintOk = true,
method = POST,
@@ -26,7 +26,6 @@ import google.registry.model.annotations.ExternalMessagingName;
import google.registry.model.domain.Domain;
import google.registry.model.host.Host;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.BadRequestException;
import google.registry.request.HttpException.NotFoundException;
import google.registry.request.Parameter;
@@ -36,7 +35,7 @@ import jakarta.inject.Inject;
/** Action that manually triggers refresh of DNS information. */
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = "/_dr/task/dnsRefresh",
automaticallyPrintOk = true,
auth = Auth.AUTH_ADMIN)
@@ -26,7 +26,6 @@ import google.registry.model.domain.Domain;
import google.registry.model.host.Host;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
@@ -34,7 +33,7 @@ import jakarta.inject.Inject;
import org.joda.time.DateTime;
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = PATH,
method = Action.Method.POST,
auth = Auth.AUTH_ADMIN)
@@ -36,8 +36,10 @@ import org.xbill.DNS.Opcode;
/**
* A transport for DNS messages. Sends/receives DNS messages over TCP using old-style {@link Socket}
* s and the message framing defined in <a href="https://tools.ietf.org/html/rfc1035">RFC 1035</a>.
* We would like use the dnsjava library's {@link org.xbill.DNS.SimpleResolver} class for this, but
* it requires {@link java.nio.channels.SocketChannel} which is not supported on AppEngine.
*
* <p>TODO(b/463732345): now that we're no longer on AppEngine, see if we can use the dnsjava
* library's {@link org.xbill.DNS.SimpleResolver} class instead of this (that requires {@link
* java.nio.channels.SocketChannel} which is not supported on AppEngine).
*/
public class DnsMessageTransport {
@@ -36,7 +36,6 @@ import google.registry.model.eppcommon.StatusValue;
import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldType;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.auth.Auth;
import google.registry.storage.drive.DriveConnection;
import google.registry.util.Clock;
@@ -58,7 +57,7 @@ import org.joda.time.DateTimeZone;
* name TLD.txt into the domain-lists bucket. Note that this overwrites the files in place.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = "/_dr/task/exportDomainLists",
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -33,7 +33,6 @@ import google.registry.model.tld.Tld;
import google.registry.model.tld.label.PremiumList.PremiumEntry;
import google.registry.model.tld.label.PremiumListDao;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.RequestParameters;
import google.registry.request.Response;
@@ -46,7 +45,7 @@ import java.util.SortedSet;
/** Action that exports the premium terms list for a TLD to Google Drive. */
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = "/_dr/task/exportPremiumTerms",
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -25,7 +25,6 @@ import com.google.common.flogger.FluentLogger;
import com.google.common.net.MediaType;
import google.registry.model.tld.Tld;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.RequestParameters;
import google.registry.request.Response;
@@ -35,7 +34,7 @@ import jakarta.inject.Inject;
/** Action that exports the publicly viewable reserved terms list for a TLD to Google Drive. */
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = "/_dr/task/exportReservedTerms",
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -34,7 +34,6 @@ import google.registry.groups.GroupsConnection.Role;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Retrier;
@@ -53,7 +52,7 @@ import javax.annotation.Nullable;
* <p>This uses the <a href="https://developers.google.com/admin-sdk/directory/">Directory API</a>.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = "/_dr/task/syncGroupMembers",
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -24,7 +24,6 @@ import static jakarta.servlet.http.HttpServletResponse.SC_OK;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryConfig.Config;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
@@ -55,7 +54,7 @@ import org.joda.time.Duration;
* @see SyncRegistrarsSheet
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = SyncRegistrarsSheetAction.PATH,
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -54,7 +54,6 @@ import google.registry.model.tld.label.ReservationType;
import google.registry.monitoring.whitebox.CheckApiMetric;
import google.registry.monitoring.whitebox.CheckApiMetric.Availability;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.RequestParameters;
import google.registry.request.Response;
@@ -72,7 +71,7 @@ import org.joda.time.DateTime;
* user controlled, lest it open an XSS vector. Do not modify this to return the domain name in the
* response.
*/
@Action(service = GaeService.PUBAPI, path = "/check", auth = Auth.AUTH_PUBLIC)
@Action(service = Action.Service.PUBAPI, path = "/check", auth = Auth.AUTH_PUBLIC)
public class CheckApiAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
@@ -15,7 +15,6 @@
package google.registry.flows;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Action.Method;
import google.registry.request.Payload;
import google.registry.request.auth.Auth;
@@ -27,7 +26,7 @@ import jakarta.servlet.http.HttpServletRequest;
* to RFC 5730. Commands must be requested via POST.
*/
@Action(
service = GaeService.DEFAULT,
service = Action.Service.FRONTEND,
path = "/_dr/epp",
method = Method.POST,
auth = Auth.AUTH_ADMIN)
@@ -22,7 +22,6 @@ import dagger.Module;
import dagger.Provides;
import google.registry.model.eppcommon.ProtocolDefinition;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Action.Method;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
@@ -31,7 +30,7 @@ import jakarta.servlet.http.HttpServletRequest;
/** Runs EPP commands directly without logging in, verifying an XSRF token from the tool. */
@Action(
service = GaeService.TOOLS,
service = Action.Service.BACKEND,
path = EppToolAction.PATH,
method = Method.POST,
auth = Auth.AUTH_ADMIN)
@@ -51,7 +51,7 @@ public class FlowReporter {
@Inject Class<? extends Flow> flowClass;
@Inject FlowReporter() {}
/** Records information about the current flow execution in the GAE request logs. */
/** Records information about the current flow execution in the request logs. */
public void recordToLogs() {
// Explicitly log flow metadata separately from the EPP XML itself so that it stays compact
// enough to be sure to fit in a single log entry (the XML part in rare cases could be long
@@ -73,7 +73,7 @@ public class FlowRunner {
eppRequestSource,
isDryRun ? "DRY_RUN" : "LIVE",
isSuperuser ? "SUPERUSER" : "NORMAL");
// Record flow info to the GAE request logs for reporting purposes if it's not a dry run.
// Record flow info to the request logs for reporting purposes if it's not a dry run.
if (!isDryRun) {
flowReporter.recordToLogs();
}
@@ -133,10 +133,9 @@ import org.joda.time.DateTime;
@ReportingSpec(ActivityReportField.DOMAIN_TRANSFER_REQUEST)
public final class DomainTransferRequestFlow implements MutatingFlow {
private static final ImmutableSet<StatusValue> DISALLOWED_STATUSES = ImmutableSet.of(
StatusValue.CLIENT_TRANSFER_PROHIBITED,
StatusValue.PENDING_DELETE,
StatusValue.SERVER_TRANSFER_PROHIBITED);
private static final ImmutableSet<StatusValue> NON_SUPERUSER_DISALLOWED_STATUSES =
ImmutableSet.of(
StatusValue.CLIENT_TRANSFER_PROHIBITED, StatusValue.SERVER_TRANSFER_PROHIBITED);
@Inject ResourceCommand resourceCommand;
@Inject ExtensionManager extensionManager;
@@ -299,8 +298,9 @@ public final class DomainTransferRequestFlow implements MutatingFlow {
DateTime now,
Optional<DomainTransferRequestSuperuserExtension> superuserExtension)
throws EppException {
verifyNoDisallowedStatuses(existingDomain, DISALLOWED_STATUSES);
verifyNoDisallowedStatuses(existingDomain, ImmutableSet.of(StatusValue.PENDING_DELETE));
if (!isSuperuser) {
verifyNoDisallowedStatuses(existingDomain, NON_SUPERUSER_DISALLOWED_STATUSES);
verifyAuthInfoPresentForResourceTransfer(authInfo);
verifyAuthInfo(authInfo.get(), existingDomain);
}
@@ -47,6 +47,7 @@ import google.registry.model.eppinput.EppInput.Options;
import google.registry.model.eppinput.EppInput.Services;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.registrar.Registrar;
import google.registry.util.PasswordUtils;
import google.registry.util.StopwatchLogger;
import jakarta.inject.Inject;
import java.util.Optional;
@@ -150,8 +151,19 @@ public class LoginFlow implements MutatingFlow {
throw new RegistrarAccountNotActiveException();
}
if (login.getNewPassword().isPresent()) {
String newPassword = login.getNewPassword().get();
// TODO(b/458423787): Remove this circa March 2026 after enough time has passed for the logins
// to have transitioned to Argon2 hashing.
if (login.getNewPassword().isPresent()
|| registrar.get().getCurrentHashAlgorithm(login.getPassword()).orElse(null)
!= PasswordUtils.HashAlgorithm.ARGON_2_ID) {
String newPassword =
login
.getNewPassword()
.orElseGet(
() -> {
logger.atInfo().log("Rehashing existing registrar password with ARGON_2_ID");
return login.getPassword();
});
// Load fresh from database (bypassing the cache) to ensure we don't save stale data.
Optional<Registrar> freshRegistrar = Registrar.loadByRegistrarId(login.getClientId());
stopwatch.tick("LoginFlow reload freshRegistrar");
@@ -44,7 +44,7 @@ public interface Keyring extends AutoCloseable {
* Returns public key for encrypting escrow deposits being staged to cloud storage.
*
* <p>This adds an additional layer of security so cloud storage administrators won't be tempted
* to go poking around the App Engine Cloud Console and see a dump of the entire database.
* to go poking around the Pantheon Cloud Console and see a dump of the entire database.
*
* <p>This keypair should only be known to the domain registry shared registry system.
*
@@ -28,7 +28,6 @@ import com.google.protobuf.Timestamp;
import google.registry.batch.CloudTasksUtils;
import google.registry.flows.EppToolAction;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
@@ -52,7 +51,7 @@ import org.joda.time.DateTime;
* least one must be specified in order for load testing to do anything.
*/
@Action(
service = GaeService.TOOLS,
service = Action.Service.BACKEND,
path = LoadTestAction.PATH,
method = Action.Method.POST,
automaticallyPrintOk = true,
@@ -76,7 +76,7 @@ public class Cursor extends UpdateAutoTimestampEntity {
*
* <p>The way we solve this problem is by having {@code RdeUploadAction} check this cursor
* before performing an upload for a given TLD. If the cursor is less than two hours old, the
* action will fail with a status code above 300 and App Engine will keep retrying the action
* action will fail with a status code above 300 and Cloud Tasks will keep retrying the action
* until it's ready.
*/
RDE_UPLOAD_SFTP(true),
@@ -37,6 +37,7 @@ import google.registry.tools.IamClient;
import google.registry.tools.ServiceConnection;
import google.registry.tools.server.UpdateUserGroupAction;
import google.registry.util.PasswordUtils;
import google.registry.util.PasswordUtils.HashAlgorithm;
import google.registry.util.RegistryEnvironment;
import jakarta.persistence.Column;
import jakarta.persistence.Embeddable;
@@ -229,6 +230,10 @@ public class User extends UpdateAutoTimestampEntity implements Buildable {
|| isNullOrEmpty(registryLockPasswordHash)) {
return false;
}
return getCurrentHashAlgorithm(registryLockPassword).isPresent();
}
public Optional<HashAlgorithm> getCurrentHashAlgorithm(String registryLockPassword) {
return PasswordUtils.verifyPassword(
registryLockPassword, registryLockPasswordHash, registryLockPasswordSalt);
}
@@ -56,7 +56,7 @@ public class ProtocolDefinition {
FEE_0_6(FeeCheckCommandExtensionV06.class, FeeCheckResponseExtensionV06.class, true),
FEE_0_11(FeeCheckCommandExtensionV11.class, FeeCheckResponseExtensionV11.class, true),
FEE_0_12(FeeCheckCommandExtensionV12.class, FeeCheckResponseExtensionV12.class, true),
FEE_1_00(FeeCheckCommandExtensionStdV1.class, FeeCheckResponseExtensionStdV1.class, true),
FEE_1_00(FeeCheckCommandExtensionStdV1.class, FeeCheckResponseExtensionStdV1.class, false),
METADATA_1_0(MetadataExtension.class, null, false);
private final Class<? extends CommandExtension> commandExtensionClass;
@@ -67,6 +67,7 @@ import google.registry.persistence.converter.CurrencyToStringMapUserType;
import google.registry.persistence.transaction.TransactionManager;
import google.registry.util.CidrAddressBlock;
import google.registry.util.PasswordUtils;
import google.registry.util.PasswordUtils.HashAlgorithm;
import jakarta.mail.internet.AddressException;
import jakarta.mail.internet.InternetAddress;
import jakarta.persistence.AttributeOverride;
@@ -672,6 +673,10 @@ public class Registrar extends UpdateAutoTimestampEntity implements Buildable, J
}
public boolean verifyPassword(String password) {
return getCurrentHashAlgorithm(password).isPresent();
}
public Optional<HashAlgorithm> getCurrentHashAlgorithm(String password) {
return PasswordUtils.verifyPassword(password, passwordHash, salt);
}
@@ -19,8 +19,7 @@ import static jakarta.servlet.http.HttpServletResponse.SC_OK;
import com.google.common.flogger.FluentLogger;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Action.GkeService;
import google.registry.request.Action.Service;
import google.registry.request.auth.Auth;
import jakarta.inject.Inject;
import jakarta.servlet.http.HttpServletResponse;
@@ -53,8 +52,7 @@ public class ReadinessProbeAction implements Runnable {
}
@Action(
service = GaeService.DEFAULT,
gkeService = GkeService.CONSOLE,
service = Service.CONSOLE,
path = ReadinessProbeConsoleAction.PATH,
auth = Auth.AUTH_PUBLIC)
public static class ReadinessProbeConsoleAction extends ReadinessProbeAction {
@@ -66,11 +64,7 @@ public class ReadinessProbeAction implements Runnable {
}
}
@Action(
service = GaeService.PUBAPI,
gkeService = GkeService.PUBAPI,
path = ReadinessProbeActionPubApi.PATH,
auth = Auth.AUTH_PUBLIC)
@Action(service = Service.PUBAPI, path = ReadinessProbeActionPubApi.PATH, auth = Auth.AUTH_PUBLIC)
public static class ReadinessProbeActionPubApi extends ReadinessProbeAction {
public static final String PATH = "/ready/pubapi";
@@ -81,8 +75,7 @@ public class ReadinessProbeAction implements Runnable {
}
@Action(
service = GaeService.DEFAULT,
gkeService = GkeService.FRONTEND,
service = Service.FRONTEND,
path = ReadinessProbeActionFrontend.PATH,
auth = Auth.AUTH_PUBLIC)
public static final class ReadinessProbeActionFrontend extends ReadinessProbeAction {
@@ -28,7 +28,7 @@ import java.util.concurrent.TimeoutException;
import org.bouncycastle.jce.provider.BouncyCastleProvider;
import org.joda.time.DateTime;
/** Base for Servlets that handle all requests to our App Engine modules. */
/** Base for Servlets that handle all requests to our modules. */
public class ServletBase extends HttpServlet {
private final RequestHandler<?> requestHandler;
@@ -16,7 +16,6 @@ package google.registry.monitoring.whitebox;
import com.google.api.services.monitoring.v3.Monitoring;
import com.google.api.services.monitoring.v3.model.MonitoredResource;
import com.google.common.collect.ImmutableMap;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import com.google.monitoring.metrics.MetricReporter;
import com.google.monitoring.metrics.MetricWriter;
@@ -29,7 +28,6 @@ import google.registry.config.RegistryConfig.Config;
import google.registry.util.Clock;
import google.registry.util.GoogleCredentialsBundle;
import google.registry.util.MetricParameters;
import google.registry.util.RegistryEnvironment;
import jakarta.inject.Named;
import jakarta.inject.Singleton;
import org.joda.time.Duration;
@@ -40,13 +38,9 @@ public final class StackdriverModule {
private StackdriverModule() {}
// We need a fake GCE zone to appease Stackdriver's resource model.
// TODO(b/265973059): Switch to resource type "gke_container".
private static final String SPOOFED_GCE_ZONE = "us-central1-f";
// We cannot use a static fake intance ID which is shared by all instances, because metrics might
// be flushed to stackdriver with delays, which lead to time inversion erros when another instance
// has already written a data point at a later time.
// We cannot use a static fake instance ID which is shared by all instances, because metrics might
// be flushed to stackdriver with delays, which lead to time inversion errors when another
// instance has already written a data point at a later time.
@Singleton
@Provides
@Named("spoofedGceInstanceId")
@@ -72,23 +66,11 @@ public final class StackdriverModule {
Lazy<MetricParameters> gkeParameters,
@Config("projectId") String projectId,
@Config("stackdriverMaxQps") int maxQps,
@Config("stackdriverMaxPointsPerRequest") int maxPointsPerRequest,
@Named("spoofedGceInstanceId") String instanceId) {
@Config("stackdriverMaxPointsPerRequest") int maxPointsPerRequest) {
MonitoredResource resource =
RegistryEnvironment.isOnJetty()
? new MonitoredResource()
.setType("gke_container")
.setLabels(gkeParameters.get().makeLabelsMap())
:
// The MonitoredResource for GAE apps is not writable (and missing fields anyway) so we
// just use the gce_instance resource type instead.
new MonitoredResource()
.setType("gce_instance")
.setLabels(
ImmutableMap.of(
// The "zone" field MUST be a valid GCE zone, so we fake one.
"zone", SPOOFED_GCE_ZONE, "instance_id", instanceId));
new MonitoredResource()
.setType("gke_container")
.setLabels(gkeParameters.get().makeLabelsMap());
return new StackdriverWriter(
monitoringClient, projectId, resource, maxQps, maxPointsPerRequest);
}
@@ -20,7 +20,6 @@ import static google.registry.request.Action.Method.HEAD;
import google.registry.rdap.RdapMetrics.EndpointType;
import google.registry.rdap.RdapObjectClasses.ReplyPayloadBase;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.NotImplementedException;
import google.registry.request.auth.Auth;
import jakarta.inject.Inject;
@@ -32,7 +31,7 @@ import jakarta.inject.Inject;
* ARIN, not domain registries.
*/
@Action(
service = GaeService.PUBAPI,
service = Action.Service.PUBAPI,
path = "/rdap/autnum/",
method = {GET, HEAD},
isPrefix = true,
@@ -29,7 +29,6 @@ import google.registry.rdap.RdapJsonFormatter.OutputDataType;
import google.registry.rdap.RdapMetrics.EndpointType;
import google.registry.rdap.RdapObjectClasses.RdapDomain;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.BadRequestException;
import google.registry.request.HttpException.NotFoundException;
import google.registry.request.auth.Auth;
@@ -38,7 +37,7 @@ import java.util.Optional;
/** RDAP action for domain requests. */
@Action(
service = GaeService.PUBAPI,
service = Action.Service.PUBAPI,
path = "/rdap/domain/",
method = {GET, HEAD},
isPrefix = true,
@@ -42,7 +42,6 @@ import google.registry.rdap.RdapMetrics.WildcardType;
import google.registry.rdap.RdapSearchResults.DomainSearchResponse;
import google.registry.rdap.RdapSearchResults.IncompletenessWarningType;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.BadRequestException;
import google.registry.request.HttpException.NotFoundException;
import google.registry.request.HttpException.UnprocessableEntityException;
@@ -71,7 +70,7 @@ import org.joda.time.DateTime;
* Data Access Protocol (RDAP)</a>
*/
@Action(
service = GaeService.PUBAPI,
service = Action.Service.PUBAPI,
path = "/rdap/domains",
method = {GET, HEAD},
auth = Auth.AUTH_PUBLIC)
@@ -30,7 +30,7 @@ import java.io.IOException;
* them the help response.
*/
@Action(
service = Action.GaeService.PUBAPI,
service = Action.Service.PUBAPI,
path = "/rdap/",
method = {GET, HEAD},
auth = Auth.AUTH_PUBLIC)
@@ -25,7 +25,6 @@ import google.registry.rdap.RdapJsonFormatter.OutputDataType;
import google.registry.rdap.RdapMetrics.EndpointType;
import google.registry.rdap.RdapObjectClasses.RdapEntity;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.NotFoundException;
import google.registry.request.auth.Auth;
import jakarta.inject.Inject;
@@ -40,7 +39,7 @@ import java.util.Optional;
* the handle of the entity with the registrar role is be [sic] equal to the IANA Registrar ID.
*/
@Action(
service = GaeService.PUBAPI,
service = Action.Service.PUBAPI,
path = "/rdap/entity/",
method = {GET, HEAD},
isPrefix = true,
@@ -31,7 +31,6 @@ import google.registry.rdap.RdapMetrics.SearchType;
import google.registry.rdap.RdapSearchResults.EntitySearchResponse;
import google.registry.rdap.RdapSearchResults.IncompletenessWarningType;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.BadRequestException;
import google.registry.request.HttpException.NotFoundException;
import google.registry.request.HttpException.UnprocessableEntityException;
@@ -56,7 +55,7 @@ import java.util.Optional;
* Data Access Protocol (RDAP)</a>
*/
@Action(
service = GaeService.PUBAPI,
service = Action.Service.PUBAPI,
path = "/rdap/entities",
method = {GET, HEAD},
auth = Auth.AUTH_PUBLIC)
@@ -22,7 +22,6 @@ import google.registry.rdap.RdapDataStructures.Notice;
import google.registry.rdap.RdapMetrics.EndpointType;
import google.registry.rdap.RdapObjectClasses.HelpResponse;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.NotFoundException;
import google.registry.request.auth.Auth;
import jakarta.inject.Inject;
@@ -30,7 +29,7 @@ import java.util.Optional;
/** RDAP action for help requests. */
@Action(
service = GaeService.PUBAPI,
service = Action.Service.PUBAPI,
path = RdapHelpAction.PATH,
method = {GET, HEAD},
isPrefix = true,
@@ -20,7 +20,6 @@ import static google.registry.request.Action.Method.HEAD;
import google.registry.rdap.RdapMetrics.EndpointType;
import google.registry.rdap.RdapObjectClasses.ReplyPayloadBase;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.NotImplementedException;
import google.registry.request.auth.Auth;
import jakarta.inject.Inject;
@@ -32,7 +31,7 @@ import jakarta.inject.Inject;
* ARIN, not domain registries.
*/
@Action(
service = GaeService.PUBAPI,
service = Action.Service.PUBAPI,
path = "/rdap/ip/",
method = {GET, HEAD},
isPrefix = true,
@@ -26,7 +26,6 @@ import google.registry.rdap.RdapJsonFormatter.OutputDataType;
import google.registry.rdap.RdapMetrics.EndpointType;
import google.registry.rdap.RdapObjectClasses.RdapNameserver;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.BadRequestException;
import google.registry.request.HttpException.NotFoundException;
import google.registry.request.auth.Auth;
@@ -35,7 +34,7 @@ import java.util.Optional;
/** RDAP action for nameserver requests. */
@Action(
service = GaeService.PUBAPI,
service = Action.Service.PUBAPI,
path = "/rdap/nameserver/",
method = {GET, HEAD},
isPrefix = true,
@@ -34,7 +34,6 @@ import google.registry.rdap.RdapMetrics.SearchType;
import google.registry.rdap.RdapSearchResults.IncompletenessWarningType;
import google.registry.rdap.RdapSearchResults.NameserverSearchResponse;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.BadRequestException;
import google.registry.request.HttpException.NotFoundException;
import google.registry.request.HttpException.UnprocessableEntityException;
@@ -57,7 +56,7 @@ import java.util.Optional;
* Data Access Protocol (RDAP)</a>
*/
@Action(
service = GaeService.PUBAPI,
service = Action.Service.PUBAPI,
path = "/rdap/nameservers",
method = {GET, HEAD},
auth = Auth.AUTH_PUBLIC)
@@ -27,7 +27,6 @@ import com.google.common.flogger.FluentLogger;
import google.registry.model.registrar.Registrar;
import google.registry.persistence.PersistenceModule;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.InternalServerErrorException;
import google.registry.request.UrlConnectionService;
import google.registry.request.UrlConnectionUtils;
@@ -54,7 +53,7 @@ import org.apache.commons.csv.CSVRecord;
* CSV endpoint requires no authentication.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = "/_dr/task/updateRegistrarRdapBaseUrls",
automaticallyPrintOk = true,
auth = Auth.AUTH_ADMIN)
@@ -32,7 +32,6 @@ import google.registry.model.rde.RdeNamingUtils;
import google.registry.model.rde.RdeRevision;
import google.registry.model.tld.Tld;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.NoContentException;
import google.registry.request.Parameter;
import google.registry.request.RequestParameters;
@@ -63,7 +62,7 @@ import org.joda.time.DateTime;
* Agreement</a>
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = BrdaCopyAction.PATH,
method = POST,
automaticallyPrintOk = true,
@@ -33,10 +33,10 @@ import org.joda.time.Duration;
* Runner applying guaranteed reliability to an {@link EscrowTask}.
*
* <p>This class implements the <i>Locking Rolling Cursor</i> pattern, which solves the problem of
* how to reliably execute App Engine tasks which can't be made idempotent.
* how to reliably execute Cloud Tasks which can't be made idempotent.
*
* <p>{@link LockHandler} is used to ensure only one task executes at a time for a given {@code
* LockedCursorTask} subclass + TLD combination. This is necessary because App Engine tasks might
* LockedCursorTask} subclass + TLD combination. This is necessary because Cloud Task tasks might
* double-execute. Normally tasks solve this by being idempotent, but that's not possible for RDE,
* which writes to a GCS filename with a deterministic name. So locks are used to guarantee
* isolation. If we can't acquire the lock, it means the task is already running, so {@link
@@ -36,7 +36,6 @@ import google.registry.model.rde.RdeRevision;
import google.registry.model.tld.Tld;
import google.registry.rde.EscrowTaskRunner.EscrowTask;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.NoContentException;
import google.registry.request.Parameter;
import google.registry.request.RequestParameters;
@@ -54,7 +53,7 @@ import org.joda.time.Duration;
* Action that uploads a small XML RDE report to ICANN after {@link RdeUploadAction} has finished.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = RdeReportAction.PATH,
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -50,7 +50,6 @@ import google.registry.model.host.Host;
import google.registry.model.rde.RdeMode;
import google.registry.model.registrar.Registrar;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.BadRequestException;
import google.registry.request.Parameter;
import google.registry.request.RequestParameters;
@@ -101,8 +100,8 @@ import org.joda.time.Duration;
*
* <h2>Logging</h2>
*
* <p>To identify the reduce worker request for a deposit in App Engine's log viewer, you can use
* search text like {@code tld=soy}, {@code watermark=2015-01-01}, and {@code mode=FULL}.
* <p>To identify the reduce worker request for a deposit in the log viewer, you can use search text
* like {@code tld=soy}, {@code watermark=2015-01-01}, and {@code mode=FULL}.
*
* <h3>Error Handling</h3>
*
@@ -205,7 +204,7 @@ import org.joda.time.Duration;
* Name Registration Data Objects Mapping</a>
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = RdeStagingAction.PATH,
method = {GET, POST},
auth = Auth.AUTH_ADMIN)
@@ -50,7 +50,6 @@ import google.registry.model.tld.Tld;
import google.registry.rde.EscrowTaskRunner.EscrowTask;
import google.registry.rde.JSchSshSession.JSchSshSessionFactory;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.NoContentException;
import google.registry.request.Parameter;
import google.registry.request.RequestParameters;
@@ -84,7 +83,7 @@ import org.joda.time.Duration;
* RdeReportAction}.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = RdeUploadAction.PATH,
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -33,7 +33,6 @@ import google.registry.gcs.GcsUtils;
import google.registry.model.registrar.Registrar;
import google.registry.reporting.billing.BillingModule.InvoiceDirectoryPrefix;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.storage.drive.DriveConnection;
@@ -45,7 +44,7 @@ import java.util.Optional;
/** Copy all registrar detail reports in a given bucket's subdirectory from GCS to Drive. */
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = CopyDetailReportsAction.PATH,
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -32,7 +32,6 @@ import google.registry.config.RegistryConfig.Config;
import google.registry.persistence.PersistenceModule;
import google.registry.reporting.ReportingModule;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
@@ -51,7 +50,7 @@ import org.joda.time.YearMonth;
* template. The pipeline then generates invoices for the month and stores them on GCS.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = GenerateInvoicesAction.PATH,
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -30,7 +30,6 @@ import google.registry.batch.CloudTasksUtils;
import google.registry.config.RegistryConfig.Config;
import google.registry.reporting.ReportingModule;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
@@ -49,7 +48,7 @@ import org.joda.time.YearMonth;
* Job States</a>
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = PublishInvoicesAction.PATH,
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -34,7 +34,6 @@ import google.registry.groups.GmailClient;
import google.registry.reporting.ReportingModule;
import google.registry.reporting.icann.IcannReportingModule.ReportType;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
@@ -66,7 +65,7 @@ import org.joda.time.format.DateTimeFormat;
* 'transactions'. If none specified - defaults to generating both.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = IcannReportingStagingAction.PATH,
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -35,7 +35,6 @@ import google.registry.model.tld.Tld.TldType;
import google.registry.model.tld.Tlds;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.ServiceUnavailableException;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
@@ -68,7 +67,7 @@ import org.joda.time.Duration;
* Defaults to "icann/monthly/[last month in yyyy-MM format]".
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = IcannReportingUploadAction.PATH,
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -107,7 +107,7 @@ public final class TransactionsReportingQueryBuilder implements QueryBuilder {
queriesBuilder.put(
getTableName(TRANSACTION_TRANSFER_LOSING, yearMonth), transactionTransferLosingQuery);
// App Engine log table suffixes use YYYYMMDD format
// Log table suffixes use YYYYMMDD format
DateTimeFormatter logTableFormatter = DateTimeFormat.forPattern("yyyyMMdd");
String attemptedAddsQuery =
SqlTemplate.create(getQueryFromFile(ATTEMPTED_ADDS + ".sql"))
@@ -32,7 +32,6 @@ import google.registry.config.RegistryConfig.Config;
import google.registry.keyring.api.KeyModule.Key;
import google.registry.reporting.ReportingModule;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
@@ -50,7 +49,7 @@ import org.joda.time.LocalDate;
* generates the specified month's Spec11 report and stores it on GCS.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = GenerateSpec11ReportAction.PATH,
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -38,7 +38,6 @@ import google.registry.config.RegistryConfig.Config;
import google.registry.reporting.ReportingModule;
import google.registry.reporting.spec11.soy.Spec11EmailSoyInfo;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
@@ -57,7 +56,7 @@ import org.json.JSONException;
* ImmutableSet)} on success or {@link Spec11EmailUtils#sendAlertEmail(String, String)} on failure.
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = PublishSpec11ReportAction.PATH,
method = POST,
auth = Auth.AUTH_ADMIN)
@@ -14,7 +14,6 @@
package google.registry.request;
import static com.google.common.base.Preconditions.checkState;
import google.registry.config.RegistryConfig;
import google.registry.request.auth.Auth;
@@ -38,46 +37,7 @@ public @interface Action {
DELETE
}
interface Service {
String getServiceId();
URL getServiceUrl();
}
enum GaeService implements Service {
BSA("bsa"),
DEFAULT("default"),
TOOLS("tools"),
BACKEND("backend"),
PUBAPI("pubapi");
private final String serviceId;
GaeService(String serviceId) {
this.serviceId = serviceId;
}
@Override
public String getServiceId() {
return serviceId;
}
@Override
public URL getServiceUrl() {
return switch (this) {
case DEFAULT -> RegistryConfig.getDefaultServer();
case TOOLS -> RegistryConfig.getToolsServer();
case BACKEND -> RegistryConfig.getBackendServer();
case BSA -> RegistryConfig.getBsaServer();
case PUBAPI -> RegistryConfig.getPubapiServer();
};
}
}
enum GkeService implements Service {
// This designation means that it defers to the GAE service, so we don't have to annotate EVERY
// action during the GKE migration.
SAME_AS_GAE("same_as_gae"),
enum Service {
FRONTEND("frontend"),
BACKEND("backend"),
PUBAPI("pubapi"),
@@ -85,27 +45,21 @@ public @interface Action {
private final String serviceId;
GkeService(String serviceId) {
Service(String serviceId) {
this.serviceId = serviceId;
}
@Override
public String getServiceId() {
checkState(this != SAME_AS_GAE, "Cannot get service Id for SAME_AS_GAE");
return serviceId;
}
@Override
public URL getServiceUrl() {
return RegistryConfig.getServiceUrl(this);
}
}
/** Which App Engine service this action lives on. */
GaeService service();
/** Which GKE service this action lives on. */
GkeService gkeService() default GkeService.SAME_AS_GAE;
Service service();
/** HTTP path to serve the action from. The path components must be percent-escaped. */
String path();
@@ -127,22 +81,4 @@ public @interface Action {
/** Authentication settings. */
Auth auth();
// TODO(jianglai): Use Action.gkeService() directly once we are off GAE.
class ServiceGetter {
public static GkeService get(Action action) {
GkeService service = action.gkeService();
if (service != GkeService.SAME_AS_GAE) {
return service;
}
GaeService gaeService = action.service();
return switch (gaeService) {
case DEFAULT -> GkeService.FRONTEND;
case BACKEND -> GkeService.BACKEND;
case TOOLS -> GkeService.BACKEND;
case BSA -> GkeService.BACKEND;
case PUBAPI -> GkeService.PUBAPI;
};
}
}
}
@@ -73,8 +73,8 @@ public abstract class HttpException extends RuntimeException {
/**
* Exception that causes a 204 response.
*
* <p>This is useful for App Engine task queue handlers that want to display an error, but don't
* want the task to automatically retry, since the status code is less than 300.
* <p>This is useful for task queue handlers that want to display an error, but don't want the
* task to automatically retry, since the status code is less than 300.
*/
public static final class NoContentException extends HttpException {
public NoContentException(String message) {
@@ -25,7 +25,7 @@ import java.net.HttpURLConnection;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
/** Dagger modules for App Engine services and other vendor classes. */
/** Dagger modules for services and other vendor classes. */
public final class Modules {
/** Dagger module for {@link UrlConnectionService}. */
@@ -23,7 +23,7 @@ import static jakarta.servlet.http.HttpServletResponse.SC_NOT_FOUND;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryConfig;
import google.registry.request.Action.GkeService;
import google.registry.request.Action.Service;
import google.registry.request.auth.AuthResult;
import google.registry.request.auth.RequestAuthenticator;
import google.registry.util.NonFinalForTesting;
@@ -139,21 +139,20 @@ public class RequestHandler<C> {
rsp.sendError(SC_NOT_FOUND);
return;
}
if (RegistryEnvironment.isOnJetty()) {
GkeService service = Action.ServiceGetter.get(route.get().action());
String expectedDomain = RegistryConfig.getServiceUrl(service).getHost();
String actualDomain = req.getServerName();
// If the request doesn't come from GKE readiness prober
String maybeUserAgent = Optional.ofNullable(req.getHeader("User-Agent")).orElse("");
if (!maybeUserAgent.startsWith("kube-probe")
// If the hostname is "localhost", it must have come from the sidecar proxy.
&& !Objects.equals("localhost", actualDomain)
&& !Objects.equals(actualDomain, expectedDomain)) {
logger.atWarning().log(
"Actual domain %s does not match expected domain %s", actualDomain, expectedDomain);
rsp.sendError(SC_NOT_FOUND);
return;
}
Service service = route.get().action().service();
String expectedDomain = RegistryConfig.getServiceUrl(service).getHost();
String actualDomain = req.getServerName();
// If the request doesn't come from GKE readiness prober
String maybeUserAgent = Optional.ofNullable(req.getHeader("User-Agent")).orElse("");
if (!maybeUserAgent.startsWith("kube-probe")
// If the hostname is "localhost", it must have come from the sidecar proxy.
&& !Objects.equals("localhost", actualDomain)
&& !Objects.equals(actualDomain, expectedDomain)
&& !RegistryEnvironment.isInTestServer()) {
logger.atWarning().log(
"Actual domain %s does not match expected domain %s", actualDomain, expectedDomain);
rsp.sendError(SC_NOT_FOUND);
return;
}
if (!route.get().isMethodAllowed(method)) {
logger.atWarning().log("Method %s not allowed for: %s", method, path);
@@ -69,7 +69,7 @@ final class Router {
ImmutableSortedMap.Builder<String, Route> routes =
new ImmutableSortedMap.Builder<>(Ordering.natural());
for (Method method : componentClass.getMethods()) {
// Make App Engine's security manager happy.
// Make the security manager happy.
method.setAccessible(true);
if (!isDaggerInstantiatorOfType(Runnable.class, method)) {
continue;
@@ -27,12 +27,12 @@ import java.util.Map;
/**
* Utility class to help in dumping routing maps.
*
* <p>Each of the App Engine services (frontend, backend, and tools) has a Dagger component used for
* routing requests (e.g., FrontendRequestComponent). This class produces a text file representation
* of the routing configuration, showing what paths map to what action classes, as well as the
* properties of the action classes' annotations (which cover things like allowable HTTP methods,
* authentication settings, etc.). The text file can be useful for documentation, and is also used
* in unit tests to check against golden routing maps to find possibly unexpected changes.
* <p>The request-handling service has a Dagger component (RequestComponent) used for routing
* requests. This class produces a text file representation of the routing configuration, showing
* what paths map to what action classes, as well as the properties of the action classes'
* annotations (which cover things like allowable HTTP methods, authentication settings, etc.). The
* text file can be useful for documentation, and is also used in unit tests to check against golden
* routing maps to find possibly unexpected changes.
*
* <p>The file has fixed-width columns with a header row. The width of the columns is determined by
* the content to be displayed. The columns are:
@@ -94,7 +94,7 @@ public class RouterDisplayHelper {
private static String routeToString(Route route, String formatString) {
return String.format(
formatString,
Action.ServiceGetter.get(route.action()).name(),
route.action().service().name(),
route.action().isPrefix() ? (route.action().path() + "(*)") : route.action().path(),
route.actionClass().getSimpleName(),
Joiner.on(",").join(route.action().method()),
@@ -112,7 +112,7 @@ public class RouterDisplayHelper {
int methodsWidth = 7;
int minLevelWidth = 3;
for (Route route : routes) {
int len = Action.ServiceGetter.get(route.action()).name().length();
int len = route.action().service().name().length();
if (len > serviceWidth) {
serviceWidth = len;
}
@@ -148,9 +148,7 @@ public class RouterDisplayHelper {
return headerToString(formatString)
+ String.format("%n")
+ Streams.stream(routes)
.sorted(
Comparator.comparing(
(Route route) -> Action.ServiceGetter.get(route.action()).ordinal()))
.sorted(Comparator.comparing((Route route) -> route.action().service().ordinal()))
.map(route -> routeToString(route, formatString))
.collect(joining(String.format("%n")));
}
@@ -54,10 +54,7 @@ public class AuthModule {
// See https://cloud.google.com/iap/docs/signed-headers-howto#securing_iap_headers.
public static final String IAP_HEADER_NAME = "X-Goog-IAP-JWT-Assertion";
public static final String BEARER_PREFIX = "Bearer ";
// TODO (jianglai): Only use GKE audience once we are fully migrated to GKE.
// See: https://cloud.google.com/iap/docs/signed-headers-howto#verifying_the_jwt_payload
private static final String IAP_GAE_AUDIENCE_FORMAT = "/projects/%d/apps/%s";
private static final String IAP_GKE_AUDIENCE_FORMAT = "/projects/%d/global/backendServices/%d";
private static final String IAP_AUDIENCE_FORMAT = "/projects/%d/global/backendServices/%d";
private static final String IAP_ISSUER_URL = "https://cloud.google.com/iap";
private static final String REGULAR_ISSUER_URL = "https://accounts.google.com";
// The backend service IDs created when setting up GKE routes. They will be included in the
@@ -89,24 +86,18 @@ public class AuthModule {
@IapOidc
@Singleton
TokenVerifier provideIapTokenVerifier(
@Config("projectId") String projectId,
@Config("projectIdNumber") long projectIdNumber,
@Named("backendServiceIdMap") Supplier<ImmutableMap<String, Long>> backendServiceIdMap) {
com.google.auth.oauth2.TokenVerifier.Builder tokenVerifierBuilder =
com.google.auth.oauth2.TokenVerifier.newBuilder().setIssuer(IAP_ISSUER_URL);
return (String service, String token) -> {
String audience;
if (RegistryEnvironment.isOnJetty()) {
Long backendServiceId = backendServiceIdMap.get().get(service);
checkNotNull(
backendServiceId,
"Backend service ID not found for service: %s, available IDs are %s",
service,
backendServiceIdMap);
audience = String.format(IAP_GKE_AUDIENCE_FORMAT, projectIdNumber, backendServiceId);
} else {
audience = String.format(IAP_GAE_AUDIENCE_FORMAT, projectIdNumber, projectId);
}
Long backendServiceId = backendServiceIdMap.get().get(service);
checkNotNull(
backendServiceId,
"Backend service ID not found for service: %s, available IDs are %s",
service,
backendServiceIdMap);
String audience = String.format(IAP_AUDIENCE_FORMAT, projectIdNumber, backendServiceId);
return tokenVerifierBuilder.setAudience(audience).build().verify(token);
};
}
@@ -86,13 +86,10 @@ public abstract class OidcTokenAuthenticationMechanism implements Authentication
}
JsonWebSignature token = null;
try {
String service = null;
if (RegistryEnvironment.isOnJetty()) {
String hostname = request.getServerName();
service = Splitter.on('.').split(hostname).iterator().next();
if (request.getHeader("canary") != null) {
service += "-canary";
}
String hostname = request.getServerName();
String service = Splitter.on('.').split(hostname).iterator().next();
if (request.getHeader("canary") != null) {
service += "-canary";
}
token = tokenVerifier.verify(service, rawIdToken);
} catch (Exception e) {
@@ -0,0 +1,177 @@
# URL: https://www.iana.org/domains/idn-tables/tables/google_latn_1.0.txt
# Policy: https://www.registry.google/about/policies/domainabuse/
U+002D # HYPHEN-MINUS
U+0030 # DIGIT ZERO
U+0031 # DIGIT ONE
U+0032 # DIGIT TWO
U+0033 # DIGIT THREE
U+0034 # DIGIT FOUR
U+0035 # DIGIT FIVE
U+0036 # DIGIT SIX
U+0037 # DIGIT SEVEN
U+0038 # DIGIT EIGHT
U+0039 # DIGIT NINE
U+0061 # LATIN SMALL LETTER A
U+0062 # LATIN SMALL LETTER B
U+0063 # LATIN SMALL LETTER C
U+0064 # LATIN SMALL LETTER D
U+0065 # LATIN SMALL LETTER E
U+0066 # LATIN SMALL LETTER F
U+0067 # LATIN SMALL LETTER G
U+0068 # LATIN SMALL LETTER H
U+0069 # LATIN SMALL LETTER I
U+006A # LATIN SMALL LETTER J
U+006B # LATIN SMALL LETTER K
U+006C # LATIN SMALL LETTER L
U+006D # LATIN SMALL LETTER M
U+006E # LATIN SMALL LETTER N
U+006F # LATIN SMALL LETTER O
U+0070 # LATIN SMALL LETTER P
U+0071 # LATIN SMALL LETTER Q
U+0072 # LATIN SMALL LETTER R
U+0073 # LATIN SMALL LETTER S
U+0074 # LATIN SMALL LETTER T
U+0075 # LATIN SMALL LETTER U
U+0076 # LATIN SMALL LETTER V
U+0077 # LATIN SMALL LETTER W
U+0078 # LATIN SMALL LETTER X
U+0079 # LATIN SMALL LETTER Y
U+007A # LATIN SMALL LETTER Z
U+00E0 # LATIN SMALL LETTER A WITH GRAVE
U+00E2 # LATIN SMALL LETTER A WITH CIRCUMFLEX
U+00E3 # LATIN SMALL LETTER A WITH TILDE
U+00E4 # LATIN SMALL LETTER A WITH DIAERESIS
U+00E5 # LATIN SMALL LETTER A WITH RING ABOVE
U+00E6 # LATIN SMALL LETTER AE
U+00E7 # LATIN SMALL LETTER C WITH CEDILLA
U+00E8 # LATIN SMALL LETTER E WITH GRAVE
U+00E9 # LATIN SMALL LETTER E WITH ACUTE
U+00EA # LATIN SMALL LETTER E WITH CIRCUMFLEX
U+00EB # LATIN SMALL LETTER E WITH DIAERESIS
U+00EC # LATIN SMALL LETTER I WITH GRAVE
U+00EE # LATIN SMALL LETTER I WITH CIRCUMFLEX
U+00F0 # LATIN SMALL LETTER ETH
U+00F2 # LATIN SMALL LETTER O WITH GRAVE
U+00F4 # LATIN SMALL LETTER O WITH CIRCUMFLEX
U+00F5 # LATIN SMALL LETTER O WITH TILDE
U+00F6 # LATIN SMALL LETTER O WITH DIAERESIS
U+00F8 # LATIN SMALL LETTER O WITH STROKE
U+00F9 # LATIN SMALL LETTER U WITH GRAVE
U+00FB # LATIN SMALL LETTER U WITH CIRCUMFLEX
U+00FD # LATIN SMALL LETTER Y WITH ACUTE
U+00FE # LATIN SMALL LETTER THORN
U+00FF # LATIN SMALL LETTER Y WITH DIAERESIS
U+0103 # LATIN SMALL LETTER A WITH BREVE
U+0105 # LATIN SMALL LETTER A WITH OGONEK
U+0107 # LATIN SMALL LETTER C WITH ACUTE
U+0109 # LATIN SMALL LETTER C WITH CIRCUMFLEX
U+010D # LATIN SMALL LETTER C WITH CARON
U+010F # LATIN SMALL LETTER D WITH CARON
U+0111 # LATIN SMALL LETTER D WITH STROKE
U+0113 # LATIN SMALL LETTER E WITH MACRON
U+0117 # LATIN SMALL LETTER E WITH DOT ABOVE
U+0119 # LATIN SMALL LETTER E WITH OGONEK
U+011B # LATIN SMALL LETTER E WITH CARON
U+011D # LATIN SMALL LETTER G WITH CIRCUMFLEX
U+011F # LATIN SMALL LETTER G WITH BREVE
U+0123 # LATIN SMALL LETTER G WITH CEDILLA
U+0125 # LATIN SMALL LETTER H WITH CIRCUMFLEX
U+0127 # LATIN SMALL LETTER H WITH STROKE
U+012B # LATIN SMALL LETTER I WITH MACRON
U+012F # LATIN SMALL LETTER I WITH OGONEK
U+0135 # LATIN SMALL LETTER J WITH CIRCUMFLEX
U+0137 # LATIN SMALL LETTER K WITH CEDILLA
U+013A # LATIN SMALL LETTER L WITH ACUTE
U+013C # LATIN SMALL LETTER L WITH CEDILLA
U+013E # LATIN SMALL LETTER L WITH CARON
U+0142 # LATIN SMALL LETTER L WITH STROKE
U+0146 # LATIN SMALL LETTER N WITH CEDILLA
U+0148 # LATIN SMALL LETTER N WITH CARON
U+0151 # LATIN SMALL LETTER O WITH DOUBLE ACUTE
U+0153 # LATIN SMALL LIGATURE OE
U+0155 # LATIN SMALL LETTER R WITH ACUTE
U+0159 # LATIN SMALL LETTER R WITH CARON
U+015B # LATIN SMALL LETTER S WITH ACUTE
U+015D # LATIN SMALL LETTER S WITH CIRCUMFLEX
U+015F # LATIN SMALL LETTER S WITH CEDILLA
U+0161 # LATIN SMALL LETTER S WITH CARON
U+0165 # LATIN SMALL LETTER T WITH CARON
U+0167 # LATIN SMALL LETTER T WITH STROKE
U+016B # LATIN SMALL LETTER U WITH MACRON
U+016D # LATIN SMALL LETTER U WITH BREVE
U+016F # LATIN SMALL LETTER U WITH RING ABOVE
U+0171 # LATIN SMALL LETTER U WITH DOUBLE ACUTE
U+0173 # LATIN SMALL LETTER U WITH OGONEK
U+0175 # LATIN SMALL LETTER W WITH CIRCUMFLEX
U+0177 # LATIN SMALL LETTER Y WITH CIRCUMFLEX
U+017A # LATIN SMALL LETTER Z WITH ACUTE
U+017E # LATIN SMALL LETTER Z WITH CARON
U+0188 # LATIN SMALL LETTER C WITH HOOK
U+0199 # LATIN SMALL LETTER K WITH HOOK
U+01A1 # LATIN SMALL LETTER O WITH HORN
U+01A5 # LATIN SMALL LETTER P WITH HOOK
U+01AD # LATIN SMALL LETTER T WITH HOOK
U+01B0 # LATIN SMALL LETTER U WITH HORN
U+01B4 # LATIN SMALL LETTER Y WITH HOOK
U+01E9 # LATIN SMALL LETTER K WITH CARON
U+01EF # LATIN SMALL LETTER EZH WITH CARON
U+0219 # LATIN SMALL LETTER S WITH COMMA BELOW
U+021B # LATIN SMALL LETTER T WITH COMMA BELOW
U+024D # LATIN SMALL LETTER R WITH STROKE
U+0253 # LATIN SMALL LETTER B WITH HOOK
U+0254 # LATIN SMALL LETTER OPEN O
U+0256 # LATIN SMALL LETTER D WITH TAIL
U+0257 # LATIN SMALL LETTER D WITH HOOK
U+0259 # LATIN SMALL LETTER SCHWA
U+025B # LATIN SMALL LETTER OPEN E
U+0260 # LATIN SMALL LETTER G WITH HOOK
U+0268 # LATIN SMALL LETTER I WITH STROKE
U+0272 # LATIN SMALL LETTER N WITH LEFT HOOK
U+0289 # LATIN SMALL LETTER U BAR
U+0292 # LATIN SMALL LETTER EZH
U+1E13 # LATIN SMALL LETTER D WITH CIRCUMFLEX BELOW
U+1E3D # LATIN SMALL LETTER L WITH CIRCUMFLEX BELOW
U+1E49 # LATIN SMALL LETTER N WITH LINE BELOW
U+1E4B # LATIN SMALL LETTER N WITH CIRCUMFLEX BELOW
U+1E63 # LATIN SMALL LETTER S WITH DOT BELOW
U+1E6D # LATIN SMALL LETTER T WITH DOT BELOW
U+1E71 # LATIN SMALL LETTER T WITH CIRCUMFLEX BELOW
U+1E8D # LATIN SMALL LETTER X WITH DIAERESIS
U+1EA1 # LATIN SMALL LETTER A WITH DOT BELOW
U+1EA5 # LATIN SMALL LETTER A WITH CIRCUMFLEX AND ACUTE
U+1EA7 # LATIN SMALL LETTER A WITH CIRCUMFLEX AND GRAVE
U+1EA9 # LATIN SMALL LETTER A WITH CIRCUMFLEX AND HOOK ABOVE
U+1EAB # LATIN SMALL LETTER A WITH CIRCUMFLEX AND TILDE
U+1EAD # LATIN SMALL LETTER A WITH CIRCUMFLEX AND DOT BELOW
U+1EAF # LATIN SMALL LETTER A WITH BREVE AND ACUTE
U+1EB1 # LATIN SMALL LETTER A WITH BREVE AND GRAVE
U+1EB3 # LATIN SMALL LETTER A WITH BREVE AND HOOK ABOVE
U+1EB5 # LATIN SMALL LETTER A WITH BREVE AND TILDE
U+1EB7 # LATIN SMALL LETTER A WITH BREVE AND DOT BELOW
U+1EB9 # LATIN SMALL LETTER E WITH DOT BELOW
U+1EBB # LATIN SMALL LETTER E WITH HOOK ABOVE
U+1EBF # LATIN SMALL LETTER E WITH CIRCUMFLEX AND ACUTE
U+1EC1 # LATIN SMALL LETTER E WITH CIRCUMFLEX AND GRAVE
U+1EC3 # LATIN SMALL LETTER E WITH CIRCUMFLEX AND HOOK ABOVE
U+1EC5 # LATIN SMALL LETTER E WITH CIRCUMFLEX AND TILDE
U+1EC7 # LATIN SMALL LETTER E WITH CIRCUMFLEX AND DOT BELOW
U+1ECB # LATIN SMALL LETTER I WITH DOT BELOW
U+1ECD # LATIN SMALL LETTER O WITH DOT BELOW
U+1ED1 # LATIN SMALL LETTER O WITH CIRCUMFLEX AND ACUTE
U+1ED3 # LATIN SMALL LETTER O WITH CIRCUMFLEX AND GRAVE
U+1ED5 # LATIN SMALL LETTER O WITH CIRCUMFLEX AND HOOK ABOVE
U+1ED7 # LATIN SMALL LETTER O WITH CIRCUMFLEX AND TILDE
U+1ED9 # LATIN SMALL LETTER O WITH CIRCUMFLEX AND DOT BELOW
U+1EDB # LATIN SMALL LETTER O WITH HORN AND ACUTE
U+1EDD # LATIN SMALL LETTER O WITH HORN AND GRAVE
U+1EDF # LATIN SMALL LETTER O WITH HORN AND HOOK ABOVE
U+1EE1 # LATIN SMALL LETTER O WITH HORN AND TILDE
U+1EE3 # LATIN SMALL LETTER O WITH HORN AND DOT BELOW
U+1EE5 # LATIN SMALL LETTER U WITH DOT BELOW
U+1EE9 # LATIN SMALL LETTER U WITH HORN AND ACUTE
U+1EEB # LATIN SMALL LETTER U WITH HORN AND GRAVE
U+1EED # LATIN SMALL LETTER U WITH HORN AND HOOK ABOVE
U+1EEF # LATIN SMALL LETTER U WITH HORN AND TILDE
U+1EF1 # LATIN SMALL LETTER U WITH HORN AND DOT BELOW
U+1EF5 # LATIN SMALL LETTER Y WITH DOT BELOW
U+1EF9 # LATIN SMALL LETTER Y WITH TILDE
@@ -37,7 +37,6 @@ import google.registry.batch.CloudTasksUtils;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.domain.Domain;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Parameter;
import google.registry.request.RequestParameters;
import google.registry.request.UrlConnectionService;
@@ -66,7 +65,7 @@ import org.joda.time.Duration;
* @see NordnVerifyAction
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = NordnUploadAction.PATH,
method = Action.Method.POST,
automaticallyPrintOk = true,
@@ -23,7 +23,6 @@ import com.google.common.annotations.VisibleForTesting;
import com.google.common.flogger.FluentLogger;
import com.google.common.io.ByteSource;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.HttpException.ConflictException;
import google.registry.request.Parameter;
import google.registry.request.RequestParameters;
@@ -51,7 +50,7 @@ import java.util.Map.Entry;
* http://tools.ietf.org/html/draft-lozano-tmch-func-spec-08#section-5.2.3.3</a>
*/
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = NordnVerifyAction.PATH,
method = Action.Method.POST,
automaticallyPrintOk = true,
@@ -113,7 +112,7 @@ public final class NordnVerifyAction implements Runnable {
logger.atInfo().log(
"LORDN verify task %s response: HTTP response code %d", actionLogId, responseCode);
if (responseCode == SC_NO_CONTENT) {
// Send a 400+ status code so App Engine will retry the task.
// Send a 400+ status code so Cloud Tasks will retry the task.
throw new ConflictException("Not ready");
}
if (responseCode != SC_OK) {
@@ -19,7 +19,6 @@ import static java.nio.charset.StandardCharsets.UTF_8;
import google.registry.config.RegistryConfig.Config;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.auth.Auth;
import jakarta.inject.Inject;
import java.io.IOException;
@@ -29,7 +28,7 @@ import java.util.Optional;
/** Action to download the latest ICANN TMCH CRL from MarksDB. */
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = "/_dr/task/tmchCrl",
method = POST,
automaticallyPrintOk = true,
@@ -22,7 +22,6 @@ import google.registry.keyring.api.KeyModule.Key;
import google.registry.model.tmch.ClaimsList;
import google.registry.model.tmch.ClaimsListDao;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.auth.Auth;
import jakarta.inject.Inject;
import java.io.IOException;
@@ -32,7 +31,7 @@ import org.bouncycastle.openpgp.PGPException;
/** Action to download the latest domain name list (aka claims list) from MarksDB. */
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = "/_dr/task/tmchDnl",
method = POST,
automaticallyPrintOk = true,
@@ -22,7 +22,6 @@ import google.registry.keyring.api.KeyModule.Key;
import google.registry.model.smd.SignedMarkRevocationList;
import google.registry.model.smd.SignedMarkRevocationListDao;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.auth.Auth;
import jakarta.inject.Inject;
import java.io.IOException;
@@ -32,7 +31,7 @@ import org.bouncycastle.openpgp.PGPException;
/** Action to download the latest signed mark revocation list from MarksDB. */
@Action(
service = GaeService.BACKEND,
service = Action.Service.BACKEND,
path = "/_dr/task/tmchSmdrl",
method = POST,
automaticallyPrintOk = true,
@@ -21,6 +21,8 @@ import com.beust.jcommander.Parameters;
import com.google.api.services.dns.Dns;
import com.google.api.services.dns.model.ManagedZone;
import com.google.api.services.dns.model.ManagedZoneDnsSecConfig;
import com.google.common.base.Joiner;
import com.google.common.base.Splitter;
import google.registry.config.RegistryConfig.Config;
import jakarta.inject.Inject;
import java.io.IOException;
@@ -52,6 +54,13 @@ final class CreateCdnsTld extends ConfirmingCommand {
description = "In Sandbox, skip the dns_name format check.")
boolean skipSandboxTldCheck;
@Parameter(
names = "--use_prod_name_servers_in_sandbox",
description =
"In Sandbox, create zone on the production name servers, e.g., for ICANN tests. "
+ "Ignored in other environments.")
boolean useProdNameServersInSandbox;
@Inject
@Config("projectId")
String projectId;
@@ -77,15 +86,25 @@ final class CreateCdnsTld extends ConfirmingCommand {
}
}
String nameServerSetName;
if (RegistryToolEnvironment.get().equals(RegistryToolEnvironment.PRODUCTION)) {
nameServerSetName = "cloud-dns-registry";
} else if (RegistryToolEnvironment.get().equals(RegistryToolEnvironment.SANDBOX)
&& useProdNameServersInSandbox) {
nameServerSetName = "cloud-dns-registry";
} else {
nameServerSetName = "cloud-dns-registry-test";
}
managedZone =
new ManagedZone()
.setDescription(description)
.setNameServerSet(
RegistryToolEnvironment.get() == RegistryToolEnvironment.PRODUCTION
? "cloud-dns-registry"
: "cloud-dns-registry-test")
.setNameServerSet(nameServerSetName)
.setDnsName(dnsName)
.setName((name != null) ? name : dnsName)
.setName(
(name != null)
? name
: Joiner.on('_').join(Splitter.on('.').omitEmptyStrings().split(dnsName)))
.setDnssecConfig(new ManagedZoneDnsSecConfig().setNonExistence("nsec").setState("on"));
}
@@ -28,8 +28,6 @@ import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.net.MediaType;
import google.registry.config.RegistryConfig.Config;
import google.registry.request.Action.GaeService;
import google.registry.request.Action.GkeService;
import google.registry.request.Action.Service;
import jakarta.inject.Inject;
import java.util.List;
@@ -80,10 +78,6 @@ class CurlCommand implements CommandWithConnection {
required = true)
private String serviceName;
@Inject
@Config("useGke")
boolean useGke;
@Inject
@Config("useCanary")
boolean useCanary;
@@ -103,11 +97,7 @@ class CurlCommand implements CommandWithConnection {
throw new IllegalArgumentException("You may not specify a body for a get method.");
}
Service service =
useGke
? GkeService.valueOf(Ascii.toUpperCase(serviceName))
: GaeService.valueOf(Ascii.toUpperCase(serviceName));
Service service = Service.valueOf(Ascii.toUpperCase(serviceName));
ServiceConnection connectionToService = connection.withService(service, useCanary);
String response =
(method == Method.GET)
@@ -0,0 +1,100 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.tools;
import static com.google.common.base.Preconditions.checkState;
import com.beust.jcommander.Parameter;
import com.beust.jcommander.Parameters;
import com.google.common.collect.ImmutableMap;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonElement;
import com.google.gson.JsonParser;
import google.registry.config.RegistryConfig.Config;
import google.registry.request.Action.Service;
import jakarta.inject.Inject;
import java.io.IOException;
import java.util.Optional;
import javax.annotation.Nullable;
/** Command to manually perform an authenticated RDAP query. */
@Parameters(separators = " =", commandDescription = "Manually perform an authenticated RDAP query")
public final class RdapQueryCommand implements CommandWithConnection {
private static final Gson GSON = new GsonBuilder().setPrettyPrinting().create();
enum RdapQueryType {
DOMAIN("/rdap/domain/%s"),
DOMAIN_SEARCH("/rdap/domains", "name"),
NAMESERVER("/rdap/nameserver/%s"),
NAMESERVER_SEARCH("/rdap/nameservers", "name"),
ENTITY("/rdap/entity/%s"),
ENTITY_SEARCH("/rdap/entities", "fn");
private final String pathFormat;
private final Optional<String> searchParamKey;
RdapQueryType(String pathFormat) {
this(pathFormat, null);
}
RdapQueryType(String pathFormat, @Nullable String searchParamKey) {
this.pathFormat = pathFormat;
this.searchParamKey = Optional.ofNullable(searchParamKey);
}
String getQueryPath(String queryTerm) {
return String.format(pathFormat, queryTerm);
}
ImmutableMap<String, String> getQueryParameters(String queryTerm) {
return searchParamKey.map(key -> ImmutableMap.of(key, queryTerm)).orElse(ImmutableMap.of());
}
}
@Parameter(names = "--type", description = "The type of RDAP query to perform.", required = true)
private RdapQueryType type;
@Parameter(
description = "The main query term (e.g., a domain name or search pattern).",
required = true)
private String queryTerm;
@Inject ServiceConnection defaultConnection;
@Inject
@Config("useCanary")
boolean useCanary;
@Override
public void setConnection(ServiceConnection connection) {
this.defaultConnection = connection;
}
@Override
public void run() throws IOException {
checkState(defaultConnection != null, "ServiceConnection was not set by RegistryCli.");
String path = type.getQueryPath(queryTerm);
ImmutableMap<String, String> queryParams = type.getQueryParameters(queryTerm);
ServiceConnection pubapiConnection = defaultConnection.withService(Service.PUBAPI, useCanary);
String rdapResponse = pubapiConnection.sendGetRequest(path, queryParams);
JsonElement rdapJson = JsonParser.parseString(rdapResponse);
System.out.println(GSON.toJson(rdapJson));
}
}
@@ -70,9 +70,6 @@ final class RegistryCli implements CommandRunner {
+ "Beam pipelines")
private String sqlAccessInfoFile = null;
@Parameter(names = "--gae", description = "Whether to use GAE runtime, instead of GKE")
private boolean useGae = false;
@Parameter(names = "--canary", description = "Whether to connect to the canary instances")
private boolean useCanary = false;
@@ -169,7 +166,6 @@ final class RegistryCli implements CommandRunner {
DaggerRegistryToolComponent.builder()
.credentialFilePath(credentialJson)
.sqlAccessInfoFile(sqlAccessInfoFile)
.useGke(!useGae)
.useCanary(useCanary)
.build();
@@ -203,7 +199,8 @@ final class RegistryCli implements CommandRunner {
"""
This error is likely the result of having another instance of
nomulus running at the same time. Check your system, shut down
the other instance, and try again.""");
the other instance, and try again.\
""");
System.err.println("===================================================================");
} else {
throw e;
@@ -213,7 +210,7 @@ final class RegistryCli implements CommandRunner {
}
private ServiceConnection getConnection() {
// Get the App Engine connection, advise the user if they are not currently logged in.
// Get the service connection, advise the user if they are not currently logged in.
if (connection == null) {
connection = component.serviceConnection();
}
@@ -98,6 +98,7 @@ public final class RegistryTool {
.put("login", LoginCommand.class)
.put("logout", LogoutCommand.class)
.put("pending_escrow", PendingEscrowCommand.class)
.put("rdap_query", RdapQueryCommand.class)
.put("recreate_billing_recurrences", RecreateBillingRecurrencesCommand.class)
.put("registrar_poc", RegistrarPocCommand.class)
.put("renew_domain", RenewDomainCommand.class)

Some files were not shown because too many files have changed in this diff Show More