1
0
mirror of https://github.com/google/nomulus synced 2026-07-12 19:12:35 +00:00

Compare commits

..

15 Commits

Author SHA1 Message Date
Pavlo Tkach 55243e7cf6 Adds cloud scheduler and tasks deployer (#1999) 2023-05-04 15:57:32 -04:00
Lai Jiang e14764b4c8 Remove DNS pull queue (#2000)
This is the last dependency on GAE pull queue, therefore we can delete
the pull queue config from queue.xml as well.
2023-05-04 13:21:53 -04:00
dependabot[bot] 68810f7a30 Bump engine.io and socket.io in /console-webapp (#2022)
Bumps [engine.io](https://github.com/socketio/engine.io) and [socket.io](https://github.com/socketio/socket.io). These dependencies needed to be updated together.

Updates `engine.io` from 6.2.1 to 6.4.2
- [Release notes](https://github.com/socketio/engine.io/releases)
- [Changelog](https://github.com/socketio/engine.io/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/engine.io/compare/6.2.1...6.4.2)

Updates `socket.io` from 4.5.2 to 4.6.1
- [Release notes](https://github.com/socketio/socket.io/releases)
- [Changelog](https://github.com/socketio/socket.io/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/socket.io/compare/4.5.2...4.6.1)

---
updated-dependencies:
- dependency-name: engine.io
  dependency-type: indirect
- dependency-name: socket.io
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-04 12:50:19 -04:00
Ben McIlwain 14d245b1e3 Remove duplicate info from create/update reserved list command output (#2020)
It was repeating the domain label twice for every reserved list entry. It used
to look like this:

baddies=baddies,FULLY_BLOCKED
2023-05-03 17:31:23 -04:00
Weimin Yu 61ab29ae9e Prober ssl cert update automation (#2019)
Defined CloudBuild script and docker image that automatically
updates probers' SSL certs
2023-05-03 15:57:50 -04:00
Weimin Yu 6742e5bf23 Remove CloudSql wipeout cron job in crash (#2017)
No more production data in crash. This allows us to repopulate crash
with test data.
2023-05-02 14:44:09 -04:00
Weimin Yu c7f69eba1d Prepare switch of credential annotation (#2014)
* Prepare switch of credential annotation

Prepare the switch from DefaultCredential to ApplicationCredential.

In nomulus tools, start using the new annotation. This is tested by
successfully using the nomulus curl command, which actually needs a
valid credential to work.

For remaining use cases of the old annotation in Nomulus server, add
some code that relies on the new credential to work. Once these code
are tested in sandbox and production, we will switch the annotations.
2023-05-01 11:23:19 -04:00
gbrodman 578988d5ea Don't allow a list of the empty string in List<String> fields (#2011)
If the user does, e.g. `--allowed_nameservers=` (or contact ids) that
shouldn't mean a list consisting solely of the empty string.

Using this parameter / converter allows us to ensure that lists of
strings look reasonable.
2023-04-28 17:59:17 -04:00
sarahcaseybot c17b8285f9 Don't apply non-premium default tokens to premium names (#2007)
* Don't apply non-premium default tokens to premium names

* Add test for renew

* Remove premium check from try/catch block

* Add check in validateToken

* Update docs

* Add validateForPremiums

* Better method name

* Shorten error message to fit as reason

* Add missing extension catch

* Remove extra javadoc

* Fix merge conflicts and change error message

* Update flow docs
2023-04-28 17:56:15 -04:00
gbrodman ff8a08f40e Fix typo in pipeline name (#2016) 2023-04-28 17:05:24 -04:00
gbrodman a341058282 Refactor / rename Billing object classes (#1993)
This includes renaming the billing classes to match the SQL table names,
as well as splitting them out into their own separate top-level classes.
The rest of the changes are mostly renaming variables and comments etc.

We now use `BillingBase` as the name of the common billing superclass,
because one-time events are called BillingEvents
2023-04-28 14:27:37 -04:00
Weimin Yu 16758879f0 Allow rotation when updating registrar cert (#2012)
* Allow rotation when updating registrar cert

When updating a registrar's primary cert, add a flag to activate
rotation of previous primary cert to failover.

This functionality is part of the prober ssl cert renewal automation.
2023-04-27 14:42:11 -04:00
Lai Jiang 2021247ab4 Update README on how to manually push schema (#2009) 2023-04-26 16:32:15 -04:00
Lai Jiang 4fc7038690 Make a few minor changes to make the linter happy (#2010)
<!-- Reviewable:start -->
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/2010)
<!-- Reviewable:end -->
2023-04-26 15:49:32 -04:00
Weimin Yu 9272e7fd14 Add a test of failover certificate (#2008)
Verifies that client can log in with correct failover certificate.
2023-04-26 15:47:47 -04:00
176 changed files with 3982 additions and 4853 deletions
+1 -1
View File
@@ -103,7 +103,7 @@ explodeWar.doLast {
file("${it.explodedAppDirectory}/WEB-INF/lib/tools.jar").setWritable(true)
}
appengineDeployAll.finalizedBy ':cloudSchedulerDeployer'
appengineDeployAll.finalizedBy ':deployCloudSchedulerAndQueue'
rootProject.deploy.dependsOn appengineDeployAll
rootProject.stage.dependsOn appengineStage
tasks['war'].dependsOn ':console-webapp:buildConsoleWebappProd'
+9 -3
View File
@@ -558,17 +558,23 @@ task coreDev {
javadocDependentTasks.each { tasks.javadoc.dependsOn(it) }
// Runs the script, which deploys cloud scheduler tasks based on the config
task cloudSchedulerDeployer {
// Runs the script, which deploys cloud scheduler and tasks based on the config
task deployCloudSchedulerAndQueue {
doLast {
def env = environment
if (!prodOrSandboxEnv) {
exec {
commandLine 'go', 'run',
"${rootDir}/release/builder/cloudSchedulerDeployer.go",
"${rootDir}/release/builder/deployCloudSchedulerAndQueue.go",
"${rootDir}/core/src/main/java/google/registry/env/${env}/default/WEB-INF/cloud-scheduler-tasks.xml",
"domain-registry-${env}"
}
exec {
commandLine 'go', 'run',
"${rootDir}/release/builder/deployCloudSchedulerAndQueue.go",
"${rootDir}/core/src/main/java/google/registry/env/common/default/WEB-INF/cloud-tasks-queue.xml",
"domain-registry-${env}"
}
}
}
}
+60 -48
View File
@@ -3898,10 +3898,13 @@
"dev": true
},
"node_modules/@types/cors": {
"version": "2.8.12",
"resolved": "https://registry.npmjs.org/@types/cors/-/cors-2.8.12.tgz",
"integrity": "sha512-vt+kDhq/M2ayberEtJcIN/hxXy1Pk+59g2FV/ZQceeaTyCtCucjL2Q7FXlFjtWn4n15KCr1NE2lNNFhp0lEThw==",
"dev": true
"version": "2.8.13",
"resolved": "https://registry.npmjs.org/@types/cors/-/cors-2.8.13.tgz",
"integrity": "sha512-RG8AStHlUiV5ysZQKq97copd2UmVYw3/pRMLefISZ3S1hK104Cwm7iLQ3fTKx+lsUH2CE8FlLaYeEA2LSeqYUA==",
"dev": true,
"dependencies": {
"@types/node": "*"
}
},
"node_modules/@types/eslint": {
"version": "8.4.6",
@@ -5907,9 +5910,9 @@
}
},
"node_modules/engine.io": {
"version": "6.2.1",
"resolved": "https://registry.npmjs.org/engine.io/-/engine.io-6.2.1.tgz",
"integrity": "sha512-ECceEFcAaNRybd3lsGQKas3ZlMVjN3cyWwMP25D2i0zWfyiytVbTpRPa34qrr+FHddtpBVOmq4H/DCv1O0lZRA==",
"version": "6.4.2",
"resolved": "https://registry.npmjs.org/engine.io/-/engine.io-6.4.2.tgz",
"integrity": "sha512-FKn/3oMiJjrOEOeUub2WCox6JhxBXq/Zn3fZOMCBxKnNYtsdKjxhl7yR3fZhM9PV+rdE75SU5SYMc+2PGzo+Tg==",
"dev": true,
"dependencies": {
"@types/cookie": "^0.4.1",
@@ -5921,16 +5924,16 @@
"cors": "~2.8.5",
"debug": "~4.3.1",
"engine.io-parser": "~5.0.3",
"ws": "~8.2.3"
"ws": "~8.11.0"
},
"engines": {
"node": ">=10.0.0"
}
},
"node_modules/engine.io-parser": {
"version": "5.0.4",
"resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-5.0.4.tgz",
"integrity": "sha512-+nVFp+5z1E3HcToEnO7ZIj3g+3k9389DvWtvJZz0T6/eOCPIyyxehFcedoYrZQrp0LgQbD9pPXhpMBKMd5QURg==",
"version": "5.0.6",
"resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-5.0.6.tgz",
"integrity": "sha512-tjuoZDMAdEhVnSFleYPCtdL2GXwVTGtNjoeJd9IhIG3C1xs9uwxqRNEu5WpnDZCaozwVlK/nuQhpodhXSIMaxw==",
"dev": true,
"engines": {
"node": ">=10.0.0"
@@ -10880,27 +10883,30 @@
}
},
"node_modules/socket.io": {
"version": "4.5.2",
"resolved": "https://registry.npmjs.org/socket.io/-/socket.io-4.5.2.tgz",
"integrity": "sha512-6fCnk4ARMPZN448+SQcnn1u8OHUC72puJcNtSgg2xS34Cu7br1gQ09YKkO1PFfDn/wyUE9ZgMAwosJed003+NQ==",
"version": "4.6.1",
"resolved": "https://registry.npmjs.org/socket.io/-/socket.io-4.6.1.tgz",
"integrity": "sha512-KMcaAi4l/8+xEjkRICl6ak8ySoxsYG+gG6/XfRCPJPQ/haCRIJBTL4wIl8YCsmtaBovcAXGLOShyVWQ/FG8GZA==",
"dev": true,
"dependencies": {
"accepts": "~1.3.4",
"base64id": "~2.0.0",
"debug": "~4.3.2",
"engine.io": "~6.2.0",
"socket.io-adapter": "~2.4.0",
"socket.io-parser": "~4.2.0"
"engine.io": "~6.4.1",
"socket.io-adapter": "~2.5.2",
"socket.io-parser": "~4.2.1"
},
"engines": {
"node": ">=10.0.0"
}
},
"node_modules/socket.io-adapter": {
"version": "2.4.0",
"resolved": "https://registry.npmjs.org/socket.io-adapter/-/socket.io-adapter-2.4.0.tgz",
"integrity": "sha512-W4N+o69rkMEGVuk2D/cvca3uYsvGlMwsySWV447y99gUPghxq42BxqLNMndb+a1mm/5/7NeXVQS7RLa2XyXvYg==",
"dev": true
"version": "2.5.2",
"resolved": "https://registry.npmjs.org/socket.io-adapter/-/socket.io-adapter-2.5.2.tgz",
"integrity": "sha512-87C3LO/NOMc+eMcpcxUBebGjkpMDkNBS9tf7KJqcDsmL936EChtVva71Dw2q4tQcuVC+hAUy4an2NO/sYXmwRA==",
"dev": true,
"dependencies": {
"ws": "~8.11.0"
}
},
"node_modules/socket.io-parser": {
"version": "4.2.1",
@@ -12219,9 +12225,9 @@
"dev": true
},
"node_modules/ws": {
"version": "8.2.3",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.2.3.tgz",
"integrity": "sha512-wBuoj1BDpC6ZQ1B7DWQBYVLphPWkm8i9Y0/3YdHjHKHiohOJ1ws+3OccDWtH+PoC9DZD5WOTrJvNbWvjS6JWaA==",
"version": "8.11.0",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.11.0.tgz",
"integrity": "sha512-HPG3wQd9sNQoT9xHyNCXoDUa+Xw/VevmY9FoHyQ+g+rrMn4j6FB4np7Z0OhdTgjx6MgQLK7jwSy1YecU1+4Asg==",
"dev": true,
"engines": {
"node": ">=10.0.0"
@@ -15175,10 +15181,13 @@
"dev": true
},
"@types/cors": {
"version": "2.8.12",
"resolved": "https://registry.npmjs.org/@types/cors/-/cors-2.8.12.tgz",
"integrity": "sha512-vt+kDhq/M2ayberEtJcIN/hxXy1Pk+59g2FV/ZQceeaTyCtCucjL2Q7FXlFjtWn4n15KCr1NE2lNNFhp0lEThw==",
"dev": true
"version": "2.8.13",
"resolved": "https://registry.npmjs.org/@types/cors/-/cors-2.8.13.tgz",
"integrity": "sha512-RG8AStHlUiV5ysZQKq97copd2UmVYw3/pRMLefISZ3S1hK104Cwm7iLQ3fTKx+lsUH2CE8FlLaYeEA2LSeqYUA==",
"dev": true,
"requires": {
"@types/node": "*"
}
},
"@types/eslint": {
"version": "8.4.6",
@@ -16746,9 +16755,9 @@
}
},
"engine.io": {
"version": "6.2.1",
"resolved": "https://registry.npmjs.org/engine.io/-/engine.io-6.2.1.tgz",
"integrity": "sha512-ECceEFcAaNRybd3lsGQKas3ZlMVjN3cyWwMP25D2i0zWfyiytVbTpRPa34qrr+FHddtpBVOmq4H/DCv1O0lZRA==",
"version": "6.4.2",
"resolved": "https://registry.npmjs.org/engine.io/-/engine.io-6.4.2.tgz",
"integrity": "sha512-FKn/3oMiJjrOEOeUub2WCox6JhxBXq/Zn3fZOMCBxKnNYtsdKjxhl7yR3fZhM9PV+rdE75SU5SYMc+2PGzo+Tg==",
"dev": true,
"requires": {
"@types/cookie": "^0.4.1",
@@ -16760,13 +16769,13 @@
"cors": "~2.8.5",
"debug": "~4.3.1",
"engine.io-parser": "~5.0.3",
"ws": "~8.2.3"
"ws": "~8.11.0"
}
},
"engine.io-parser": {
"version": "5.0.4",
"resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-5.0.4.tgz",
"integrity": "sha512-+nVFp+5z1E3HcToEnO7ZIj3g+3k9389DvWtvJZz0T6/eOCPIyyxehFcedoYrZQrp0LgQbD9pPXhpMBKMd5QURg==",
"version": "5.0.6",
"resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-5.0.6.tgz",
"integrity": "sha512-tjuoZDMAdEhVnSFleYPCtdL2GXwVTGtNjoeJd9IhIG3C1xs9uwxqRNEu5WpnDZCaozwVlK/nuQhpodhXSIMaxw==",
"dev": true
},
"enhanced-resolve": {
@@ -20509,24 +20518,27 @@
"dev": true
},
"socket.io": {
"version": "4.5.2",
"resolved": "https://registry.npmjs.org/socket.io/-/socket.io-4.5.2.tgz",
"integrity": "sha512-6fCnk4ARMPZN448+SQcnn1u8OHUC72puJcNtSgg2xS34Cu7br1gQ09YKkO1PFfDn/wyUE9ZgMAwosJed003+NQ==",
"version": "4.6.1",
"resolved": "https://registry.npmjs.org/socket.io/-/socket.io-4.6.1.tgz",
"integrity": "sha512-KMcaAi4l/8+xEjkRICl6ak8ySoxsYG+gG6/XfRCPJPQ/haCRIJBTL4wIl8YCsmtaBovcAXGLOShyVWQ/FG8GZA==",
"dev": true,
"requires": {
"accepts": "~1.3.4",
"base64id": "~2.0.0",
"debug": "~4.3.2",
"engine.io": "~6.2.0",
"socket.io-adapter": "~2.4.0",
"socket.io-parser": "~4.2.0"
"engine.io": "~6.4.1",
"socket.io-adapter": "~2.5.2",
"socket.io-parser": "~4.2.1"
}
},
"socket.io-adapter": {
"version": "2.4.0",
"resolved": "https://registry.npmjs.org/socket.io-adapter/-/socket.io-adapter-2.4.0.tgz",
"integrity": "sha512-W4N+o69rkMEGVuk2D/cvca3uYsvGlMwsySWV447y99gUPghxq42BxqLNMndb+a1mm/5/7NeXVQS7RLa2XyXvYg==",
"dev": true
"version": "2.5.2",
"resolved": "https://registry.npmjs.org/socket.io-adapter/-/socket.io-adapter-2.5.2.tgz",
"integrity": "sha512-87C3LO/NOMc+eMcpcxUBebGjkpMDkNBS9tf7KJqcDsmL936EChtVva71Dw2q4tQcuVC+hAUy4an2NO/sYXmwRA==",
"dev": true,
"requires": {
"ws": "~8.11.0"
}
},
"socket.io-parser": {
"version": "4.2.1",
@@ -21486,9 +21498,9 @@
"dev": true
},
"ws": {
"version": "8.2.3",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.2.3.tgz",
"integrity": "sha512-wBuoj1BDpC6ZQ1B7DWQBYVLphPWkm8i9Y0/3YdHjHKHiohOJ1ws+3OccDWtH+PoC9DZD5WOTrJvNbWvjS6JWaA==",
"version": "8.11.0",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.11.0.tgz",
"integrity": "sha512-HPG3wQd9sNQoT9xHyNCXoDUa+Xw/VevmY9FoHyQ+g+rrMn4j6FB4np7Z0OhdTgjx6MgQLK7jwSy1YecU1+4Asg==",
"dev": true,
"requires": {}
},
+2 -2
View File
@@ -749,8 +749,8 @@ if (environment == 'alpha') {
],
expandBilling :
[
mainClass: 'google.registry.beam.billing.ExpandRecurringBillingEventsPipeline',
metaData : 'google/registry/beam/expand_recurring_billing_events_pipeline_metadata.json'
mainClass: 'google.registry.beam.billing.ExpandBillingRecurrencesPipeline',
metaData : 'google/registry/beam/expand_billing_recurrences_pipeline_metadata.json'
],
rde :
[
@@ -17,7 +17,6 @@ package google.registry.batch;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_REQUESTED_TIME;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_RESAVE_TIMES;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_RESOURCE_KEY;
import static google.registry.batch.CannedScriptExecutionAction.SCRIPT_PARAM;
import static google.registry.request.RequestParameters.extractBooleanParameter;
import static google.registry.request.RequestParameters.extractIntParameter;
import static google.registry.request.RequestParameters.extractLongParameter;
@@ -105,16 +104,15 @@ public class BatchModule {
}
@Provides
@Parameter(ExpandRecurringBillingEventsAction.PARAM_START_TIME)
@Parameter(ExpandBillingRecurrencesAction.PARAM_START_TIME)
static Optional<DateTime> provideStartTime(HttpServletRequest req) {
return extractOptionalDatetimeParameter(
req, ExpandRecurringBillingEventsAction.PARAM_START_TIME);
return extractOptionalDatetimeParameter(req, ExpandBillingRecurrencesAction.PARAM_START_TIME);
}
@Provides
@Parameter(ExpandRecurringBillingEventsAction.PARAM_END_TIME)
@Parameter(ExpandBillingRecurrencesAction.PARAM_END_TIME)
static Optional<DateTime> provideEndTime(HttpServletRequest req) {
return extractOptionalDatetimeParameter(req, ExpandRecurringBillingEventsAction.PARAM_END_TIME);
return extractOptionalDatetimeParameter(req, ExpandBillingRecurrencesAction.PARAM_END_TIME);
}
@Provides
@@ -124,9 +122,9 @@ public class BatchModule {
}
@Provides
@Parameter(ExpandRecurringBillingEventsAction.PARAM_ADVANCE_CURSOR)
@Parameter(ExpandBillingRecurrencesAction.PARAM_ADVANCE_CURSOR)
static boolean provideAdvanceCursor(HttpServletRequest req) {
return extractBooleanParameter(req, ExpandRecurringBillingEventsAction.PARAM_ADVANCE_CURSOR);
return extractBooleanParameter(req, ExpandBillingRecurrencesAction.PARAM_ADVANCE_CURSOR);
}
@Provides
@@ -140,11 +138,4 @@ public class BatchModule {
static boolean provideIsDryRun(HttpServletRequest req) {
return extractBooleanParameter(req, PARAM_DRY_RUN);
}
// TODO(b/234424397): remove method after credential changes are rolled out.
@Provides
@Parameter(SCRIPT_PARAM)
static String provideScriptName(HttpServletRequest req) {
return extractRequiredParameter(req, SCRIPT_PARAM);
}
}
@@ -16,11 +16,9 @@ package google.registry.batch;
import static google.registry.request.Action.Method.POST;
import com.google.common.collect.ImmutableMap;
import com.google.common.flogger.FluentLogger;
import google.registry.batch.cannedscript.GroupsApiChecker;
import google.registry.batch.cannedscript.CannedScripts;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
import javax.inject.Inject;
@@ -35,7 +33,7 @@ import javax.inject.Inject;
* <p>This action can be invoked using the Nomulus CLI command: {@code nomulus -e ${env} curl
* --service BACKEND -X POST -u '/_dr/task/executeCannedScript?script=${script_name}'}
*/
// TODO(b/234424397): remove class after credential changes are rolled out.
// TODO(b/277239043): remove class after credential changes are rolled out.
@Action(
service = Action.Service.BACKEND,
path = "/_dr/task/executeCannedScript",
@@ -45,29 +43,18 @@ import javax.inject.Inject;
public class CannedScriptExecutionAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
static final String SCRIPT_PARAM = "script";
static final ImmutableMap<String, Runnable> SCRIPTS =
ImmutableMap.of("runGroupsApiChecks", GroupsApiChecker::runGroupsApiChecks);
private final String scriptName;
@Inject
CannedScriptExecutionAction(@Parameter(SCRIPT_PARAM) String scriptName) {
logger.atInfo().log("Received request to run script %s", scriptName);
this.scriptName = scriptName;
CannedScriptExecutionAction() {
logger.atInfo().log("Received request to run scripts.");
}
@Override
public void run() {
if (!SCRIPTS.containsKey(scriptName)) {
throw new IllegalArgumentException("Script not found:" + scriptName);
}
try {
SCRIPTS.get(scriptName).run();
logger.atInfo().log("Finished running %s.", scriptName);
CannedScripts.runAllChecks();
logger.atInfo().log("Finished running scripts.");
} catch (Throwable t) {
logger.atWarning().withCause(t).log("Error executing %s", scriptName);
logger.atWarning().withCause(t).log("Error executing scripts.");
throw new RuntimeException("Execution failed.");
}
}
@@ -19,6 +19,7 @@ import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.batch.BatchModule.PARAM_DRY_RUN;
import static google.registry.config.RegistryEnvironment.PRODUCTION;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_DELETE;
import static google.registry.model.tld.Registries.getTldsOfType;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
@@ -32,7 +33,6 @@ import com.google.common.collect.Sets;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryEnvironment;
import google.registry.dns.DnsUtils;
import google.registry.model.CreateAutoTimestamp;
import google.registry.model.EppResourceUtils;
import google.registry.model.domain.Domain;
@@ -98,8 +98,6 @@ public class DeleteProberDataAction implements Runnable {
/** Number of domains to retrieve and delete per SQL transaction. */
private static final int BATCH_SIZE = 1000;
@Inject DnsUtils dnsUtils;
@Inject
@Parameter(PARAM_DRY_RUN)
boolean isDryRun;
@@ -222,7 +220,7 @@ public class DeleteProberDataAction implements Runnable {
}
}
private void hardDeleteDomainsAndHosts(
private static void hardDeleteDomainsAndHosts(
ImmutableList<String> domainRepoIds, ImmutableList<String> hostNames) {
tm().query("DELETE FROM Host WHERE hostName IN :hostNames")
.setParameter("hostNames", hostNames)
@@ -264,6 +262,6 @@ public class DeleteProberDataAction implements Runnable {
// messages, or auto-renews because those will all be hard-deleted the next time the job runs
// anyway.
tm().putAll(ImmutableList.of(deletedDomain, historyEntry));
dnsUtils.requestDomainDnsRefresh(deletedDomain.getDomainName());
requestDomainDnsRefresh(deletedDomain.getDomainName());
}
}
@@ -29,11 +29,11 @@ import com.google.api.services.dataflow.model.LaunchFlexTemplateRequest;
import com.google.api.services.dataflow.model.LaunchFlexTemplateResponse;
import com.google.common.collect.ImmutableMap;
import com.google.common.flogger.FluentLogger;
import google.registry.beam.billing.ExpandRecurringBillingEventsPipeline;
import google.registry.beam.billing.ExpandBillingRecurrencesPipeline;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryEnvironment;
import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.common.Cursor;
import google.registry.request.Action;
import google.registry.request.Parameter;
@@ -46,20 +46,20 @@ import javax.inject.Inject;
import org.joda.time.DateTime;
/**
* An action that kicks off a {@link ExpandRecurringBillingEventsPipeline} dataflow job to expand
* {@link Recurring} billing events into synthetic {@link OneTime} events.
* An action that kicks off a {@link ExpandBillingRecurrencesPipeline} dataflow job to expand {@link
* BillingRecurrence} billing events into synthetic {@link BillingEvent} events.
*/
@Action(
service = Action.Service.BACKEND,
path = "/_dr/task/expandRecurringBillingEvents",
path = "/_dr/task/expandBillingRecurrences",
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
public class ExpandRecurringBillingEventsAction implements Runnable {
public class ExpandBillingRecurrencesAction implements Runnable {
public static final String PARAM_START_TIME = "startTime";
public static final String PARAM_END_TIME = "endTime";
public static final String PARAM_ADVANCE_CURSOR = "advanceCursor";
private static final String PIPELINE_NAME = "expand_recurring_billing_events_pipeline";
private static final String PIPELINE_NAME = "expand_billing_recurrences_pipeline";
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
@Inject Clock clock;
@@ -97,7 +97,7 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
@Inject Response response;
@Inject
ExpandRecurringBillingEventsAction() {}
ExpandBillingRecurrencesAction() {}
@Override
public void run() {
@@ -133,7 +133,7 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
.put("advanceCursor", Boolean.toString(advanceCursor))
.build());
logger.atInfo().log(
"Launching recurring billing event expansion pipeline for event time range [%s, %s)%s.",
"Launching billing recurrence expansion pipeline for event time range [%s, %s)%s.",
startTime,
endTime,
isDryRun ? " in dry run mode" : advanceCursor ? "" : " without advancing the cursor");
@@ -152,7 +152,7 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
response.setStatus(SC_OK);
response.setPayload(
String.format(
"Launched recurring billing event expansion pipeline: %s",
"Launched billing recurrence expansion pipeline: %s",
launchResponse.getJob().getId()));
} catch (IOException e) {
logger.atWarning().withCause(e).log("Pipeline Launch failed");
@@ -0,0 +1,199 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch.cannedscript;
import com.google.api.gax.core.FixedCredentialsProvider;
import com.google.api.services.bigquery.Bigquery;
import com.google.api.services.dataflow.Dataflow;
import com.google.api.services.dns.Dns;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;
import com.google.cloud.tasks.v2.CloudTasksClient;
import com.google.cloud.tasks.v2.CloudTasksSettings;
import com.google.common.base.Supplier;
import com.google.common.base.Suppliers;
import com.google.common.flogger.FluentLogger;
import dagger.Component;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule;
import google.registry.config.CredentialModule.ApplicationDefaultCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.util.GoogleCredentialsBundle;
import google.registry.util.UtilsModule;
import java.io.IOException;
import java.util.Optional;
import javax.inject.Singleton;
/** Canned actions invoked from {@link google.registry.batch.CannedScriptExecutionAction}. */
// TODO(b/277239043): remove class after credential changes are rolled out.
public class CannedScripts {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final Supplier<CannedScriptsComponent> COMPONENT_SUPPLIER =
Suppliers.memoize(DaggerCannedScripts_CannedScriptsComponent::create);
public static void runAllChecks() {
CannedScriptsComponent component = COMPONENT_SUPPLIER.get();
String projectId = component.projectId();
Bigquery bigquery = component.bigQuery();
try {
bigquery.datasets().list(projectId).execute().getDatasets().stream()
.findAny()
.ifPresent(
datasets ->
logger.atInfo().log("Found a BQ dataset [%s]", datasets.getFriendlyName()));
logger.atInfo().log("Finished accessing BQ.");
} catch (IOException ioe) {
logger.atSevere().withCause(ioe).log("Failed to access bigquery.");
}
try {
Dataflow dataflow = component.dataflow();
dataflow.projects().jobs().list(projectId).execute().getJobs().stream()
.findAny()
.ifPresent(job -> logger.atInfo().log("Found a job [%s]", job.getName()));
logger.atInfo().log("Finished accessing Dataflow.");
} catch (IOException ioe) {
logger.atSevere().withCause(ioe).log("Failed to access dataflow.");
}
try {
Storage gcs = component.gcs();
gcs.listAcls(projectId + "-beam");
logger.atInfo().log("Finished accessing gcs.");
} catch (RuntimeException e) {
logger.atSevere().withCause(e).log("Failed to access gcs.");
}
try {
Dns dns = component.dns();
dns.managedZones().list(projectId).execute().getManagedZones().stream()
.findAny()
.ifPresent(zone -> logger.atInfo().log("Found one zone [%s].", zone.getName()));
logger.atInfo().log("Finished accessing dns.");
} catch (IOException ioe) {
logger.atSevere().withCause(ioe).log("Failed to access dns.");
}
try {
CloudTasksClient client = component.cloudtasksClient();
com.google.cloud.tasks.v2.Queue queue =
client.getQueue(
String.format(
"projects/%s/locations/%s/queues/async-actions",
projectId, component.locationId()));
logger.atInfo().log("Got async queue state [%s]", queue.getState().name());
logger.atInfo().log("Finished accessing cloudtasks.");
} catch (RuntimeException e) {
logger.atSevere().withCause(e).log("Failed to access cloudtasks.");
}
}
@Singleton
@Component(
modules = {
ConfigModule.class,
CredentialModule.class,
CannedScriptsModule.class,
UtilsModule.class
})
interface CannedScriptsComponent {
Bigquery bigQuery();
CloudTasksClient cloudtasksClient();
Dataflow dataflow();
Dns dns();
Storage gcs();
@Config("projectId")
String projectId();
@Config("locationId")
String locationId();
}
@Module
static class CannedScriptsModule {
@Provides
static Bigquery provideBigquery(
@ApplicationDefaultCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Bigquery.Builder(
credentialsBundle.getHttpTransport(),
credentialsBundle.getJsonFactory(),
credentialsBundle.getHttpRequestInitializer())
.setApplicationName(projectId)
.build();
}
@Provides
static Dataflow provideDataflow(
@ApplicationDefaultCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Dataflow.Builder(
credentialsBundle.getHttpTransport(),
credentialsBundle.getJsonFactory(),
credentialsBundle.getHttpRequestInitializer())
.setApplicationName(String.format("%s billing", projectId))
.build();
}
@Provides
static Storage provideGcs(
@ApplicationDefaultCredential GoogleCredentialsBundle credentialsBundle) {
return StorageOptions.newBuilder()
.setCredentials(credentialsBundle.getGoogleCredentials())
.build()
.getService();
}
@Provides
static Dns provideDns(
@ApplicationDefaultCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId,
@Config("cloudDnsRootUrl") Optional<String> rootUrl,
@Config("cloudDnsServicePath") Optional<String> servicePath) {
Dns.Builder builder =
new Dns.Builder(
credentialsBundle.getHttpTransport(),
credentialsBundle.getJsonFactory(),
credentialsBundle.getHttpRequestInitializer())
.setApplicationName(projectId);
rootUrl.ifPresent(builder::setRootUrl);
servicePath.ifPresent(builder::setServicePath);
return builder.build();
}
@Provides
public static CloudTasksClient provideCloudTasksClient(
@ApplicationDefaultCredential GoogleCredentialsBundle credentials) {
CloudTasksClient client;
try {
client =
CloudTasksClient.create(
CloudTasksSettings.newBuilder()
.setCredentialsProvider(
FixedCredentialsProvider.create(credentials.getGoogleCredentials()))
.build());
} catch (IOException e) {
throw new RuntimeException(e);
}
return client;
}
}
}
@@ -1,122 +0,0 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch.cannedscript;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.util.RegistrarUtils.normalizeRegistrarId;
import com.google.api.services.admin.directory.Directory;
import com.google.api.services.groupssettings.Groupssettings;
import com.google.common.base.Supplier;
import com.google.common.base.Suppliers;
import com.google.common.base.Throwables;
import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import dagger.Component;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule;
import google.registry.config.CredentialModule.AdcDelegatedCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.groups.DirectoryGroupsConnection;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.util.GoogleCredentialsBundle;
import google.registry.util.UtilsModule;
import java.util.List;
import java.util.Set;
import javax.inject.Singleton;
/**
* Verifies that the credential with the {@link AdcDelegatedCredential} annotation can be used to
* access the Google Workspace Groups API.
*/
// TODO(b/234424397): remove class after credential changes are rolled out.
public class GroupsApiChecker {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final Supplier<GroupsConnectionComponent> COMPONENT_SUPPLIER =
Suppliers.memoize(DaggerGroupsApiChecker_GroupsConnectionComponent::create);
public static void runGroupsApiChecks() {
GroupsConnectionComponent component = COMPONENT_SUPPLIER.get();
DirectoryGroupsConnection groupsConnection = component.groupsConnection();
List<Registrar> registrars =
Streams.stream(Registrar.loadAllCached())
.filter(registrar -> registrar.isLive() && registrar.getType() == Registrar.Type.REAL)
.collect(toImmutableList());
for (Registrar registrar : registrars) {
for (final RegistrarPoc.Type type : RegistrarPoc.Type.values()) {
String groupKey =
String.format(
"%s-%s-contacts@%s",
normalizeRegistrarId(registrar.getRegistrarId()),
type.getDisplayName(),
component.gSuiteDomainName());
try {
Set<String> currentMembers = groupsConnection.getMembersOfGroup(groupKey);
logger.atInfo().log("Found %s members for %s.", currentMembers.size(), groupKey);
} catch (Exception e) {
Throwables.throwIfUnchecked(e);
throw new RuntimeException(e);
}
}
}
}
@Singleton
@Component(
modules = {
ConfigModule.class,
CredentialModule.class,
GroupsApiModule.class,
UtilsModule.class
})
interface GroupsConnectionComponent {
DirectoryGroupsConnection groupsConnection();
@Config("gSuiteDomainName")
String gSuiteDomainName();
}
@Module
static class GroupsApiModule {
@Provides
static Directory provideDirectory(
@AdcDelegatedCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Directory.Builder(
credentialsBundle.getHttpTransport(),
credentialsBundle.getJsonFactory(),
credentialsBundle.getHttpRequestInitializer())
.setApplicationName(projectId)
.build();
}
@Provides
static Groupssettings provideGroupsSettings(
@AdcDelegatedCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Groupssettings.Builder(
credentialsBundle.getHttpTransport(),
credentialsBundle.getJsonFactory(),
credentialsBundle.getHttpRequestInitializer())
.setApplicationName(projectId)
.build();
}
}
}
@@ -64,7 +64,7 @@ public abstract class BillingEvent implements Serializable {
"amount",
"flags");
/** Returns the unique ID for the {@code OneTime} associated with this event. */
/** Returns the unique ID for the {@code BillingEvent} associated with this event. */
abstract long id();
/** Returns the UTC DateTime this event becomes billable. */
@@ -38,10 +38,10 @@ import google.registry.flows.custom.CustomLogicModule;
import google.registry.flows.domain.DomainPricingLogic;
import google.registry.flows.domain.DomainPricingLogic.AllocationTokenInvalidForPremiumNameException;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingEvent.Cancellation;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingBase.Flag;
import google.registry.model.billing.BillingCancellation;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.common.Cursor;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
@@ -77,48 +77,49 @@ import org.apache.beam.sdk.values.PDone;
import org.joda.time.DateTime;
/**
* Definition of a Dataflow Flex pipeline template, which expands {@link Recurring} to {@link
* OneTime} when an autorenew occurs within the given time frame.
* Definition of a Dataflow Flex pipeline template, which expands {@link BillingRecurrence} to
* {@link BillingEvent} when an autorenew occurs within the given time frame.
*
* <p>This pipeline works in three stages:
*
* <ul>
* <li>Gather the {@link Recurring}s that are in scope for expansion. The exact condition of
* {@link Recurring}s to include can be found in {@link #getRecurringsInScope(Pipeline)}.
* <li>Expand the {@link Recurring}s to {@link OneTime} (and corresponding {@link DomainHistory})
* that fall within the [{@link #startTime}, {@link #endTime}) window, excluding those that
* are already present (to make this pipeline idempotent when running with the same parameters
* multiple times, either in parallel or in sequence). The {@link Recurring} is also updated
* with the information on when it was last expanded, so it would not be in scope for
* expansion until at least a year later.
* <li>Gather the {@link BillingRecurrence}s that are in scope for expansion. The exact condition
* of {@link BillingRecurrence}s to include can be found in {@link
* #getRecurrencesInScope(Pipeline)}.
* <li>Expand the {@link BillingRecurrence}s to {@link BillingEvent} (and corresponding {@link
* DomainHistory}) that fall within the [{@link #startTime}, {@link #endTime}) window,
* excluding those that are already present (to make this pipeline idempotent when running
* with the same parameters multiple times, either in parallel or in sequence). The {@link
* BillingRecurrence} is also updated with the information on when it was last expanded, so it
* would not be in scope for expansion until at least a year later.
* <li>If the cursor for billing events should be advanced, advance it to {@link #endTime} after
* all of the expansions in the previous step is done, only when it is currently at {@link
* #startTime}.
* </ul>
*
* <p>Note that the creation of new {@link OneTime} and {@link DomainHistory} is done speculatively
* as soon as its event time is in scope for expansion (i.e. within the window of operation). If a
* domain is subsequently cancelled during the autorenew grace period, a {@link Cancellation} would
* have been created to cancel the {@link OneTime} out. Similarly, a {@link DomainHistory} for the
* delete will be created which negates the effect of the speculatively created {@link
* DomainHistory}, specifically for the transaction records. Both the {@link OneTime} and {@link
* DomainHistory} will only be used (and cancelled out) when the billing time becomes effective,
* which is after the grace period, when the cancellations would have been written, if need be. This
* is no different from what we do with manual renewals or normal creates, where entities are always
* created for the action regardless of whether their effects will be negated later due to
* subsequent actions within respective grace periods.
* <p>Note that the creation of new {@link BillingEvent} and {@link DomainHistory} is done
* speculatively as soon as its event time is in scope for expansion (i.e. within the window of
* operation). If a domain is subsequently cancelled during the autorenew grace period, a {@link
* BillingCancellation} would have been created to cancel the {@link BillingEvent} out. Similarly, a
* {@link DomainHistory} for the delete will be created which negates the effect of the
* speculatively created {@link DomainHistory}, specifically for the transaction records. Both the
* {@link BillingEvent} and {@link DomainHistory} will only be used (and cancelled out) when the
* billing time becomes effective, which is after the grace period, when the cancellations would
* have been written, if need be. This is no different from what we do with manual renewals or
* normal creates, where entities are always created for the action regardless of whether their
* effects will be negated later due to subsequent actions within respective grace periods.
*
* <p>To stage this template locally, run {@code ./nom_build :core:sBP --environment=alpha \
* --pipeline=expandBilling}.
*
* <p>Then, you can run the staged template via the API client library, gCloud or a raw REST call.
*
* @see Cancellation#forGracePeriod
* @see BillingCancellation#forGracePeriod
* @see google.registry.flows.domain.DomainFlowUtils#createCancelingRecords
* @see <a href="https://cloud.google.com/dataflow/docs/guides/templates/using-flex-templates">Using
* Flex Templates</a>
*/
public class ExpandRecurringBillingEventsPipeline implements Serializable {
public class ExpandBillingRecurrencesPipeline implements Serializable {
private static final long serialVersionUID = -5827984301386630194L;
@@ -128,7 +129,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
static {
PipelineComponent pipelineComponent =
DaggerExpandRecurringBillingEventsPipeline_PipelineComponent.create();
DaggerExpandBillingRecurrencesPipeline_PipelineComponent.create();
domainPricingLogic = pipelineComponent.domainPricingLogic();
batchSize = pipelineComponent.batchSize();
}
@@ -139,8 +140,8 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
private final DateTime endTime;
private final boolean isDryRun;
private final boolean advanceCursor;
private final Counter recurringsInScopeCounter =
Metrics.counter("ExpandBilling", "Recurrings in scope for expansion");
private final Counter recurrencesInScopeCounter =
Metrics.counter("ExpandBilling", "Recurrences in scope for expansion");
// Note that this counter is only accurate when running in dry run mode. Because SQL persistence
// is a side effect and not idempotent, a transaction to save OneTimes could be successful but the
// transform that contains it could be still be retried, rolling back the counter increment. The
@@ -150,8 +151,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
private final Counter oneTimesToExpandCounter =
Metrics.counter("ExpandBilling", "OneTimes that would be expanded");
ExpandRecurringBillingEventsPipeline(
ExpandRecurringBillingEventsPipelineOptions options, Clock clock) {
ExpandBillingRecurrencesPipeline(ExpandBillingRecurrencesPipelineOptions options, Clock clock) {
startTime = DateTime.parse(options.getStartTime());
endTime = DateTime.parse(options.getEndTime());
checkArgument(
@@ -170,16 +170,16 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
}
void setupPipeline(Pipeline pipeline) {
PCollection<KV<Integer, Long>> recurringIds = getRecurringsInScope(pipeline);
PCollection<Void> expanded = expandRecurrings(recurringIds);
PCollection<KV<Integer, Long>> recurrenceIds = getRecurrencesInScope(pipeline);
PCollection<Void> expanded = expandRecurrences(recurrenceIds);
if (!isDryRun && advanceCursor) {
advanceCursor(expanded);
}
}
PCollection<KV<Integer, Long>> getRecurringsInScope(Pipeline pipeline) {
PCollection<KV<Integer, Long>> getRecurrencesInScope(Pipeline pipeline) {
return pipeline.apply(
"Read all Recurrings in scope",
"Read all Recurrences in scope",
// Use native query because JPQL does not support timestamp arithmetics.
RegistryJpaIO.read(
"SELECT billing_recurrence_id "
@@ -203,7 +203,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
endTime.minusYears(1)),
true,
(BigInteger id) -> {
recurringsInScopeCounter.inc();
recurrencesInScopeCounter.inc();
// Note that because all elements are mapped to the same dummy key, the next
// batching transform will effectively be serial. This however does not matter for
// our use case because the elements were obtained from a SQL read query, which
@@ -222,13 +222,13 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
.withCoder(KvCoder.of(VarIntCoder.of(), VarLongCoder.of())));
}
private PCollection<Void> expandRecurrings(PCollection<KV<Integer, Long>> recurringIds) {
return recurringIds
private PCollection<Void> expandRecurrences(PCollection<KV<Integer, Long>> recurrenceIds) {
return recurrenceIds
.apply(
"Group into batches",
GroupIntoBatches.<Integer, Long>ofSize(batchSize).withShardedKey())
.apply(
"Expand and save Recurrings into OneTimes and corresponding DomainHistories",
"Expand and save Recurrences into OneTimes and corresponding DomainHistories",
MapElements.into(voids())
.via(
element -> {
@@ -237,7 +237,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
() -> {
ImmutableSet.Builder<ImmutableObject> results =
new ImmutableSet.Builder<>();
ids.forEach(id -> expandOneRecurring(id, results));
ids.forEach(id -> expandOneRecurrence(id, results));
if (!isDryRun) {
tm().putAll(results.build());
}
@@ -246,14 +246,16 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
}));
}
private void expandOneRecurring(Long recurringId, ImmutableSet.Builder<ImmutableObject> results) {
Recurring recurring = tm().loadByKey(Recurring.createVKey(recurringId));
private void expandOneRecurrence(
Long recurrenceId, ImmutableSet.Builder<ImmutableObject> results) {
BillingRecurrence billingRecurrence =
tm().loadByKey(BillingRecurrence.createVKey(recurrenceId));
// Determine the complete set of EventTimes this recurring event should expand to within
// Determine the complete set of EventTimes this recurrence event should expand to within
// [max(recurrenceLastExpansion + 1 yr, startTime), min(recurrenceEndTime, endTime)).
//
// This range should always be legal for recurrings that are returned from the query. However,
// it is possible that the recurring has changed between when the read transformation occurred
// This range should always be legal for recurrences that are returned from the query. However,
// it is possible that the recurrence has changed between when the read transformation occurred
// and now. This could be caused by some out-of-process mutations (such as a domain deletion
// closing out a previously open-ended recurrence), or more subtly, Beam could execute the same
// work multiple times due to transient communication issues between workers and the scheduler.
@@ -264,23 +266,25 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
// to work on the same batch. The second worker would see a new recurrence_last_expansion that
// causes the range to be illegal.
//
// The best way to handle any unexpected behavior is to simply drop the recurring from
// The best way to handle any unexpected behavior is to simply drop the recurrence from
// expansion, if its new state still calls for an expansion, it would be picked up the next time
// the pipeline runs.
ImmutableSet<DateTime> eventTimes;
try {
eventTimes =
ImmutableSet.copyOf(
recurring
billingRecurrence
.getRecurrenceTimeOfYear()
.getInstancesInRange(
Range.closedOpen(
latestOf(recurring.getRecurrenceLastExpansion().plusYears(1), startTime),
earliestOf(recurring.getRecurrenceEndTime(), endTime))));
latestOf(
billingRecurrence.getRecurrenceLastExpansion().plusYears(1),
startTime),
earliestOf(billingRecurrence.getRecurrenceEndTime(), endTime))));
} catch (IllegalArgumentException e) {
return;
}
Domain domain = tm().loadByKey(Domain.createVKey(recurring.getDomainRepoId()));
Domain domain = tm().loadByKey(Domain.createVKey(billingRecurrence.getDomainRepoId()));
Tld tld = Tld.get(domain.getTld());
// Find the times for which the OneTime billing event are already created, making this expansion
@@ -292,7 +296,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
"SELECT eventTime FROM BillingEvent WHERE cancellationMatchingBillingEvent ="
+ " :key",
DateTime.class)
.setParameter("key", recurring.createVKey())
.setParameter("key", billingRecurrence.createVKey())
.getResultList());
Set<DateTime> eventTimesToExpand = difference(eventTimes, existingEventTimes);
@@ -301,7 +305,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
return;
}
DateTime recurrenceLastExpansionTime = recurring.getRecurrenceLastExpansion();
DateTime recurrenceLastExpansionTime = billingRecurrence.getRecurrenceLastExpansion();
// Create new OneTime and DomainHistory for EventTimes that needs to be expanded.
for (DateTime eventTime : eventTimesToExpand) {
@@ -333,7 +337,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
DomainHistory historyEntry =
new DomainHistory.Builder()
.setBySuperuser(false)
.setRegistrarId(recurring.getRegistrarId())
.setRegistrarId(billingRecurrence.getRegistrarId())
.setModificationTime(tm().getTransactionTime())
.setDomain(domain)
.setPeriod(Period.create(1, YEARS))
@@ -385,35 +389,43 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
// It is OK to always create a OneTime, even though the domain might be deleted or transferred
// later during autorenew grace period, as a cancellation will always be written out in those
// instances.
OneTime oneTime = null;
BillingEvent billingEvent = null;
try {
oneTime =
new OneTime.Builder()
billingEvent =
new BillingEvent.Builder()
.setBillingTime(billingTime)
.setRegistrarId(recurring.getRegistrarId())
.setRegistrarId(billingRecurrence.getRegistrarId())
// Determine the cost for a one-year renewal.
.setCost(
domainPricingLogic
.getRenewPrice(
tld, recurring.getTargetId(), eventTime, 1, recurring, Optional.empty())
tld,
billingRecurrence.getTargetId(),
eventTime,
1,
billingRecurrence,
Optional.empty())
.getRenewCost())
.setEventTime(eventTime)
.setFlags(union(recurring.getFlags(), Flag.SYNTHETIC))
.setFlags(union(billingRecurrence.getFlags(), Flag.SYNTHETIC))
.setDomainHistory(historyEntry)
.setPeriodYears(1)
.setReason(recurring.getReason())
.setReason(billingRecurrence.getReason())
.setSyntheticCreationTime(endTime)
.setCancellationMatchingBillingEvent(recurring)
.setTargetId(recurring.getTargetId())
.setCancellationMatchingBillingEvent(billingRecurrence)
.setTargetId(billingRecurrence.getTargetId())
.build();
} catch (AllocationTokenInvalidForPremiumNameException e) {
// This should not be reached since we are not using an allocation token
return;
}
results.add(oneTime);
results.add(billingEvent);
}
results.add(
recurring.asBuilder().setRecurrenceLastExpansion(recurrenceLastExpansionTime).build());
billingRecurrence
.asBuilder()
.setRecurrenceLastExpansion(recurrenceLastExpansionTime)
.build());
}
private PDone advanceCursor(PCollection<Void> persisted) {
@@ -456,11 +468,11 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
}
public static void main(String[] args) {
PipelineOptionsFactory.register(ExpandRecurringBillingEventsPipelineOptions.class);
ExpandRecurringBillingEventsPipelineOptions options =
PipelineOptionsFactory.register(ExpandBillingRecurrencesPipelineOptions.class);
ExpandBillingRecurrencesPipelineOptions options =
PipelineOptionsFactory.fromArgs(args)
.withValidation()
.as(ExpandRecurringBillingEventsPipelineOptions.class);
.as(ExpandBillingRecurrencesPipelineOptions.class);
// Hardcode the transaction level to be at serializable we do not want concurrent runs of the
// pipeline for the same window to create duplicate OneTimes. This ensures that the set of
// existing OneTimes do not change by the time new OneTimes are inserted within a transaction.
@@ -468,7 +480,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
// Per PostgreSQL, serializable isolation level does not introduce any blocking beyond that
// present in repeatable read other than some overhead related to monitoring possible
// serializable anomalies. Therefore, in most cases, since each worker of the same job works on
// a different set of recurrings, it is not possible for their execution order to affect
// a different set of recurrences, it is not possible for their execution order to affect
// serialization outcome, and the performance penalty should be minimum when using serializable
// compared to using repeatable read.
//
@@ -480,7 +492,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
// See: https://www.postgresql.org/docs/current/transaction-iso.html
options.setIsolationOverride(TransactionIsolationLevel.TRANSACTION_SERIALIZABLE);
Pipeline pipeline = Pipeline.create(options);
new ExpandRecurringBillingEventsPipeline(options, new SystemClock()).run(pipeline);
new ExpandBillingRecurrencesPipeline(options, new SystemClock()).run(pipeline);
}
@Singleton
@@ -18,7 +18,7 @@ import google.registry.beam.common.RegistryPipelineOptions;
import org.apache.beam.sdk.options.Default;
import org.apache.beam.sdk.options.Description;
public interface ExpandRecurringBillingEventsPipelineOptions extends RegistryPipelineOptions {
public interface ExpandBillingRecurrencesPipelineOptions extends RegistryPipelineOptions {
@Description(
"The inclusive lower bound of on the range of event times that will be expanded, in ISO 8601"
+ " format")
@@ -22,8 +22,8 @@ import google.registry.beam.billing.BillingEvent.InvoiceGroupingKey;
import google.registry.beam.billing.BillingEvent.InvoiceGroupingKey.InvoiceGroupingKeyCoder;
import google.registry.beam.common.RegistryJpaIO;
import google.registry.beam.common.RegistryJpaIO.Read;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingBase.Flag;
import google.registry.model.billing.BillingEvent;
import google.registry.model.registrar.Registrar;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.reporting.billing.BillingModule;
@@ -86,29 +86,30 @@ public class InvoicingPipeline implements Serializable {
void setupPipeline(Pipeline pipeline) {
options.setIsolationOverride(TransactionIsolationLevel.TRANSACTION_READ_COMMITTED);
PCollection<BillingEvent> billingEvents = readFromCloudSql(options, pipeline);
PCollection<google.registry.beam.billing.BillingEvent> billingEvents =
readFromCloudSql(options, pipeline);
saveInvoiceCsv(billingEvents, options);
saveDetailedCsv(billingEvents, options);
}
static PCollection<BillingEvent> readFromCloudSql(
static PCollection<google.registry.beam.billing.BillingEvent> readFromCloudSql(
InvoicingPipelineOptions options, Pipeline pipeline) {
Read<Object[], BillingEvent> read =
RegistryJpaIO.<Object[], BillingEvent>read(
Read<Object[], google.registry.beam.billing.BillingEvent> read =
RegistryJpaIO.<Object[], google.registry.beam.billing.BillingEvent>read(
makeCloudSqlQuery(options.getYearMonth()), false, row -> parseRow(row).orElse(null))
.withCoder(SerializableCoder.of(BillingEvent.class));
.withCoder(SerializableCoder.of(google.registry.beam.billing.BillingEvent.class));
PCollection<BillingEvent> billingEventsWithNulls =
PCollection<google.registry.beam.billing.BillingEvent> billingEventsWithNulls =
pipeline.apply("Read BillingEvents from Cloud SQL", read);
// Remove null billing events
return billingEventsWithNulls.apply(Filter.by(Objects::nonNull));
}
private static Optional<BillingEvent> parseRow(Object[] row) {
OneTime oneTime = (OneTime) row[0];
private static Optional<google.registry.beam.billing.BillingEvent> parseRow(Object[] row) {
BillingEvent billingEvent = (BillingEvent) row[0];
Registrar registrar = (Registrar) row[1];
CurrencyUnit currency = oneTime.getCost().getCurrencyUnit();
CurrencyUnit currency = billingEvent.getCost().getCurrencyUnit();
if (!registrar.getBillingAccountMap().containsKey(currency)) {
logger.atSevere().log(
"Registrar %s does not have a product account key for the currency unit: %s",
@@ -117,37 +118,40 @@ public class InvoicingPipeline implements Serializable {
}
return Optional.of(
BillingEvent.create(
oneTime.getId(),
oneTime.getBillingTime(),
oneTime.getEventTime(),
google.registry.beam.billing.BillingEvent.create(
billingEvent.getId(),
billingEvent.getBillingTime(),
billingEvent.getEventTime(),
registrar.getRegistrarId(),
registrar.getBillingAccountMap().get(currency),
registrar.getPoNumber().orElse(""),
DomainNameUtils.getTldFromDomainName(oneTime.getTargetId()),
oneTime.getReason().toString(),
oneTime.getTargetId(),
oneTime.getDomainRepoId(),
Optional.ofNullable(oneTime.getPeriodYears()).orElse(0),
oneTime.getCost().getCurrencyUnit().toString(),
oneTime.getCost().getAmount().doubleValue(),
DomainNameUtils.getTldFromDomainName(billingEvent.getTargetId()),
billingEvent.getReason().toString(),
billingEvent.getTargetId(),
billingEvent.getDomainRepoId(),
Optional.ofNullable(billingEvent.getPeriodYears()).orElse(0),
billingEvent.getCost().getCurrencyUnit().toString(),
billingEvent.getCost().getAmount().doubleValue(),
String.join(
" ", oneTime.getFlags().stream().map(Flag::toString).collect(toImmutableSet()))));
" ",
billingEvent.getFlags().stream().map(Flag::toString).collect(toImmutableSet()))));
}
/** Transform that converts a {@code BillingEvent} into an invoice CSV row. */
private static class GenerateInvoiceRows
extends PTransform<PCollection<BillingEvent>, PCollection<String>> {
extends PTransform<
PCollection<google.registry.beam.billing.BillingEvent>, PCollection<String>> {
private static final long serialVersionUID = -8090619008258393728L;
@Override
public PCollection<String> expand(PCollection<BillingEvent> input) {
public PCollection<String> expand(
PCollection<google.registry.beam.billing.BillingEvent> input) {
return input
.apply(
"Map to invoicing key",
MapElements.into(TypeDescriptor.of(InvoiceGroupingKey.class))
.via(BillingEvent::getInvoiceGroupingKey))
.via(google.registry.beam.billing.BillingEvent::getInvoiceGroupingKey))
.apply(
"Filter out free events", Filter.by((InvoiceGroupingKey key) -> key.unitPrice() != 0))
.setCoder(new InvoiceGroupingKeyCoder())
@@ -161,7 +165,8 @@ public class InvoicingPipeline implements Serializable {
/** Saves the billing events to a single overall invoice CSV file. */
static void saveInvoiceCsv(
PCollection<BillingEvent> billingEvents, InvoicingPipelineOptions options) {
PCollection<google.registry.beam.billing.BillingEvent> billingEvents,
InvoicingPipelineOptions options) {
billingEvents
.apply("Generate overall invoice rows", new GenerateInvoiceRows())
.apply(
@@ -182,16 +187,17 @@ public class InvoicingPipeline implements Serializable {
/** Saves the billing events to detailed report CSV files keyed by registrar-tld pairs. */
static void saveDetailedCsv(
PCollection<BillingEvent> billingEvents, InvoicingPipelineOptions options) {
PCollection<google.registry.beam.billing.BillingEvent> billingEvents,
InvoicingPipelineOptions options) {
String yearMonth = options.getYearMonth();
billingEvents.apply(
"Write detailed report for each registrar-tld pair",
FileIO.<String, BillingEvent>writeDynamic()
FileIO.<String, google.registry.beam.billing.BillingEvent>writeDynamic()
.to(
String.format(
"%s/%s/%s",
options.getBillingBucketUrl(), BillingModule.INVOICES_DIRECTORY, yearMonth))
.by(BillingEvent::getDetailedReportGroupingKey)
.by(google.registry.beam.billing.BillingEvent::getDetailedReportGroupingKey)
.withNumShards(1)
.withDestinationCoder(StringUtf8Coder.of())
.withNaming(
@@ -200,8 +206,8 @@ public class InvoicingPipeline implements Serializable {
String.format(
"%s_%s_%s.csv", BillingModule.DETAIL_REPORT_PREFIX, yearMonth, key))
.via(
Contextful.fn(BillingEvent::toCsv),
TextIO.sink().withHeader(BillingEvent.getHeader())));
Contextful.fn(google.registry.beam.billing.BillingEvent::toCsv),
TextIO.sink().withHeader(google.registry.beam.billing.BillingEvent.getHeader())));
}
/** Create the Cloud SQL query for a given yearMonth at runtime. */
@@ -1,41 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.dns;
/** Static class for DNS-related constants. */
public class DnsConstants {
private DnsConstants() {}
/** The name of the DNS pull queue. */
public static final String DNS_PULL_QUEUE_NAME = "dns-pull"; // See queue.xml.
/** The name of the DNS publish push queue. */
public static final String DNS_PUBLISH_PUSH_QUEUE_NAME = "dns-publish"; // See queue.xml.
/** The parameter to use for storing the target type ("domain" or "host" or "zone"). */
public static final String DNS_TARGET_TYPE_PARAM = "Target-Type";
/** The parameter to use for storing the target name (domain or host name) with the task. */
public static final String DNS_TARGET_NAME_PARAM = "Target-Name";
/** The parameter to use for storing the creation time with the task. */
public static final String DNS_TARGET_CREATE_TIME_PARAM = "Create-Time";
/** The possible values of the {@code DNS_TARGET_TYPE_PARAM} parameter. */
public enum TargetType {
DOMAIN,
HOST
}
}
@@ -14,8 +14,6 @@
package google.registry.dns;
import static google.registry.dns.DnsConstants.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.DnsConstants.DNS_PULL_QUEUE_NAME;
import static google.registry.dns.RefreshDnsOnHostRenameAction.PARAM_HOST_KEY;
import static google.registry.request.RequestParameters.extractEnumParameter;
import static google.registry.request.RequestParameters.extractIntParameter;
@@ -24,20 +22,17 @@ import static google.registry.request.RequestParameters.extractOptionalParameter
import static google.registry.request.RequestParameters.extractRequiredParameter;
import static google.registry.request.RequestParameters.extractSetOfParameters;
import com.google.appengine.api.taskqueue.Queue;
import com.google.appengine.api.taskqueue.QueueFactory;
import com.google.common.hash.HashFunction;
import com.google.common.hash.Hashing;
import dagger.Binds;
import dagger.Module;
import dagger.Provides;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.dns.DnsUtils.TargetType;
import google.registry.dns.writer.DnsWriterZone;
import google.registry.request.Parameter;
import google.registry.request.RequestParameters;
import java.util.Optional;
import java.util.Set;
import javax.inject.Named;
import javax.servlet.http.HttpServletRequest;
import org.joda.time.DateTime;
@@ -71,18 +66,6 @@ public abstract class DnsModule {
return Hashing.murmur3_32_fixed();
}
@Provides
@Named(DNS_PULL_QUEUE_NAME)
static Queue provideDnsPullQueue() {
return QueueFactory.getQueue(DNS_PULL_QUEUE_NAME);
}
@Provides
@Named(DNS_PUBLISH_PUSH_QUEUE_NAME)
static Queue provideDnsUpdatePushQueue() {
return QueueFactory.getQueue(DNS_PUBLISH_PUSH_QUEUE_NAME);
}
@Provides
@Parameter(PARAM_PUBLISH_TASK_ENQUEUED)
static DateTime provideCreateTime(HttpServletRequest req) {
@@ -1,170 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.dns;
import static com.google.appengine.api.taskqueue.QueueFactory.getQueue;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.dns.DnsConstants.DNS_PULL_QUEUE_NAME;
import static google.registry.dns.DnsConstants.DNS_TARGET_CREATE_TIME_PARAM;
import static google.registry.dns.DnsConstants.DNS_TARGET_NAME_PARAM;
import static google.registry.dns.DnsConstants.DNS_TARGET_TYPE_PARAM;
import static google.registry.model.tld.Registries.assertTldExists;
import static google.registry.request.RequestParameters.PARAM_TLD;
import static google.registry.util.DomainNameUtils.getTldFromDomainName;
import static java.util.concurrent.TimeUnit.MILLISECONDS;
import com.google.appengine.api.taskqueue.Queue;
import com.google.appengine.api.taskqueue.QueueConstants;
import com.google.appengine.api.taskqueue.TaskHandle;
import com.google.appengine.api.taskqueue.TaskOptions;
import com.google.appengine.api.taskqueue.TaskOptions.Method;
import com.google.appengine.api.taskqueue.TransientFailureException;
import com.google.apphosting.api.DeadlineExceededException;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableList;
import com.google.common.flogger.FluentLogger;
import com.google.common.net.InternetDomainName;
import com.google.common.util.concurrent.RateLimiter;
import google.registry.config.RegistryConfig;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.model.tld.Registries;
import google.registry.util.Clock;
import google.registry.util.NonFinalForTesting;
import java.util.List;
import java.util.Optional;
import java.util.logging.Level;
import javax.inject.Inject;
import javax.inject.Named;
import org.joda.time.Duration;
/**
* Methods for manipulating the queue used for DNS write tasks.
*
* <p>This includes a {@link RateLimiter} to limit the {@link Queue#leaseTasks} call rate to 9 QPS,
* to stay under the 10 QPS limit for this function.
*
* <p>Note that overlapping calls to {@link ReadDnsQueueAction} (the only place where {@link
* DnsQueue#leaseTasks} is used) will have different rate limiters, so they could exceed the allowed
* rate. This should be rare though - because {@link DnsQueue#leaseTasks} is only used in {@link
* ReadDnsQueueAction}, which is run as a cron job with running time shorter than the cron repeat
* time - meaning there should never be two instances running at once.
*
* @see RegistryConfig.ConfigModule#provideReadDnsRefreshRequestsRuntime()
*/
public class DnsQueue {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final Queue queue;
final Clock clock;
// Queue.leaseTasks is limited to 10 requests per second as per
// https://cloud.google.com/appengine/docs/standard/java/javadoc/com/google/appengine/api/taskqueue/Queue.html
// "If you generate more than 10 LeaseTasks requests per second, only the first 10 requests will
// return results. The others will return no results."
private static final RateLimiter rateLimiter = RateLimiter.create(9);
@Inject
DnsQueue(@Named(DNS_PULL_QUEUE_NAME) Queue queue, Clock clock) {
this.queue = queue;
this.clock = clock;
}
Clock getClock() {
return clock;
}
@VisibleForTesting
public static DnsQueue createForTesting(Clock clock) {
return new DnsQueue(getQueue(DNS_PULL_QUEUE_NAME), clock);
}
@NonFinalForTesting
@VisibleForTesting
long leaseTasksBatchSize = QueueConstants.maxLeaseCount();
/** Enqueues the given task type with the given target name to the DNS queue. */
private TaskHandle addToQueue(
TargetType targetType, String targetName, String tld, Duration countdown) {
logger.atInfo().log(
"Adding task type=%s, target=%s, tld=%s to pull queue %s (%d tasks currently on queue).",
targetType, targetName, tld, DNS_PULL_QUEUE_NAME, queue.fetchStatistics().getNumTasks());
return queue.add(
TaskOptions.Builder.withDefaults()
.method(Method.PULL)
.countdownMillis(countdown.getMillis())
.param(DNS_TARGET_TYPE_PARAM, targetType.toString())
.param(DNS_TARGET_NAME_PARAM, targetName)
.param(DNS_TARGET_CREATE_TIME_PARAM, clock.nowUtc().toString())
.param(PARAM_TLD, tld));
}
/** Adds a task to the queue to refresh the DNS information for the specified subordinate host. */
TaskHandle addHostRefreshTask(String hostName) {
Optional<InternetDomainName> tld = Registries.findTldForName(InternetDomainName.from(hostName));
checkArgument(
tld.isPresent(), String.format("%s is not a subordinate host to a known tld", hostName));
return addToQueue(TargetType.HOST, hostName, tld.get().toString(), Duration.ZERO);
}
/** Enqueues a task to refresh DNS for the specified domain now. */
TaskHandle addDomainRefreshTask(String domainName) {
return addDomainRefreshTask(domainName, Duration.ZERO);
}
/** Enqueues a task to refresh DNS for the specified domain at some point in the future. */
TaskHandle addDomainRefreshTask(String domainName, Duration countdown) {
return addToQueue(
TargetType.DOMAIN,
domainName,
assertTldExists(getTldFromDomainName(domainName)),
countdown);
}
/**
* Returns the maximum number of tasks that can be leased with {@link #leaseTasks}.
*
* <p>If this many tasks are returned, then there might be more tasks still waiting in the queue.
*
* <p>If less than this number of tasks are returned, then there are no more items in the queue.
*/
public long getLeaseTasksBatchSize() {
return leaseTasksBatchSize;
}
/** Returns handles for a batch of tasks, leased for the specified duration. */
public List<TaskHandle> leaseTasks(Duration leaseDuration) {
try {
rateLimiter.acquire();
int numTasks = queue.fetchStatistics().getNumTasks();
logger.at((numTasks >= leaseTasksBatchSize) ? Level.WARNING : Level.INFO).log(
"There are %d tasks in the DNS queue '%s'.", numTasks, DNS_PULL_QUEUE_NAME);
return queue.leaseTasks(leaseDuration.getMillis(), MILLISECONDS, leaseTasksBatchSize);
} catch (TransientFailureException | DeadlineExceededException e) {
logger.atSevere().withCause(e).log("Failed leasing tasks too fast.");
return ImmutableList.of();
}
}
/** Delete a list of tasks, removing them from the queue permanently. */
public void deleteTasks(List<TaskHandle> tasks) {
try {
queue.deleteTask(tasks);
} catch (TransientFailureException | DeadlineExceededException e) {
logger.atSevere().withCause(e).log("Failed deleting tasks too fast.");
}
}
}
@@ -19,57 +19,42 @@ import static google.registry.persistence.transaction.TransactionManagerFactory.
import com.google.common.collect.ImmutableList;
import com.google.common.net.InternetDomainName;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.model.common.DatabaseMigrationStateSchedule;
import google.registry.model.common.DatabaseMigrationStateSchedule.MigrationState;
import google.registry.model.common.DnsRefreshRequest;
import google.registry.model.tld.Registries;
import google.registry.model.tld.Tld;
import java.util.Collection;
import java.util.Optional;
import javax.inject.Inject;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/** Utility class to handle DNS refresh requests. */
// TODO: Make this a static util function once we are done with the DNS pull queue migration.
public class DnsUtils {
public final class DnsUtils {
private final DnsQueue dnsQueue;
/** The name of the DNS publish push queue. */
public static final String DNS_PUBLISH_PUSH_QUEUE_NAME = "dns-publish"; // See queue.xml.
@Inject
DnsUtils(DnsQueue dnsQueue) {
this.dnsQueue = dnsQueue;
}
private DnsUtils() {}
private void requestDnsRefresh(String name, TargetType type, Duration delay) {
private static void requestDnsRefresh(String name, TargetType type, Duration delay) {
// Throws an IllegalArgumentException if the name is not under a managed TLD -- we only update
// DNS for names that are under our management.
String tld = Registries.findTldForNameOrThrow(InternetDomainName.from(name)).toString();
if (usePullQueue()) {
if (TargetType.HOST.equals(type)) {
dnsQueue.addHostRefreshTask(name);
} else {
dnsQueue.addDomainRefreshTask(name, delay);
}
} else {
tm().transact(
() ->
tm().insert(
new DnsRefreshRequest(
type, name, tld, tm().getTransactionTime().plus(delay))));
}
tm().transact(
() ->
tm().insert(
new DnsRefreshRequest(
type, name, tld, tm().getTransactionTime().plus(delay))));
}
public void requestDomainDnsRefresh(String domainName, Duration delay) {
public static void requestDomainDnsRefresh(String domainName, Duration delay) {
requestDnsRefresh(domainName, TargetType.DOMAIN, delay);
}
public void requestDomainDnsRefresh(String domainName) {
public static void requestDomainDnsRefresh(String domainName) {
requestDomainDnsRefresh(domainName, Duration.ZERO);
}
public void requestHostDnsRefresh(String hostName) {
public static void requestHostDnsRefresh(String hostName) {
requestDnsRefresh(hostName, TargetType.HOST, Duration.ZERO);
}
@@ -85,7 +70,7 @@ public class DnsUtils {
* <li>The last time they were processed is before the cooldown period.
* </ul>
*/
public ImmutableList<DnsRefreshRequest> readAndUpdateRequestsWithLatestProcessTime(
public static ImmutableList<DnsRefreshRequest> readAndUpdateRequestsWithLatestProcessTime(
String tld, Duration cooldown, int batchSize) {
return tm().transact(
() -> {
@@ -105,7 +90,7 @@ public class DnsUtils {
// queued up for publishing, not when it is actually published by the DNS
// writer. This timestamp acts as a cooldown so the same request will not be
// retried too frequently. See DnsRefreshRequest.getLastProcessTime for a
// detailed explaination.
// detailed explanation.
.map(e -> e.updateProcessTime(transactionTime))
.collect(toImmutableList());
tm().updateAll(requests);
@@ -119,7 +104,7 @@ public class DnsUtils {
* <p>Note that if a request entity has already been deleted, the method still succeeds without
* error because all we care about is that it no longer exists after the method runs.
*/
public void deleteRequests(Collection<DnsRefreshRequest> requests) {
public static void deleteRequests(Collection<DnsRefreshRequest> requests) {
tm().transact(
() ->
tm().delete(
@@ -140,8 +125,9 @@ public class DnsUtils {
return dnsAPlusAaaaTtl.getStandardSeconds();
}
private boolean usePullQueue() {
return !DatabaseMigrationStateSchedule.getValueAtTime(dnsQueue.getClock().nowUtc())
.equals(MigrationState.DNS_SQL);
/** The possible values of the {@code DNS_TARGET_TYPE_PARAM} parameter. */
public enum TargetType {
DOMAIN,
HOST
}
}
@@ -15,7 +15,6 @@
package google.registry.dns;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.dns.DnsConstants.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.DnsModule.PARAM_DNS_WRITER;
import static google.registry.dns.DnsModule.PARAM_DOMAINS;
import static google.registry.dns.DnsModule.PARAM_HOSTS;
@@ -23,6 +22,9 @@ import static google.registry.dns.DnsModule.PARAM_LOCK_INDEX;
import static google.registry.dns.DnsModule.PARAM_NUM_PUBLISH_LOCKS;
import static google.registry.dns.DnsModule.PARAM_PUBLISH_TASK_ENQUEUED;
import static google.registry.dns.DnsModule.PARAM_REFRESH_REQUEST_TIME;
import static google.registry.dns.DnsUtils.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.dns.DnsUtils.requestHostDnsRefresh;
import static google.registry.model.EppResourceUtils.loadByForeignKey;
import static google.registry.request.Action.Method.POST;
import static google.registry.request.RequestParameters.PARAM_TLD;
@@ -85,7 +87,6 @@ public final class PublishDnsUpdatesAction implements Runnable, Callable<Void> {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final DnsUtils dnsUtils;
private final DnsWriterProxy dnsWriterProxy;
private final DnsMetrics dnsMetrics;
private final Duration timeout;
@@ -94,10 +95,10 @@ public final class PublishDnsUpdatesAction implements Runnable, Callable<Void> {
/**
* The DNS writer to use for this batch.
*
* <p>This comes from the fanout in {@link ReadDnsQueueAction} which dispatches each batch to be
* published by each DNS writer on the TLD. So this field contains the value of one of the DNS
* writers configured in {@link Tld#getDnsWriters()}, as of the time the batch was written out
* (and not necessarily currently).
* <p>This comes from the fanout in {@link ReadDnsRefreshRequestsAction} which dispatches each
* batch to be published by each DNS writer on the TLD. So this field contains the value of one of
* the DNS writers configured in {@link Tld#getDnsWriters()}, as of the time the batch was written
* out (and not necessarily currently).
*/
private final String dnsWriter;
@@ -139,7 +140,6 @@ public final class PublishDnsUpdatesAction implements Runnable, Callable<Void> {
@Config("gSuiteOutgoingEmailAddress") InternetAddress gSuiteOutgoingEmailAddress,
@Header(APP_ENGINE_RETRY_HEADER) Optional<Integer> appEngineRetryCount,
@Header(CLOUD_TASKS_RETRY_HEADER) Optional<Integer> cloudTasksRetryCount,
DnsUtils dnsUtils,
DnsWriterProxy dnsWriterProxy,
DnsMetrics dnsMetrics,
LockHandler lockHandler,
@@ -147,12 +147,11 @@ public final class PublishDnsUpdatesAction implements Runnable, Callable<Void> {
CloudTasksUtils cloudTasksUtils,
SendEmailService sendEmailService,
Response response) {
this.dnsUtils = dnsUtils;
this.dnsWriterProxy = dnsWriterProxy;
this.dnsMetrics = dnsMetrics;
this.timeout = timeout;
this.sendEmailService = sendEmailService;
this.retryCount =
retryCount =
cloudTasksRetryCount.orElse(
appEngineRetryCount.orElseThrow(
() -> new IllegalStateException("Missing a valid retry count header")));
@@ -276,7 +275,7 @@ public final class PublishDnsUpdatesAction implements Runnable, Callable<Void> {
return null;
}
private InternetAddress emailToInternetAddress(String email) {
private static InternetAddress emailToInternetAddress(String email) {
try {
return new InternetAddress(email, true);
} catch (Exception e) {
@@ -306,7 +305,7 @@ public final class PublishDnsUpdatesAction implements Runnable, Callable<Void> {
registrar.get().getContacts().stream()
.filter(c -> c.getTypes().contains(RegistrarPoc.Type.ADMIN))
.map(RegistrarPoc::getEmailAddress)
.map(this::emailToInternetAddress)
.map(PublishDnsUpdatesAction::emailToInternetAddress)
.collect(toImmutableList());
sendEmailService.sendEmail(
@@ -356,10 +355,10 @@ public final class PublishDnsUpdatesAction implements Runnable, Callable<Void> {
private void requeueBatch() {
logger.atInfo().log("Requeueing batch for retry.");
for (String domain : nullToEmpty(domains)) {
dnsUtils.requestDomainDnsRefresh(domain);
requestDomainDnsRefresh(domain);
}
for (String host : nullToEmpty(hosts)) {
dnsUtils.requestHostDnsRefresh(host);
requestHostDnsRefresh(host);
}
}
@@ -1,401 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.dns;
import static com.google.common.collect.ImmutableSetMultimap.toImmutableSetMultimap;
import static com.google.common.collect.Sets.difference;
import static google.registry.dns.DnsConstants.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.DnsConstants.DNS_TARGET_CREATE_TIME_PARAM;
import static google.registry.dns.DnsConstants.DNS_TARGET_NAME_PARAM;
import static google.registry.dns.DnsConstants.DNS_TARGET_TYPE_PARAM;
import static google.registry.dns.DnsModule.PARAM_DNS_WRITER;
import static google.registry.dns.DnsModule.PARAM_DOMAINS;
import static google.registry.dns.DnsModule.PARAM_HOSTS;
import static google.registry.dns.DnsModule.PARAM_LOCK_INDEX;
import static google.registry.dns.DnsModule.PARAM_NUM_PUBLISH_LOCKS;
import static google.registry.dns.DnsModule.PARAM_PUBLISH_TASK_ENQUEUED;
import static google.registry.dns.DnsModule.PARAM_REFRESH_REQUEST_TIME;
import static google.registry.request.RequestParameters.PARAM_TLD;
import static google.registry.util.DomainNameUtils.getSecondLevelDomain;
import static java.nio.charset.StandardCharsets.UTF_8;
import com.google.appengine.api.taskqueue.TaskHandle;
import com.google.auto.value.AutoValue;
import com.google.cloud.tasks.v2.Task;
import com.google.common.collect.ComparisonChain;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableMultimap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSetMultimap;
import com.google.common.collect.Iterables;
import com.google.common.collect.Ordering;
import com.google.common.flogger.FluentLogger;
import com.google.common.hash.HashFunction;
import com.google.common.hash.Hashing;
import google.registry.batch.CloudTasksUtils;
import google.registry.config.RegistryConfig.Config;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.model.tld.Registries;
import google.registry.model.tld.Tld;
import google.registry.request.Action;
import google.registry.request.Action.Service;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import java.io.UnsupportedEncodingException;
import java.util.Collection;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.stream.Collectors;
import javax.inject.Inject;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/**
* Action for fanning out DNS refresh tasks by TLD, using data taken from the DNS pull queue.
*
* <h3>Parameters Reference</h3>
*
* <ul>
* <li>{@code jitterSeconds} Randomly delay each task by up to this many seconds.
* </ul>
*/
@Action(
service = Action.Service.BACKEND,
path = "/_dr/cron/readDnsQueue",
automaticallyPrintOk = true,
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
public final class ReadDnsQueueAction implements Runnable {
private static final String PARAM_JITTER_SECONDS = "jitterSeconds";
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
/**
* Buffer time since the end of this action until retriable tasks are available again.
*
* <p>We read batches of tasks from the queue in a loop. Any task that will need to be retried has
* to be kept out of the queue for the duration of this action - otherwise we will lease it again
* in a subsequent loop.
*
* <p>The 'requestedMaximumDuration' value is the maximum delay between the first and last calls
* to lease tasks, hence we want the lease duration to be (slightly) longer than that.
* LEASE_PADDING is the value we add to {@link #requestedMaximumDuration} to make sure the lease
* duration is indeed longer.
*/
private static final Duration LEASE_PADDING = Duration.standardMinutes(1);
private final int tldUpdateBatchSize;
private final Duration requestedMaximumDuration;
private final Optional<Integer> jitterSeconds;
private final Clock clock;
private final DnsQueue dnsQueue;
private final HashFunction hashFunction;
private final CloudTasksUtils cloudTasksUtils;
@Inject
ReadDnsQueueAction(
@Config("dnsTldUpdateBatchSize") int tldUpdateBatchSize,
@Config("readDnsRefreshRequestsActionRuntime") Duration requestedMaximumDuration,
@Parameter(PARAM_JITTER_SECONDS) Optional<Integer> jitterSeconds,
Clock clock,
DnsQueue dnsQueue,
HashFunction hashFunction,
CloudTasksUtils cloudTasksUtils) {
this.tldUpdateBatchSize = tldUpdateBatchSize;
this.requestedMaximumDuration = requestedMaximumDuration;
this.jitterSeconds = jitterSeconds;
this.clock = clock;
this.dnsQueue = dnsQueue;
this.hashFunction = hashFunction;
this.cloudTasksUtils = cloudTasksUtils;
}
/** Container for items we pull out of the DNS pull queue and process for fanout. */
@AutoValue
abstract static class RefreshItem implements Comparable<RefreshItem> {
static RefreshItem create(TargetType type, String name, DateTime creationTime) {
return new AutoValue_ReadDnsQueueAction_RefreshItem(type, name, creationTime);
}
abstract TargetType type();
abstract String name();
abstract DateTime creationTime();
@Override
public int compareTo(RefreshItem other) {
return ComparisonChain.start()
.compare(this.type(), other.type())
.compare(this.name(), other.name())
.compare(this.creationTime(), other.creationTime())
.result();
}
}
/** Leases all tasks from the pull queue and creates per-tld update actions for them. */
@Override
public void run() {
DateTime requestedEndTime = clock.nowUtc().plus(requestedMaximumDuration);
ImmutableSet<String> tlds = Registries.getTlds();
while (requestedEndTime.isAfterNow()) {
List<TaskHandle> tasks = dnsQueue.leaseTasks(requestedMaximumDuration.plus(LEASE_PADDING));
logger.atInfo().log("Leased %d DNS update tasks.", tasks.size());
if (!tasks.isEmpty()) {
dispatchTasks(ImmutableSet.copyOf(tasks), tlds);
}
if (tasks.size() < dnsQueue.getLeaseTasksBatchSize()) {
return;
}
}
}
/** A set of tasks grouped based on the action to take on them. */
@AutoValue
abstract static class ClassifiedTasks {
/**
* List of tasks we want to keep in the queue (want to retry in the future).
*
* <p>Normally, any task we lease from the queue will be deleted - either because we are going
* to process it now (these tasks are part of refreshItemsByTld), or because these tasks are
* "corrupt" in some way (don't parse, don't have the required parameters etc.).
*
* <p>Some tasks however are valid, but can't be processed at the moment. These tasks will be
* kept in the queue for future processing.
*
* <p>This includes tasks belonging to paused TLDs (which we want to process once the TLD is
* unpaused) and tasks belonging to (currently) unknown TLDs.
*/
abstract ImmutableSet<TaskHandle> tasksToKeep();
/** The paused TLDs for which we found at least one refresh request. */
abstract ImmutableSet<String> pausedTlds();
/**
* The unknown TLDs for which we found at least one refresh request.
*
* <p>Unknown TLDs might have valid requests because the list of TLDs is heavily cached. Hence,
* when we add a new TLD - it is possible that some instances will not have that TLD in their
* list yet. We don't want to discard these tasks, so we wait a bit and retry them again.
*
* <p>This is less likely for production TLDs but is quite likely for test TLDs where we might
* create a TLD and then use it within seconds.
*/
abstract ImmutableSet<String> unknownTlds();
/**
* All the refresh items we need to actually fulfill, grouped by TLD.
*
* <p>By default, the multimap is ordered - this can be changed with the builder.
*
* <p>The items for each TLD will be grouped together, and domains and hosts will be grouped
* within a TLD.
*
* <p>The grouping and ordering of domains and hosts is not technically necessary, but a
* predictable ordering makes it possible to write detailed tests.
*/
abstract ImmutableSetMultimap<String, RefreshItem> refreshItemsByTld();
static Builder builder() {
Builder builder = new AutoValue_ReadDnsQueueAction_ClassifiedTasks.Builder();
builder
.refreshItemsByTldBuilder()
.orderKeysBy(Ordering.natural())
.orderValuesBy(Ordering.natural());
return builder;
}
@AutoValue.Builder
abstract static class Builder {
abstract ImmutableSet.Builder<TaskHandle> tasksToKeepBuilder();
abstract ImmutableSet.Builder<String> pausedTldsBuilder();
abstract ImmutableSet.Builder<String> unknownTldsBuilder();
abstract ImmutableSetMultimap.Builder<String, RefreshItem> refreshItemsByTldBuilder();
abstract ClassifiedTasks build();
}
}
/**
* Creates per-tld update actions for tasks leased from the pull queue.
*
* <p>Will return "irrelevant" tasks to the queue for future processing. "Irrelevant" tasks are
* tasks for paused TLDs or tasks for TLDs not part of {@link Registries#getTlds()}.
*/
private void dispatchTasks(ImmutableSet<TaskHandle> tasks, ImmutableSet<String> tlds) {
ClassifiedTasks classifiedTasks = classifyTasks(tasks, tlds);
if (!classifiedTasks.pausedTlds().isEmpty()) {
logger.atInfo().log(
"The dns-pull queue is paused for TLDs: %s.", classifiedTasks.pausedTlds());
}
if (!classifiedTasks.unknownTlds().isEmpty()) {
logger.atWarning().log(
"The dns-pull queue has unknown TLDs: %s.", classifiedTasks.unknownTlds());
}
bucketRefreshItems(classifiedTasks.refreshItemsByTld());
if (!classifiedTasks.tasksToKeep().isEmpty()) {
logger.atWarning().log(
"Keeping %d DNS update tasks in the queue.", classifiedTasks.tasksToKeep().size());
}
// Delete the tasks we don't want to see again from the queue.
//
// tasksToDelete includes both the tasks that we already fulfilled in this call (were part of
// refreshItemsByTld) and "corrupt" tasks we weren't able to parse correctly (this shouldn't
// happen, and we logged a "severe" error)
//
// We let the lease on the rest of the tasks (the tasks we want to keep) expire on its own. The
// reason we don't release these tasks back to the queue immediately is that we are going to
// immediately read another batch of tasks from the queue - and we don't want to get the same
// tasks again.
ImmutableSet<TaskHandle> tasksToDelete =
difference(tasks, classifiedTasks.tasksToKeep()).immutableCopy();
logger.atInfo().log("Removing %d DNS update tasks from the queue.", tasksToDelete.size());
dnsQueue.deleteTasks(tasksToDelete.asList());
logger.atInfo().log("Done processing DNS tasks.");
}
/**
* Classifies the given tasks based on what action we need to take on them.
*
* <p>Note that some tasks might appear in multiple categories (if multiple actions are to be
* taken on them) or in no category (if no action is to be taken on them)
*/
private static ClassifiedTasks classifyTasks(
ImmutableSet<TaskHandle> tasks, ImmutableSet<String> tlds) {
ClassifiedTasks.Builder classifiedTasksBuilder = ClassifiedTasks.builder();
// Read all tasks on the DNS pull queue and load them into the refresh item multimap.
for (TaskHandle task : tasks) {
try {
Map<String, String> params = ImmutableMap.copyOf(task.extractParams());
DateTime creationTime = DateTime.parse(params.get(DNS_TARGET_CREATE_TIME_PARAM));
String tld = params.get(PARAM_TLD);
if (tld == null) {
logger.atSevere().log(
"Discarding invalid DNS refresh request %s; no TLD specified.", task);
} else if (!tlds.contains(tld)) {
classifiedTasksBuilder.tasksToKeepBuilder().add(task);
classifiedTasksBuilder.unknownTldsBuilder().add(tld);
} else if (Tld.get(tld).getDnsPaused()) {
classifiedTasksBuilder.tasksToKeepBuilder().add(task);
classifiedTasksBuilder.pausedTldsBuilder().add(tld);
} else {
String typeString = params.get(DNS_TARGET_TYPE_PARAM);
String name = params.get(DNS_TARGET_NAME_PARAM);
TargetType type = TargetType.valueOf(typeString);
switch (type) {
case DOMAIN:
case HOST:
classifiedTasksBuilder
.refreshItemsByTldBuilder()
.put(tld, RefreshItem.create(type, name, creationTime));
break;
default:
logger.atSevere().log(
"Discarding DNS refresh request %s of type %s.", task, typeString);
break;
}
}
} catch (RuntimeException | UnsupportedEncodingException e) {
logger.atSevere().withCause(e).log("Discarding invalid DNS refresh request %s.", task);
}
}
return classifiedTasksBuilder.build();
}
/**
* Subdivides the tld to {@link RefreshItem} multimap into buckets by lock index, if applicable.
*
* <p>If the tld has numDnsPublishLocks <= 1, we enqueue all updates on the default lock 1 of 1.
*/
private void bucketRefreshItems(ImmutableSetMultimap<String, RefreshItem> refreshItemsByTld) {
// Loop through the multimap by TLD and generate refresh tasks for the hosts and domains for
// each configured DNS writer.
for (Map.Entry<String, Collection<RefreshItem>> tldRefreshItemsEntry
: refreshItemsByTld.asMap().entrySet()) {
String tld = tldRefreshItemsEntry.getKey();
int numPublishLocks = Tld.get(tld).getNumDnsPublishLocks();
// 1 lock or less implies no TLD-wide locks, simply enqueue everything under lock 1 of 1
if (numPublishLocks <= 1) {
enqueueUpdates(tld, 1, 1, tldRefreshItemsEntry.getValue());
} else {
tldRefreshItemsEntry.getValue().stream()
.collect(
toImmutableSetMultimap(
refreshItem -> getLockIndex(tld, numPublishLocks, refreshItem),
refreshItem -> refreshItem))
.asMap()
.forEach((key, value) -> enqueueUpdates(tld, key, numPublishLocks, value));
}
}
}
/**
* Returns the lock index for a given refreshItem.
*
* <p>We hash the second level domain for all records, to group in-bailiwick hosts (the only ones
* we refresh DNS for) with their superordinate domains. We use consistent hashing to determine
* the lock index because it gives us [0,N) bucketing properties out of the box, then add 1 to
* make indexes within [1,N].
*/
private int getLockIndex(String tld, int numPublishLocks, RefreshItem refreshItem) {
String domain = getSecondLevelDomain(refreshItem.name(), tld);
return Hashing.consistentHash(hashFunction.hashString(domain, UTF_8), numPublishLocks) + 1;
}
/**
* Creates DNS refresh tasks for all writers for the tld within a lock index and batches large
* updates into smaller chunks.
*/
private void enqueueUpdates(
String tld, int lockIndex, int numPublishLocks, Collection<RefreshItem> items) {
for (List<RefreshItem> chunk : Iterables.partition(items, tldUpdateBatchSize)) {
DateTime earliestCreateTime =
chunk.stream().map(RefreshItem::creationTime).min(Comparator.naturalOrder()).get();
for (String dnsWriter : Tld.get(tld).getDnsWriters()) {
Task task =
cloudTasksUtils.createPostTaskWithJitter(
PublishDnsUpdatesAction.PATH,
Service.BACKEND,
ImmutableMultimap.<String, String>builder()
.put(PARAM_TLD, tld)
.put(PARAM_DNS_WRITER, dnsWriter)
.put(PARAM_LOCK_INDEX, Integer.toString(lockIndex))
.put(PARAM_NUM_PUBLISH_LOCKS, Integer.toString(numPublishLocks))
.put(PARAM_PUBLISH_TASK_ENQUEUED, clock.nowUtc().toString())
.put(PARAM_REFRESH_REQUEST_TIME, earliestCreateTime.toString())
.put(
PARAM_DOMAINS,
chunk.stream()
.filter(item -> item.type() == TargetType.DOMAIN)
.map(RefreshItem::name)
.collect(Collectors.joining(",")))
.put(
PARAM_HOSTS,
chunk.stream()
.filter(item -> item.type() == TargetType.HOST)
.map(RefreshItem::name)
.collect(Collectors.joining(",")))
.build(),
jitterSeconds);
cloudTasksUtils.enqueue(DNS_PUBLISH_PUSH_QUEUE_NAME, task);
}
}
}
}
@@ -15,7 +15,6 @@
package google.registry.dns;
import static com.google.common.collect.ImmutableSetMultimap.toImmutableSetMultimap;
import static google.registry.dns.DnsConstants.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.DnsModule.PARAM_DNS_JITTER_SECONDS;
import static google.registry.dns.DnsModule.PARAM_DNS_WRITER;
import static google.registry.dns.DnsModule.PARAM_DOMAINS;
@@ -24,6 +23,9 @@ import static google.registry.dns.DnsModule.PARAM_LOCK_INDEX;
import static google.registry.dns.DnsModule.PARAM_NUM_PUBLISH_LOCKS;
import static google.registry.dns.DnsModule.PARAM_PUBLISH_TASK_ENQUEUED;
import static google.registry.dns.DnsModule.PARAM_REFRESH_REQUEST_TIME;
import static google.registry.dns.DnsUtils.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.DnsUtils.deleteRequests;
import static google.registry.dns.DnsUtils.readAndUpdateRequestsWithLatestProcessTime;
import static google.registry.request.Action.Method.POST;
import static google.registry.request.RequestParameters.PARAM_TLD;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
@@ -39,7 +41,7 @@ import com.google.common.hash.HashFunction;
import com.google.common.hash.Hashing;
import google.registry.batch.CloudTasksUtils;
import google.registry.config.RegistryConfig.Config;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.dns.DnsUtils.TargetType;
import google.registry.model.common.DnsRefreshRequest;
import google.registry.model.tld.Tld;
import google.registry.request.Action;
@@ -72,7 +74,6 @@ public final class ReadDnsRefreshRequestsAction implements Runnable {
private final Optional<Integer> jitterSeconds;
private final String tld;
private final Clock clock;
private final DnsUtils dnsUtils;
private final HashFunction hashFunction;
private final CloudTasksUtils cloudTasksUtils;
@@ -83,7 +84,6 @@ public final class ReadDnsRefreshRequestsAction implements Runnable {
@Parameter(PARAM_DNS_JITTER_SECONDS) Optional<Integer> jitterSeconds,
@Parameter(PARAM_TLD) String tld,
Clock clock,
DnsUtils dnsUtils,
HashFunction hashFunction,
CloudTasksUtils cloudTasksUtils) {
this.tldUpdateBatchSize = tldUpdateBatchSize;
@@ -91,7 +91,6 @@ public final class ReadDnsRefreshRequestsAction implements Runnable {
this.jitterSeconds = jitterSeconds;
this.tld = tld;
this.clock = clock;
this.dnsUtils = dnsUtils;
this.hashFunction = hashFunction;
this.cloudTasksUtils = cloudTasksUtils;
}
@@ -112,7 +111,7 @@ public final class ReadDnsRefreshRequestsAction implements Runnable {
int processBatchSize = tldUpdateBatchSize * Tld.get(tld).getNumDnsPublishLocks();
while (requestedEndTime.isAfter(clock.nowUtc())) {
ImmutableList<DnsRefreshRequest> requests =
dnsUtils.readAndUpdateRequestsWithLatestProcessTime(
readAndUpdateRequestsWithLatestProcessTime(
tld, requestedMaximumDuration, processBatchSize);
logger.atInfo().log("Read %d DNS update requests for TLD %s.", requests.size(), tld);
if (!requests.isEmpty()) {
@@ -138,7 +137,7 @@ public final class ReadDnsRefreshRequestsAction implements Runnable {
(lockIndex, bucketedRequests) -> {
try {
enqueueUpdates(lockIndex, numPublishLocks, bucketedRequests);
dnsUtils.deleteRequests(bucketedRequests);
deleteRequests(bucketedRequests);
logger.atInfo().log(
"Processed %d DNS update requests for TLD %s.", bucketedRequests.size(), tld);
} catch (Exception e) {
@@ -14,9 +14,11 @@
package google.registry.dns;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.dns.DnsUtils.requestHostDnsRefresh;
import static google.registry.model.EppResourceUtils.loadByForeignKey;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.dns.DnsUtils.TargetType;
import google.registry.model.EppResource;
import google.registry.model.EppResource.ForeignKeyedEppResource;
import google.registry.model.annotations.ExternalMessagingName;
@@ -39,7 +41,6 @@ import javax.inject.Inject;
public final class RefreshDnsAction implements Runnable {
private final Clock clock;
private final DnsUtils dnsUtils;
private final String domainOrHostName;
private final TargetType type;
@@ -47,12 +48,10 @@ public final class RefreshDnsAction implements Runnable {
RefreshDnsAction(
@Parameter("domainOrHostName") String domainOrHostName,
@Parameter("type") TargetType type,
Clock clock,
DnsUtils dnsUtils) {
Clock clock) {
this.domainOrHostName = domainOrHostName;
this.type = type;
this.clock = clock;
this.dnsUtils = dnsUtils;
}
@Override
@@ -63,11 +62,11 @@ public final class RefreshDnsAction implements Runnable {
switch (type) {
case DOMAIN:
loadAndVerifyExistence(Domain.class, domainOrHostName);
dnsUtils.requestDomainDnsRefresh(domainOrHostName);
requestDomainDnsRefresh(domainOrHostName);
break;
case HOST:
verifyHostIsSubordinate(loadAndVerifyExistence(Host.class, domainOrHostName));
dnsUtils.requestHostDnsRefresh(domainOrHostName);
requestHostDnsRefresh(domainOrHostName);
break;
default:
throw new BadRequestException("Unsupported type: " + type);
@@ -14,6 +14,7 @@
package google.registry.dns;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.dns.RefreshDnsOnHostRenameAction.PATH;
import static google.registry.model.EppResourceUtils.getLinkedDomainKeys;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
@@ -45,14 +46,11 @@ public class RefreshDnsOnHostRenameAction implements Runnable {
private final VKey<Host> hostKey;
private final Response response;
private final DnsUtils dnsUtils;
@Inject
RefreshDnsOnHostRenameAction(
@Parameter(PARAM_HOST_KEY) String hostKey, Response response, DnsUtils dnsUtils) {
RefreshDnsOnHostRenameAction(@Parameter(PARAM_HOST_KEY) String hostKey, Response response) {
this.hostKey = VKey.createEppVKeyFromString(hostKey);
this.response = response;
this.dnsUtils = dnsUtils;
}
@Override
@@ -76,7 +74,7 @@ public class RefreshDnsOnHostRenameAction implements Runnable {
.stream()
.map(domainKey -> tm().loadByKey(domainKey))
.filter(Domain::shouldPublishToDns)
.forEach(domain -> dnsUtils.requestDomainDnsRefresh(domain.getDomainName()));
.forEach(domain -> requestDomainDnsRefresh(domain.getDomainName()));
}
if (!hostValid) {
@@ -173,7 +173,7 @@ public class CloudDnsWriter extends BaseDnsWriter {
domainRecords.add(
new ResourceRecordSet()
.setName(absoluteDomainName)
.setTtl((int) (tld.getDnsNsTtl().orElse(defaultNsTtl).getStandardSeconds()))
.setTtl((int) tld.getDnsNsTtl().orElse(defaultNsTtl).getStandardSeconds())
.setType("NS")
.setKind("dns#resourceRecordSet")
.setRrdatas(ImmutableList.copyOf(nsRrData)));
@@ -279,11 +279,12 @@ public class CloudDnsWriter extends BaseDnsWriter {
}
/** Returns the glue records for in-bailiwick nameservers for the given domain+records. */
private Stream<String> filterGlueRecords(String domainName, Stream<ResourceRecordSet> records) {
private static Stream<String> filterGlueRecords(
String domainName, Stream<ResourceRecordSet> records) {
return records
.filter(record -> record.getType().equals("NS"))
.filter(record -> "NS".equals(record.getType()))
.flatMap(record -> record.getRrdatas().stream())
.filter(hostName -> hostName.endsWith("." + domainName) && !hostName.equals(domainName));
.filter(hostName -> hostName.endsWith('.' + domainName) && !hostName.equals(domainName));
}
/** Mutate the zone with the provided map of hostnames to desired DNS records. */
@@ -366,8 +367,8 @@ public class CloudDnsWriter extends BaseDnsWriter {
* <p>This call should be used in conjunction with {@link #getResourceRecordsForDomains} in a
* get-and-set retry loop.
*
* <p>See {@link "https://cloud.google.com/dns/troubleshooting"} for a list of errors produced by
* the Google Cloud DNS API.
* <p>See {@link "<a href="https://cloud.google.com/dns/troubleshooting">Troubleshoot Cloud
* DNS</a>"} for a list of errors produced by the Google Cloud DNS API.
*
* @throws ZoneStateException if the operation could not be completely successfully because the
* records to delete do not exist, already exist or have been modified with different
@@ -420,12 +421,12 @@ public class CloudDnsWriter extends BaseDnsWriter {
* @param hostName the fully qualified hostname
*/
private static String getAbsoluteHostName(String hostName) {
return hostName.endsWith(".") ? hostName : hostName + ".";
return hostName.endsWith(".") ? hostName : hostName + '.';
}
/** Zone state on Cloud DNS does not match the expected state. */
static class ZoneStateException extends RuntimeException {
public ZoneStateException(String reason) {
ZoneStateException(String reason) {
super("Zone state on Cloud DNS does not match the expected state: " + reason);
}
}
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<taskentries>
<entries>
<task>
<url>/_dr/task/rdeStaging</url>
<name>rdeStaging</name>
@@ -80,12 +80,13 @@
</task>
<task>
<url><![CDATA[/_dr/task/expandRecurringBillingEvents?advanceCursor]]></url>
<name>expandRecurringBillingEvents</name>
<url><![CDATA[/_dr/task/expandBillingRecurrences?advanceCursor]]></url>
<name>expandBillingRecurrences</name>
<description>
This job runs an action that creates synthetic OneTime billing events from Recurring billing
events. Events are created for all instances of Recurring billing events that should exist
between the RECURRING_BILLING cursor's time and the execution time of the action.
This job runs an action that creates synthetic one-time billing events
from billing recurrences. Events are created for all recurrences that
should exist between the RECURRING_BILLING cursor's time and the execution
time of the action.
</description>
<schedule>0 3 * * *</schedule>
</task>
@@ -128,16 +129,6 @@
<schedule>0 5 * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/cron/readDnsQueue?jitterSeconds=45]]></url>
<name>readDnsQueue</name>
<description>
Lease all tasks from the dns-pull queue, group by TLD, and invoke PublishDnsUpdates for each
group.
</description>
<schedule>*/1 * * * *</schedule>
</task>
<task>
<url>
<![CDATA[/_dr/cron/fanout?queue=dns-refresh&forEachRealTld&forEachTestTld&endpoint=/_dr/task/readDnsRefreshRequests&dnsJitterSeconds=45]]></url>
@@ -147,4 +138,4 @@
</description>
<schedule>*/1 * * * *</schedule>
</task>
</taskentries>
</entries>
@@ -151,12 +151,6 @@
<url-pattern>/_dr/task/nordnVerify</url-pattern>
</servlet-mapping>
<!-- Reads the DNS push and pull queues and kick off the appropriate tasks to update zone. -->
<servlet-mapping>
<servlet-name>backend-servlet</servlet-name>
<url-pattern>/_dr/cron/readDnsQueue</url-pattern>
</servlet-mapping>
<!-- Reads the DNS refresh requests and kick off the appropriate tasks to update zone. -->
<servlet-mapping>
<servlet-name>backend-servlet</servlet-name>
@@ -246,10 +240,10 @@
<url-pattern>/_dr/task/refreshDnsOnHostRename</url-pattern>
</servlet-mapping>
<!-- Action to expand recurring billing events into OneTimes. -->
<!-- Action to expand BillingRecurrences into BillingEvents. -->
<servlet-mapping>
<servlet-name>backend-servlet</servlet-name>
<url-pattern>/_dr/task/expandRecurringBillingEvents</url-pattern>
<url-pattern>/_dr/task/expandBillingRecurrences</url-pattern>
</servlet-mapping>
<!-- Background action to delete domains past end of autorenewal. -->
@@ -0,0 +1,113 @@
<?xml version="1.0" encoding="UTF-8"?>
<entries>
<!-- Queue template with all supported params -->
<!-- More information - https://cloud.google.com/sdk/gcloud/reference/tasks/queues/create -->
<!--
<queue>
<name></name>
<max-attempts></max-attempts>
<max-backoff></max-backoff>
<max-concurrent-dispatches></max-concurrent-dispatches>
<max-dispatches-per-second></max-dispatches-per-second>
<max-doublings></max-doublings>
<max-retry-duration></max-retry-duration>
<min-backoff></min-backoff>
</queue>
-->
<!-- Queue for reading DNS update requests and batching them off to the dns-publish queue. -->
<queue>
<name>dns-refresh</name>
<max-dispatches-per-second>100</max-dispatches-per-second>
</queue>
<!-- Queue for publishing DNS updates in batches. -->
<queue>
<name>dns-publish</name>
<max-dispatches-per-second>100</max-dispatches-per-second>
<!-- 30 sec backoff increasing linearly up to 30 minutes. -->
<min-backoff>30s</min-backoff>
<max-backoff>1800s</max-backoff>
<max-doublings>0</max-doublings>
</queue>
<!-- Queue for uploading RDE deposits to the escrow provider. -->
<queue>
<name>rde-upload</name>
<max-dispatches-per-second>0.166666667</max-dispatches-per-second>
<max-concurrent-dispatches>5</max-concurrent-dispatches>
<max-retry-duration>14400s</max-retry-duration>
</queue>
<!-- Queue for uploading RDE reports to ICANN. -->
<queue>
<name>rde-report</name>
<max-dispatches-per-second>1</max-dispatches-per-second>
<max-concurrent-dispatches>1</max-concurrent-dispatches>
<max-retry-duration>14400s</max-retry-duration>
</queue>
<!-- Queue for copying BRDA deposits to GCS. -->
<queue>
<name>brda</name>
<max-dispatches-per-second>0.016666667</max-dispatches-per-second>
<max-concurrent-dispatches>10</max-concurrent-dispatches>
<max-retry-duration>82800s</max-retry-duration>
</queue>
<!-- Queue for tasks that trigger domain DNS update upon host rename. -->
<queue>
<name>async-host-rename</name>
<max-dispatches-per-second>1</max-dispatches-per-second>
</queue>
<!-- Queue for tasks that wait for a Beam pipeline to complete (i.e. Spec11 and invoicing). -->
<queue>
<name>beam-reporting</name>
<max-dispatches-per-second>0.016666667</max-dispatches-per-second>
<max-concurrent-dispatches>1</max-concurrent-dispatches>
<max-attempts>5</max-attempts>
<min-backoff>180s</min-backoff>
<max-backoff>180s</max-backoff>
</queue>
<!-- Queue for tasks that communicate with TMCH MarksDB webserver. -->
<queue>
<name>marksdb</name>
<max-dispatches-per-second>0.016666667</max-dispatches-per-second>
<max-concurrent-dispatches>1</max-concurrent-dispatches>
<max-retry-duration>39600s</max-retry-duration> <!-- cron interval minus hour -->
</queue>
<!-- Queue for tasks to produce LORDN CSV reports, populated by a Cloud Scheduler fanout job. -->
<queue>
<name>nordn</name>
<max-dispatches-per-second>1</max-dispatches-per-second>
<max-concurrent-dispatches>10</max-concurrent-dispatches>
<max-retry-duration>39600s</max-retry-duration> <!-- cron interval minus hour -->
</queue>
<!-- Queue for tasks that sync data to Google Spreadsheets. -->
<queue>
<name>sheet</name>
<max-dispatches-per-second>1</max-dispatches-per-second>
<!-- max-concurrent-dispatches is intentionally omitted. -->
<max-retry-duration>3600s</max-retry-duration>
</queue>
<!-- Queue for infrequent cron tasks (i.e. hourly or less often) that should retry three times on failure. -->
<queue>
<name>retryable-cron-tasks</name>
<max-dispatches-per-second>1</max-dispatches-per-second>
<max-attempts>3</max-attempts>
</queue>
<!-- &lt;!&ndash; Queue for async actions that should be run at some point in the future. &ndash;&gt;-->
<queue>
<name>async-actions</name>
<max-dispatches-per-second>1</max-dispatches-per-second>
<max-concurrent-dispatches>5</max-concurrent-dispatches>
</queue>
</entries>
@@ -1,11 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<!-- TODO: @ptkach - Delete once Cloud Api deployer is up and running -->
<queue-entries>
<queue>
<name>dns-pull</name>
<mode>pull</mode>
</queue>
<!-- Queue for reading DNS update requests and batching them off to the dns-publish queue. -->
<queue>
<name>dns-refresh</name>
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<taskentries>
<entries>
<!--
/cron/fanout params:
@@ -129,16 +129,6 @@
<schedule>0 5 * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/cron/readDnsQueue?jitterSeconds=45]]></url>
<name>readDnsQueue</name>
<description>
Lease all tasks from the dns-pull queue, group by TLD, and invoke PublishDnsUpdates for each
group.
</description>
<schedule>*/1 * * * *</schedule>
</task>
<task>
<url>
<![CDATA[/_dr/cron/fanout?queue=dns-refresh&forEachRealTld&forEachTestTld&endpoint=/_dr/task/readDnsRefreshRequests&dnsJitterSeconds=45]]></url>
@@ -158,16 +148,4 @@
</description>
<schedule>7 3 * * *</schedule>
</task>
<!--
The next two wipeout jobs are required when crash has production data.
-->
<task>
<url><![CDATA[/_dr/task/wipeOutCloudSql]]></url>
<name>wipeOutCloudSql</name>
<description>
This job runs an action that deletes all data in Cloud SQL.
</description>
<schedule>7 3 * * 6</schedule>
</task>
</taskentries>
</entries>
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<taskentries>
<entries>
<task>
<url>/_dr/task/rdeStaging</url>
<name>rdeStaging</name>
@@ -122,12 +122,13 @@
</task>
<task>
<url><![CDATA[/_dr/task/expandRecurringBillingEvents?advanceCursor]]></url>
<name>expandRecurringBillingEvents</name>
<url><![CDATA[/_dr/task/expandBillingRecurrences?advanceCursor]]></url>
<name>expandBillingRecurrences</name>
<description>
This job runs an action that creates synthetic OneTime billing events from Recurring billing
events. Events are created for all instances of Recurring billing events that should exist
between the RECURRING_BILLING cursor's time and the execution time of the action.
This job runs an action that creates synthetic one-time billing events
from billing recurrences. Events are created for all recurrences that
should exist between the RECURRING_BILLING cursor's time and the execution
time of the action.
</description>
<schedule>0 3 * * *</schedule>
</task>
@@ -200,16 +201,6 @@
<schedule>0 5 * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/cron/readDnsQueue?jitterSeconds=45]]></url>
<name>readDnsQueue</name>
<description>
Lease all tasks from the dns-pull queue, group by TLD, and invoke PublishDnsUpdates for each
group.
</description>
<schedule>*/1 * * * *</schedule>
</task>
<task>
<url>
<![CDATA[/_dr/cron/fanout?queue=dns-refresh&forEachRealTld&forEachTestTld&endpoint=/_dr/task/readDnsRefreshRequests&dnsJitterSeconds=45]]></url>
@@ -253,7 +244,7 @@
reports to the associated registrars' drive folders.
See GenerateInvoicesAction for more details.
</description>
<!--WARNING: This must occur AFTER expandRecurringBillingEvents and AFTER exportSnapshot, as
<!--WARNING: This must occur AFTER expandBillingRecurrences and AFTER exportSnapshot, as
it uses Bigquery as the source of truth for billable events. ExportSnapshot usually takes
about 2 hours to complete, so we give 11 hours to be safe. Normally, we give 24+ hours (see
icannReportingStaging), but the invoicing team prefers receiving the e-mail on the first of
@@ -282,4 +273,4 @@
</description>
<schedule>0 15 * * 1</schedule>
</task>
</taskentries>
</entries>
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<taskentries>
<entries>
<task>
<url>/_dr/task/rdeStaging</url>
<name>rdeStaging</name>
@@ -11,16 +11,6 @@
<schedule>7 0 * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/cron/readDnsQueue?jitterSeconds=45]]></url>
<name>readDnsQueue</name>
<description>
Lease all tasks from the dns-pull queue, group by TLD, and invoke PublishDnsUpdates for each
group.
</description>
<schedule>*/1 * * * *</schedule>
</task>
<task>
<url>
<![CDATA[/_dr/cron/fanout?queue=dns-refresh&forEachRealTld&forEachTestTld&endpoint=/_dr/task/readDnsRefreshRequests&dnsJitterSeconds=45]]></url>
@@ -78,4 +68,4 @@
</description>
<schedule>7 3 * * 6</schedule>
</task>
</taskentries>
</entries>
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<taskentries>
<entries>
<task>
<url>/_dr/task/rdeStaging</url>
<name>rdeStaging</name>
@@ -82,12 +82,13 @@
</task>
<task>
<url><![CDATA[/_dr/task/expandRecurringBillingEvents?advanceCursor]]></url>
<name>expandRecurringBillingEvents</name>
<url><![CDATA[/_dr/task/expandBillingRecurrences?advanceCursor]]></url>
<name>expandBillingRecurrences</name>
<description>
This job runs an action that creates synthetic OneTime billing events from Recurring billing
events. Events are created for all instances of Recurring billing events that should exist
between the RECURRING_BILLING cursor's time and the execution time of the action.
This job runs an action that creates synthetic one-time billing events
from billing recurrences. Events are created for all recurrences that
should exist between the RECURRING_BILLING cursor's time and the execution
time of the action.
</description>
<schedule>0 3 * * *</schedule>
</task>
@@ -142,16 +143,6 @@
<schedule>0 5 * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/cron/readDnsQueue?jitterSeconds=45]]></url>
<name>readDnsQueue</name>
<description>
Lease all tasks from the dns-pull queue, group by TLD, and invoke PublishDnsUpdates for each
group.
</description>
<schedule>*/1 * * * *</schedule>
</task>
<task>
<url>
<![CDATA[/_dr/cron/fanout?queue=dns-refresh&forEachRealTld&forEachTestTld&endpoint=/_dr/task/readDnsRefreshRequests&dnsJitterSeconds=45]]></url>
@@ -171,4 +162,4 @@
</description>
<schedule>0 15 * * 1</schedule>
</task>
</taskentries>
</entries>
@@ -50,6 +50,7 @@ import google.registry.flows.annotations.ReportingSpec;
import google.registry.flows.custom.DomainCheckFlowCustomLogic;
import google.registry.flows.custom.DomainCheckFlowCustomLogic.BeforeResponseParameters;
import google.registry.flows.custom.DomainCheckFlowCustomLogic.BeforeResponseReturnData;
import google.registry.flows.domain.DomainPricingLogic.AllocationTokenInvalidForPremiumNameException;
import google.registry.flows.domain.token.AllocationTokenDomainCheckResults;
import google.registry.flows.domain.token.AllocationTokenFlowUtils;
import google.registry.flows.domain.token.AllocationTokenFlowUtils.AllocationTokenNotInPromotionException;
@@ -59,7 +60,7 @@ import google.registry.flows.domain.token.AllocationTokenFlowUtils.AllocationTok
import google.registry.flows.domain.token.AllocationTokenFlowUtils.AllocationTokenNotValidForTldException;
import google.registry.model.EppResource;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Check;
import google.registry.model.domain.fee.FeeCheckCommandExtension;
@@ -81,6 +82,7 @@ import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldState;
import google.registry.model.tld.label.ReservationType;
import google.registry.persistence.VKey;
import google.registry.pricing.PricingEngineProxy;
import google.registry.util.Clock;
import java.util.HashSet;
import java.util.Optional;
@@ -271,8 +273,7 @@ public final class DomainCheckFlow implements Flow {
new ImmutableList.Builder<>();
ImmutableMap<String, Domain> domainObjs =
loadDomainsForRestoreChecks(feeCheck, domainNames, existingDomains);
ImmutableMap<String, BillingEvent.Recurring> recurrences =
loadRecurrencesForDomains(domainObjs);
ImmutableMap<String, BillingRecurrence> recurrences = loadRecurrencesForDomains(domainObjs);
for (FeeCheckCommandExtensionItem feeCheckItem : feeCheck.getItems()) {
for (String domainName : getDomainNamesToCheckForFee(feeCheckItem, domainNames.keySet())) {
@@ -292,6 +293,7 @@ public final class DomainCheckFlow implements Flow {
allocationToken.get(),
feeCheckItem.getCommandName(),
registrarId,
PricingEngineProxy.isDomainPremium(domainName, now),
now);
}
handleFeeRequest(
@@ -306,7 +308,8 @@ public final class DomainCheckFlow implements Flow {
availableDomains.contains(domainName),
recurrences.getOrDefault(domainName, null));
responseItems.add(builder.setDomainNameIfSupported(domainName).build());
} catch (AllocationTokenNotValidForCommandException
} catch (AllocationTokenInvalidForPremiumNameException
| AllocationTokenNotValidForCommandException
| AllocationTokenNotValidForDomainException
| AllocationTokenNotValidForRegistrarException
| AllocationTokenNotValidForTldException
@@ -377,16 +380,15 @@ public final class DomainCheckFlow implements Flow {
Maps.transformEntries(existingDomainsToLoad, (k, v) -> (Domain) loadedDomains.get(v)));
}
private ImmutableMap<String, BillingEvent.Recurring> loadRecurrencesForDomains(
private ImmutableMap<String, BillingRecurrence> loadRecurrencesForDomains(
ImmutableMap<String, Domain> domainObjs) {
return tm().transact(
() -> {
ImmutableMap<VKey<? extends BillingEvent.Recurring>, BillingEvent.Recurring>
recurrences =
tm().loadByKeys(
domainObjs.values().stream()
.map(Domain::getAutorenewBillingEvent)
.collect(toImmutableSet()));
ImmutableMap<VKey<? extends BillingRecurrence>, BillingRecurrence> recurrences =
tm().loadByKeys(
domainObjs.values().stream()
.map(Domain::getAutorenewBillingEvent)
.collect(toImmutableSet()));
return ImmutableMap.copyOf(
Maps.transformValues(
domainObjs, d -> recurrences.get(d.getAutorenewBillingEvent())));
@@ -16,6 +16,7 @@ package google.registry.flows.domain;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.flows.FlowUtils.persistEntityChanges;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.verifyResourceDoesNotExist;
@@ -59,7 +60,6 @@ import com.google.auto.value.AutoValue;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.net.InternetDomainName;
import google.registry.dns.DnsUtils;
import google.registry.flows.EppException;
import google.registry.flows.EppException.CommandUseErrorException;
import google.registry.flows.EppException.ParameterValuePolicyErrorException;
@@ -77,11 +77,11 @@ import google.registry.flows.domain.token.AllocationTokenFlowUtils;
import google.registry.flows.exceptions.ResourceAlreadyExistsForThisClientException;
import google.registry.flows.exceptions.ResourceCreateContentionException;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingBase.Flag;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand;
import google.registry.model.domain.DomainCommand.Create;
@@ -227,7 +227,6 @@ public final class DomainCreateFlow implements TransactionalFlow {
@Inject DomainCreateFlowCustomLogic flowCustomLogic;
@Inject DomainFlowTmchUtils tmchUtils;
@Inject DomainPricingLogic pricingLogic;
@Inject DnsUtils dnsUtils;
@Inject DomainCreateFlow() {}
@@ -348,8 +347,8 @@ public final class DomainCreateFlow implements TransactionalFlow {
HistoryEntryId domainHistoryId = new HistoryEntryId(repoId, historyRevisionId);
historyBuilder.setRevisionId(historyRevisionId);
// Bill for the create.
BillingEvent.OneTime createBillingEvent =
createOneTimeBillingEvent(
BillingEvent createBillingEvent =
createBillingEvent(
tld,
isAnchorTenant,
isSunriseCreate,
@@ -360,7 +359,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
allocationToken,
now);
// Create a new autorenew billing event and poll message starting at the expiration time.
BillingEvent.Recurring autorenewBillingEvent =
BillingRecurrence autorenewBillingEvent =
createAutorenewBillingEvent(
domainHistoryId,
registrationExpirationTime,
@@ -426,7 +425,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
allocationToken.get(), domainHistory.getHistoryEntryId()));
}
if (domain.shouldPublishToDns()) {
dnsUtils.requestDomainDnsRefresh(domain.getDomainName());
requestDomainDnsRefresh(domain.getDomainName());
}
EntityChanges entityChanges =
flowCustomLogic.beforeSave(
@@ -572,7 +571,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
return historyBuilder.setType(DOMAIN_CREATE).setPeriod(period).setDomain(domain).build();
}
private BillingEvent.OneTime createOneTimeBillingEvent(
private BillingEvent createBillingEvent(
Tld tld,
boolean isAnchorTenant,
boolean isSunriseCreate,
@@ -594,7 +593,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
// it if it's reserved for other reasons.
flagsBuilder.add(Flag.RESERVED);
}
return new BillingEvent.OneTime.Builder()
return new BillingEvent.Builder()
.setReason(Reason.CREATE)
.setTargetId(targetId)
.setRegistrarId(registrarId)
@@ -612,11 +611,11 @@ public final class DomainCreateFlow implements TransactionalFlow {
.build();
}
private Recurring createAutorenewBillingEvent(
private BillingRecurrence createAutorenewBillingEvent(
HistoryEntryId domainHistoryId,
DateTime registrationExpirationTime,
RenewalPriceInfo renewalpriceInfo) {
return new BillingEvent.Recurring.Builder()
return new BillingRecurrence.Builder()
.setReason(Reason.RENEW)
.setFlags(ImmutableSet.of(Flag.AUTO_RENEW))
.setTargetId(targetId)
@@ -640,9 +639,9 @@ public final class DomainCreateFlow implements TransactionalFlow {
.build();
}
private static BillingEvent.OneTime createEapBillingEvent(
FeesAndCredits feesAndCredits, BillingEvent.OneTime createBillingEvent) {
return new BillingEvent.OneTime.Builder()
private static BillingEvent createEapBillingEvent(
FeesAndCredits feesAndCredits, BillingEvent createBillingEvent) {
return new BillingEvent.Builder()
.setReason(Reason.FEE_EARLY_ACCESS)
.setTargetId(createBillingEvent.getTargetId())
.setRegistrarId(createBillingEvent.getRegistrarId())
@@ -671,7 +670,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
/**
* Determines the {@link RenewalPriceBehavior} and the renewal price that needs be stored in the
* {@link Recurring} billing events.
* {@link BillingRecurrence} billing events.
*
* <p>By default, the renewal price is calculated during the process of renewal. Renewal price
* should be the createCost if and only if the renewal price behavior in the {@link
@@ -697,7 +696,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
}
}
/** A class to store renewal info used in {@link Recurring} billing events. */
/** A class to store renewal info used in {@link BillingRecurrence} billing events. */
@AutoValue
public abstract static class RenewalPriceInfo {
static DomainCreateFlow.RenewalPriceInfo create(
@@ -16,6 +16,7 @@ package google.registry.flows.domain;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Strings.isNullOrEmpty;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.persistEntityChanges;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
@@ -44,7 +45,6 @@ import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Sets;
import google.registry.batch.AsyncTaskEnqueuer;
import google.registry.dns.DnsUtils;
import google.registry.flows.EppException;
import google.registry.flows.EppException.AssociationProhibitsOperationException;
import google.registry.flows.ExtensionManager;
@@ -61,8 +61,9 @@ import google.registry.flows.custom.DomainDeleteFlowCustomLogic.BeforeResponseRe
import google.registry.flows.custom.DomainDeleteFlowCustomLogic.BeforeSaveParameters;
import google.registry.flows.custom.EntityChanges;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingCancellation;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.GracePeriod;
@@ -129,7 +130,6 @@ public final class DomainDeleteFlow implements TransactionalFlow {
@Inject @TargetId String targetId;
@Inject @Superuser boolean isSuperuser;
@Inject DomainHistory.Builder historyBuilder;
@Inject DnsUtils dnsUtils;
@Inject Trid trid;
@Inject AsyncTaskEnqueuer asyncTaskEnqueuer;
@Inject EppResponse.Builder responseBuilder;
@@ -232,15 +232,15 @@ public final class DomainDeleteFlow implements TransactionalFlow {
// No cancellation is written if the grace period was not for a billable event.
if (gracePeriod.hasBillingEvent()) {
entitiesToSave.add(
BillingEvent.Cancellation.forGracePeriod(gracePeriod, now, domainHistoryId, targetId));
if (gracePeriod.getOneTimeBillingEvent() != null) {
BillingCancellation.forGracePeriod(gracePeriod, now, domainHistoryId, targetId));
if (gracePeriod.getBillingEvent() != null) {
// Take the amount of registration time being refunded off the expiration time.
// This can be either add grace periods or renew grace periods.
BillingEvent.OneTime oneTime = tm().loadByKey(gracePeriod.getOneTimeBillingEvent());
newExpirationTime = newExpirationTime.minusYears(oneTime.getPeriodYears());
} else if (gracePeriod.getRecurringBillingEvent() != null) {
BillingEvent billingEvent = tm().loadByKey(gracePeriod.getBillingEvent());
newExpirationTime = newExpirationTime.minusYears(billingEvent.getPeriodYears());
} else if (gracePeriod.getBillingRecurrence() != null) {
// Take 1 year off the registration if in the autorenew grace period (no need to load the
// recurring billing event; all autorenews are for 1 year).
// recurrence billing event; all autorenews are for 1 year).
newExpirationTime = newExpirationTime.minusYears(1);
}
}
@@ -252,15 +252,16 @@ public final class DomainDeleteFlow implements TransactionalFlow {
buildDomainHistory(newDomain, tld, now, durationUntilDelete, inAddGracePeriod);
handlePendingTransferOnDelete(existingDomain, newDomain, now, domainHistory);
// Close the autorenew billing event and poll message. This may delete the poll message. Store
// the updated recurring billing event, we'll need it later and can't reload it.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
BillingEvent.Recurring recurringBillingEvent =
// the updated recurrence billing event, we'll need it later and can't reload it.
BillingRecurrence existingBillingRecurrence =
tm().loadByKey(existingDomain.getAutorenewBillingEvent());
BillingRecurrence billingRecurrence =
updateAutorenewRecurrenceEndTime(
existingDomain, existingRecurring, now, domainHistory.getHistoryEntryId());
existingDomain, existingBillingRecurrence, now, domainHistory.getHistoryEntryId());
// If there's a pending transfer, the gaining client's autorenew billing
// event and poll message will already have been deleted in
// ResourceDeleteFlow since it's listed in serverApproveEntities.
dnsUtils.requestDomainDnsRefresh(existingDomain.getDomainName());
requestDomainDnsRefresh(existingDomain.getDomainName());
entitiesToSave.add(newDomain, domainHistory);
EntityChanges entityChanges =
@@ -280,7 +281,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
? SUCCESS_WITH_ACTION_PENDING
: SUCCESS)
.setResponseExtensions(
getResponseExtensions(recurringBillingEvent, existingDomain, now))
getResponseExtensions(billingRecurrence, existingDomain, now))
.build());
persistEntityChanges(entityChanges);
return responseBuilder
@@ -380,7 +381,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
@Nullable
private ImmutableList<FeeTransformResponseExtension> getResponseExtensions(
BillingEvent.Recurring recurringBillingEvent, Domain existingDomain, DateTime now) {
BillingRecurrence billingRecurrence, Domain existingDomain, DateTime now) {
FeeTransformResponseExtension.Builder feeResponseBuilder = getDeleteResponseBuilder();
if (feeResponseBuilder == null) {
return ImmutableList.of();
@@ -388,7 +389,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
ImmutableList.Builder<Credit> creditsBuilder = new ImmutableList.Builder<>();
for (GracePeriod gracePeriod : existingDomain.getGracePeriods()) {
if (gracePeriod.hasBillingEvent()) {
Money cost = getGracePeriodCost(recurringBillingEvent, gracePeriod, now);
Money cost = getGracePeriodCost(billingRecurrence, gracePeriod, now);
creditsBuilder.add(Credit.create(
cost.negated().getAmount(), FeeType.CREDIT, gracePeriod.getType().getXmlName()));
feeResponseBuilder.setCurrency(checkNotNull(cost.getCurrencyUnit()));
@@ -402,14 +403,14 @@ public final class DomainDeleteFlow implements TransactionalFlow {
}
private Money getGracePeriodCost(
BillingEvent.Recurring recurringBillingEvent, GracePeriod gracePeriod, DateTime now) {
BillingRecurrence billingRecurrence, GracePeriod gracePeriod, DateTime now) {
if (gracePeriod.getType() == GracePeriodStatus.AUTO_RENEW) {
// If we updated the autorenew billing event, reuse it.
DateTime autoRenewTime =
recurringBillingEvent.getRecurrenceTimeOfYear().getLastInstanceBeforeOrAt(now);
billingRecurrence.getRecurrenceTimeOfYear().getLastInstanceBeforeOrAt(now);
return getDomainRenewCost(targetId, autoRenewTime, 1);
}
return tm().loadByKey(checkNotNull(gracePeriod.getOneTimeBillingEvent())).getCost();
return tm().loadByKey(checkNotNull(gracePeriod.getBillingEvent())).getCost();
}
@Nullable
@@ -429,7 +430,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
/** Domain to be deleted has subordinate hosts. */
static class DomainToDeleteHasHostsException extends AssociationProhibitsOperationException {
public DomainToDeleteHasHostsException() {
DomainToDeleteHasHostsException() {
super("Domain to be deleted has subordinate hosts");
}
}
@@ -74,13 +74,13 @@ import google.registry.flows.EppException.ParameterValueSyntaxErrorException;
import google.registry.flows.EppException.RequiredParameterMissingException;
import google.registry.flows.EppException.StatusProhibitsOperationException;
import google.registry.flows.EppException.UnimplementedOptionException;
import google.registry.flows.domain.DomainPricingLogic.AllocationTokenInvalidForPremiumNameException;
import google.registry.flows.domain.token.AllocationTokenFlowUtils;
import google.registry.flows.exceptions.ResourceHasClientUpdateProhibitedException;
import google.registry.model.EppResource;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingBase.Flag;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.contact.Contact;
import google.registry.model.domain.DesignatedContact;
import google.registry.model.domain.DesignatedContact.Type;
@@ -556,8 +556,8 @@ public class DomainFlowUtils {
* Fills in a builder with the data needed for an autorenew billing event for this domain. This
* does not copy over the id of the current autorenew billing event.
*/
public static BillingEvent.Recurring.Builder newAutorenewBillingEvent(Domain domain) {
return new BillingEvent.Recurring.Builder()
public static BillingRecurrence.Builder newAutorenewBillingEvent(Domain domain) {
return new BillingRecurrence.Builder()
.setReason(Reason.RENEW)
.setFlags(ImmutableSet.of(Flag.AUTO_RENEW))
.setTargetId(domain.getDomainName())
@@ -584,11 +584,11 @@ public class DomainFlowUtils {
* (if opening the message interval). This may cause an autorenew billing event to have an end
* time earlier than its event time (i.e. if it's being ended before it was ever triggered).
*
* <p>Returns the new autorenew recurring billing event.
* <p>Returns the new autorenew recurrence billing event.
*/
public static Recurring updateAutorenewRecurrenceEndTime(
public static BillingRecurrence updateAutorenewRecurrenceEndTime(
Domain domain,
Recurring existingRecurring,
BillingRecurrence existingBillingRecurrence,
DateTime newEndTime,
@Nullable HistoryEntryId historyId) {
Optional<PollMessage.Autorenew> autorenewPollMessage =
@@ -623,9 +623,10 @@ public class DomainFlowUtils {
tm().put(updatedAutorenewPollMessage);
}
Recurring newRecurring = existingRecurring.asBuilder().setRecurrenceEndTime(newEndTime).build();
tm().put(newRecurring);
return newRecurring;
BillingRecurrence newBillingRecurrence =
existingBillingRecurrence.asBuilder().setRecurrenceEndTime(newEndTime).build();
tm().put(newBillingRecurrence);
return newBillingRecurrence;
}
/**
@@ -642,7 +643,7 @@ public class DomainFlowUtils {
DomainPricingLogic pricingLogic,
Optional<AllocationToken> allocationToken,
boolean isAvailable,
@Nullable Recurring recurringBillingEvent)
@Nullable BillingRecurrence billingRecurrence)
throws EppException {
DateTime now = currentDate;
// Use the custom effective date specified in the fee check request, if there is one.
@@ -698,7 +699,7 @@ public class DomainFlowUtils {
fees =
pricingLogic
.getRenewPrice(
tld, domainNameString, now, years, recurringBillingEvent, allocationToken)
tld, domainNameString, now, years, billingRecurrence, allocationToken)
.getFees();
break;
case RESTORE:
@@ -724,9 +725,7 @@ public class DomainFlowUtils {
}
builder.setAvailIfSupported(true);
fees =
pricingLogic
.getTransferPrice(tld, domainNameString, now, recurringBillingEvent)
.getFees();
pricingLogic.getTransferPrice(tld, domainNameString, now, billingRecurrence).getFees();
break;
case UPDATE:
builder.setAvailIfSupported(true);
@@ -1219,8 +1218,15 @@ public class DomainFlowUtils {
for (Optional<AllocationToken> token : tokenList) {
try {
AllocationTokenFlowUtils.validateToken(
InternetDomainName.from(domainName), token.get(), commandName, registrarId, now);
} catch (AssociationProhibitsOperationException | StatusProhibitsOperationException e) {
InternetDomainName.from(domainName),
token.get(),
commandName,
registrarId,
isDomainPremium(domainName, now),
now);
} catch (AssociationProhibitsOperationException
| StatusProhibitsOperationException
| AllocationTokenInvalidForPremiumNameException e) {
// Allocation token was not valid for this registration, continue to check the next token in
// the list
continue;
@@ -16,6 +16,7 @@ package google.registry.flows.domain;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.flows.domain.DomainFlowUtils.zeroInCurrency;
import static google.registry.flows.domain.token.AllocationTokenFlowUtils.validateTokenForPossiblePremiumName;
import static google.registry.pricing.PricingEngineProxy.getPricesForDomainName;
import static google.registry.util.DomainNameUtils.getTldFromDomainName;
import static google.registry.util.PreconditionsUtils.checkArgumentPresent;
@@ -29,7 +30,7 @@ import google.registry.flows.custom.DomainPricingCustomLogic.RenewPriceParameter
import google.registry.flows.custom.DomainPricingCustomLogic.RestorePriceParameters;
import google.registry.flows.custom.DomainPricingCustomLogic.TransferPriceParameters;
import google.registry.flows.custom.DomainPricingCustomLogic.UpdatePriceParameters;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.fee.BaseFee;
import google.registry.model.domain.fee.BaseFee.FeeType;
import google.registry.model.domain.fee.Fee;
@@ -113,20 +114,20 @@ public final class DomainPricingLogic {
String domainName,
DateTime dateTime,
int years,
@Nullable Recurring recurringBillingEvent,
@Nullable BillingRecurrence billingRecurrence,
Optional<AllocationToken> allocationToken)
throws AllocationTokenInvalidForPremiumNameException {
checkArgument(years > 0, "Number of years must be positive");
Money renewCost;
DomainPrices domainPrices = getPricesForDomainName(domainName, dateTime);
boolean isRenewCostPremiumPrice;
// recurring billing event is null if the domain is still available. Billing events are created
// recurrence is null if the domain is still available. Billing events are created
// in the process of domain creation.
if (recurringBillingEvent == null) {
if (billingRecurrence == null) {
renewCost = getDomainRenewCostWithDiscount(domainPrices, years, allocationToken);
isRenewCostPremiumPrice = domainPrices.isPremium();
} else {
switch (recurringBillingEvent.getRenewalPriceBehavior()) {
switch (billingRecurrence.getRenewalPriceBehavior()) {
case DEFAULT:
renewCost = getDomainRenewCostWithDiscount(domainPrices, years, allocationToken);
isRenewCostPremiumPrice = domainPrices.isPremium();
@@ -135,11 +136,11 @@ public final class DomainPricingLogic {
// as the creation price, which is stored in the billing event as the renewal price
case SPECIFIED:
checkArgumentPresent(
recurringBillingEvent.getRenewalPrice(),
billingRecurrence.getRenewalPrice(),
"Unexpected behavior: renewal price cannot be null when renewal behavior is"
+ " SPECIFIED");
// Don't apply allocation token to renewal price when SPECIFIED
renewCost = recurringBillingEvent.getRenewalPrice().get().multipliedBy(years);
renewCost = billingRecurrence.getRenewalPrice().get().multipliedBy(years);
isRenewCostPremiumPrice = false;
break;
// if the renewal price behavior is nonpremium, it means that the domain should be renewed
@@ -157,7 +158,7 @@ public final class DomainPricingLogic {
throw new IllegalArgumentException(
String.format(
"Unknown RenewalPriceBehavior enum value: %s",
recurringBillingEvent.getRenewalPriceBehavior()));
billingRecurrence.getRenewalPriceBehavior()));
}
}
return customLogic.customizeRenewPrice(
@@ -200,10 +201,10 @@ public final class DomainPricingLogic {
/** Returns a new transfer price for the pricer. */
FeesAndCredits getTransferPrice(
Tld tld, String domainName, DateTime dateTime, @Nullable Recurring recurringBillingEvent)
Tld tld, String domainName, DateTime dateTime, @Nullable BillingRecurrence billingRecurrence)
throws EppException {
FeesAndCredits renewPrice =
getRenewPrice(tld, domainName, dateTime, 1, recurringBillingEvent, Optional.empty());
getRenewPrice(tld, domainName, dateTime, 1, billingRecurrence, Optional.empty());
return customLogic.customizeTransferPrice(
TransferPriceParameters.newBuilder()
.setFeesAndCredits(
@@ -257,12 +258,7 @@ public final class DomainPricingLogic {
private Money getDomainCostWithDiscount(
boolean isPremium, int years, Optional<AllocationToken> allocationToken, Money oneYearCost)
throws AllocationTokenInvalidForPremiumNameException {
if (allocationToken.isPresent()
&& allocationToken.get().getDiscountFraction() != 0.0
&& isPremium
&& !allocationToken.get().shouldDiscountPremiums()) {
throw new AllocationTokenInvalidForPremiumNameException();
}
validateTokenForPossiblePremiumName(allocationToken, isPremium);
Money totalDomainFlowCost = oneYearCost.multipliedBy(years);
// Apply the allocation token discount, if applicable.
@@ -281,8 +277,8 @@ public final class DomainPricingLogic {
/** An allocation token was provided that is invalid for premium domains. */
public static class AllocationTokenInvalidForPremiumNameException
extends CommandUseErrorException {
AllocationTokenInvalidForPremiumNameException() {
super("A nonzero discount code cannot be applied to premium domains");
public AllocationTokenInvalidForPremiumNameException() {
super("Token not valid for premium name");
}
}
}
@@ -54,10 +54,9 @@ import google.registry.flows.custom.DomainRenewFlowCustomLogic.BeforeSaveParamet
import google.registry.flows.custom.EntityChanges;
import google.registry.flows.domain.token.AllocationTokenFlowUtils;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Renew;
import google.registry.model.domain.DomainHistory;
@@ -203,7 +202,7 @@ public final class DomainRenewFlow implements TransactionalFlow {
validateRegistrationPeriod(now, newExpirationTime);
Optional<FeeRenewCommandExtension> feeRenew =
eppInput.getSingleExtension(FeeRenewCommandExtension.class);
Recurring existingRecurringBillingEvent =
BillingRecurrence existingBillingRecurrence =
tm().loadByKey(existingDomain.getAutorenewBillingEvent());
FeesAndCredits feesAndCredits =
pricingLogic.getRenewPrice(
@@ -211,7 +210,7 @@ public final class DomainRenewFlow implements TransactionalFlow {
targetId,
now,
years,
existingRecurringBillingEvent,
existingBillingRecurrence,
allocationToken);
validateFeeChallenge(feeRenew, feesAndCredits, defaultTokenUsed);
flowCustomLogic.afterValidation(
@@ -223,15 +222,15 @@ public final class DomainRenewFlow implements TransactionalFlow {
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder.setRevisionId(domainHistoryId.getRevisionId());
// Bill for this explicit renew itself.
BillingEvent.OneTime explicitRenewEvent =
BillingEvent explicitRenewEvent =
createRenewBillingEvent(
tldStr, feesAndCredits.getTotalCost(), years, domainHistoryId, allocationToken, now);
// Create a new autorenew billing event and poll message starting at the new expiration time.
BillingEvent.Recurring newAutorenewEvent =
BillingRecurrence newAutorenewEvent =
newAutorenewBillingEvent(existingDomain)
.setEventTime(newExpirationTime)
.setRenewalPrice(existingRecurringBillingEvent.getRenewalPrice().orElse(null))
.setRenewalPriceBehavior(existingRecurringBillingEvent.getRenewalPriceBehavior())
.setRenewalPrice(existingBillingRecurrence.getRenewalPrice().orElse(null))
.setRenewalPriceBehavior(existingBillingRecurrence.getRenewalPriceBehavior())
.setDomainHistoryId(domainHistoryId)
.build();
PollMessage.Autorenew newAutorenewPollMessage =
@@ -240,8 +239,8 @@ public final class DomainRenewFlow implements TransactionalFlow {
.setDomainHistoryId(domainHistoryId)
.build();
// End the old autorenew billing event and poll message now. This may delete the poll message.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
updateAutorenewRecurrenceEndTime(existingDomain, existingRecurring, now, domainHistoryId);
updateAutorenewRecurrenceEndTime(
existingDomain, existingBillingRecurrence, now, domainHistoryId);
Domain newDomain =
existingDomain
.asBuilder()
@@ -338,14 +337,14 @@ public final class DomainRenewFlow implements TransactionalFlow {
}
}
private OneTime createRenewBillingEvent(
private BillingEvent createRenewBillingEvent(
String tld,
Money renewCost,
int years,
HistoryEntryId domainHistoryId,
Optional<AllocationToken> allocationToken,
DateTime now) {
return new BillingEvent.OneTime.Builder()
return new BillingEvent.Builder()
.setReason(Reason.RENEW)
.setTargetId(targetId)
.setRegistrarId(registrarId)
@@ -14,6 +14,7 @@
package google.registry.flows.domain;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -34,7 +35,6 @@ import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.net.InternetDomainName;
import google.registry.dns.DnsUtils;
import google.registry.flows.EppException;
import google.registry.flows.EppException.CommandUseErrorException;
import google.registry.flows.EppException.StatusProhibitsOperationException;
@@ -45,9 +45,9 @@ import google.registry.flows.FlowModule.TargetId;
import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Update;
import google.registry.model.domain.DomainHistory;
@@ -122,7 +122,6 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
@Inject @TargetId String targetId;
@Inject @Superuser boolean isSuperuser;
@Inject DomainHistory.Builder historyBuilder;
@Inject DnsUtils dnsUtils;
@Inject EppResponse.Builder responseBuilder;
@Inject DomainPricingLogic pricingLogic;
@Inject DomainRestoreRequestFlow() {}
@@ -161,7 +160,7 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
entitiesToSave.add(
createRestoreBillingEvent(domainHistoryId, feesAndCredits.getRestoreCost(), now));
BillingEvent.Recurring autorenewEvent =
BillingRecurrence autorenewEvent =
newAutorenewBillingEvent(existingDomain)
.setEventTime(newExpirationTime)
.setRecurrenceEndTime(END_OF_TIME)
@@ -185,7 +184,7 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
entitiesToSave.add(newDomain, domainHistory, autorenewEvent, autorenewPollMessage);
tm().putAll(entitiesToSave.build());
tm().delete(existingDomain.getDeletePollMessage());
dnsUtils.requestDomainDnsRefresh(existingDomain.getDomainName());
requestDomainDnsRefresh(existingDomain.getDomainName());
return responseBuilder
.setExtensions(createResponseExtensions(feesAndCredits, feeUpdate, isExpired))
.build();
@@ -231,7 +230,7 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
private static Domain performRestore(
Domain existingDomain,
DateTime newExpirationTime,
BillingEvent.Recurring autorenewEvent,
BillingRecurrence autorenewEvent,
PollMessage.Autorenew autorenewPollMessage,
DateTime now,
String registrarId) {
@@ -252,19 +251,19 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
.build();
}
private OneTime createRenewBillingEvent(
private BillingEvent createRenewBillingEvent(
HistoryEntryId domainHistoryId, Money renewCost, DateTime now) {
return prepareBillingEvent(domainHistoryId, renewCost, now).setReason(Reason.RENEW).build();
}
private BillingEvent.OneTime createRestoreBillingEvent(
private BillingEvent createRestoreBillingEvent(
HistoryEntryId domainHistoryId, Money restoreCost, DateTime now) {
return prepareBillingEvent(domainHistoryId, restoreCost, now).setReason(Reason.RESTORE).build();
}
private OneTime.Builder prepareBillingEvent(
private BillingEvent.Builder prepareBillingEvent(
HistoryEntryId domainHistoryId, Money cost, DateTime now) {
return new BillingEvent.OneTime.Builder()
return new BillingEvent.Builder()
.setTargetId(targetId)
.setRegistrarId(registrarId)
.setEventTime(now)
@@ -305,14 +304,14 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
/** Restore command cannot have other changes specified. */
static class RestoreCommandIncludesChangesException extends CommandUseErrorException {
public RestoreCommandIncludesChangesException() {
RestoreCommandIncludesChangesException() {
super("Restore command cannot have other changes specified");
}
}
/** Domain is not eligible for restore. */
static class DomainNotEligibleForRestoreException extends StatusProhibitsOperationException {
public DomainNotEligibleForRestoreException() {
DomainNotEligibleForRestoreException() {
super("Domain is not eligible for restore");
}
}
@@ -45,11 +45,12 @@ import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.flows.domain.token.AllocationTokenFlowUtils;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingBase.Flag;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.billing.BillingCancellation;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.GracePeriod;
@@ -149,16 +150,18 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
String gainingRegistrarId = transferData.getGainingRegistrarId();
// Create a transfer billing event for 1 year, unless the superuser extension was used to set
// the transfer period to zero. There is not a transfer cost if the transfer period is zero.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
BillingRecurrence existingBillingRecurrence =
tm().loadByKey(existingDomain.getAutorenewBillingEvent());
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder.setRevisionId(domainHistoryId.getRevisionId());
boolean hasPackageToken = existingDomain.getCurrentPackageToken().isPresent();
Money renewalPrice = hasPackageToken ? null : existingRecurring.getRenewalPrice().orElse(null);
Optional<BillingEvent.OneTime> billingEvent =
Money renewalPrice =
hasPackageToken ? null : existingBillingRecurrence.getRenewalPrice().orElse(null);
Optional<BillingEvent> billingEvent =
transferData.getTransferPeriod().getValue() == 0
? Optional.empty()
: Optional.of(
new BillingEvent.OneTime.Builder()
new BillingEvent.Builder()
.setReason(Reason.TRANSFER)
.setTargetId(targetId)
.setRegistrarId(gainingRegistrarId)
@@ -170,9 +173,9 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
targetId,
transferData.getTransferRequestTime(),
// When removing a domain from a package it should return to the
// default recurring billing behavior so the existing recurring
// default recurrence billing behavior so the existing recurrence
// billing event should not be passed in.
hasPackageToken ? null : existingRecurring)
hasPackageToken ? null : existingBillingRecurrence)
.getRenewCost())
.setEventTime(now)
.setBillingTime(now.plus(Tld.get(tldStr).getTransferGracePeriodLength()))
@@ -192,18 +195,18 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
// still needs to be charged for the auto-renew.
if (billingEvent.isPresent()) {
entitiesToSave.add(
BillingEvent.Cancellation.forGracePeriod(
autorenewGrace, now, domainHistoryId, targetId));
BillingCancellation.forGracePeriod(autorenewGrace, now, domainHistoryId, targetId));
}
}
// Close the old autorenew event and poll message at the transfer time (aka now). This may end
// up deleting the poll message.
updateAutorenewRecurrenceEndTime(existingDomain, existingRecurring, now, domainHistoryId);
updateAutorenewRecurrenceEndTime(
existingDomain, existingBillingRecurrence, now, domainHistoryId);
DateTime newExpirationTime =
computeExDateForApprovalTime(existingDomain, now, transferData.getTransferPeriod());
// Create a new autorenew event starting at the expiration time.
BillingEvent.Recurring autorenewEvent =
new BillingEvent.Recurring.Builder()
BillingRecurrence autorenewEvent =
new BillingRecurrence.Builder()
.setReason(Reason.RENEW)
.setFlags(ImmutableSet.of(Flag.AUTO_RENEW))
.setTargetId(targetId)
@@ -212,7 +215,7 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
.setRenewalPriceBehavior(
hasPackageToken
? RenewalPriceBehavior.DEFAULT
: existingRecurring.getRenewalPriceBehavior())
: existingBillingRecurrence.getRenewalPriceBehavior())
.setRenewalPrice(renewalPrice)
.setRecurrenceEndTime(END_OF_TIME)
.setDomainHistoryId(domainHistoryId)
@@ -248,12 +251,10 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
.setGracePeriods(
billingEvent
.map(
oneTime ->
event ->
ImmutableSet.of(
GracePeriod.forBillingEvent(
GracePeriodStatus.TRANSFER,
existingDomain.getRepoId(),
oneTime)))
GracePeriodStatus.TRANSFER, existingDomain.getRepoId(), event)))
.orElseGet(ImmutableSet::of))
.setLastEppUpdateTime(now)
.setLastEppUpdateRegistrarId(registrarId)
@@ -39,7 +39,7 @@ import google.registry.flows.FlowModule.Superuser;
import google.registry.flows.FlowModule.TargetId;
import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.metadata.MetadataExtension;
@@ -117,9 +117,10 @@ public final class DomainTransferCancelFlow implements TransactionalFlow {
targetId, newDomain.getTransferData(), null, domainHistoryId));
// Reopen the autorenew event and poll message that we closed for the implicit transfer. This
// may recreate the autorenew poll message if it was deleted when the transfer request was made.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
BillingRecurrence existingBillingRecurrence =
tm().loadByKey(existingDomain.getAutorenewBillingEvent());
updateAutorenewRecurrenceEndTime(
existingDomain, existingRecurring, END_OF_TIME, domainHistory.getHistoryEntryId());
existingDomain, existingBillingRecurrence, END_OF_TIME, domainHistory.getHistoryEntryId());
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
tm().delete(existingDomain.getTransferData().getServerApproveEntities());
@@ -41,7 +41,7 @@ import google.registry.flows.FlowModule.Superuser;
import google.registry.flows.FlowModule.TargetId;
import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.metadata.MetadataExtension;
@@ -116,9 +116,10 @@ public final class DomainTransferRejectFlow implements TransactionalFlow {
targetId, newDomain.getTransferData(), null, now, domainHistoryId));
// Reopen the autorenew event and poll message that we closed for the implicit transfer. This
// may end up recreating the poll message if it was deleted upon the transfer request.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
BillingRecurrence existingBillingRecurrence =
tm().loadByKey(existingDomain.getAutorenewBillingEvent());
updateAutorenewRecurrenceEndTime(
existingDomain, existingRecurring, END_OF_TIME, domainHistory.getHistoryEntryId());
existingDomain, existingBillingRecurrence, END_OF_TIME, domainHistory.getHistoryEntryId());
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
tm().delete(existingDomain.getTransferData().getServerApproveEntities());
@@ -52,7 +52,7 @@ import google.registry.flows.exceptions.InvalidTransferPeriodValueException;
import google.registry.flows.exceptions.ObjectAlreadySponsoredException;
import google.registry.flows.exceptions.TransferPeriodMustBeOneYearException;
import google.registry.flows.exceptions.TransferPeriodZeroAndFeeTransferExtensionException;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Transfer;
import google.registry.model.domain.DomainHistory;
@@ -195,13 +195,14 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
throw new TransferPeriodZeroAndFeeTransferExtensionException();
}
// If the period is zero, then there is no fee for the transfer.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
BillingRecurrence existingBillingRecurrence =
tm().loadByKey(existingDomain.getAutorenewBillingEvent());
Optional<FeesAndCredits> feesAndCredits;
if (period.getValue() == 0) {
feesAndCredits = Optional.empty();
} else if (!existingDomain.getCurrentPackageToken().isPresent()) {
feesAndCredits =
Optional.of(pricingLogic.getTransferPrice(tld, targetId, now, existingRecurring));
Optional.of(pricingLogic.getTransferPrice(tld, targetId, now, existingBillingRecurrence));
} else {
// If existing domain is in a package, calculate the transfer price with default renewal price
// behavior
@@ -243,7 +244,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
serverApproveNewExpirationTime,
domainHistoryId,
existingDomain,
existingRecurring,
existingBillingRecurrence,
trid,
gainingClientId,
feesAndCredits.map(FeesAndCredits::getTotalCost),
@@ -274,7 +275,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
// cloneProjectedAtTime() will replace these old autorenew entities with the server approve ones
// that we've created in this flow and stored in pendingTransferData.
updateAutorenewRecurrenceEndTime(
existingDomain, existingRecurring, automaticTransferTime, domainHistoryId);
existingDomain, existingBillingRecurrence, automaticTransferTime, domainHistoryId);
Domain newDomain =
existingDomain
.asBuilder()
@@ -20,11 +20,12 @@ import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import google.registry.model.billing.BillingBase.Flag;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.billing.BillingCancellation;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.Period;
@@ -66,8 +67,8 @@ public final class DomainTransferUtils {
// Unless superuser sets period to 0, add a transfer billing event.
transferDataBuilder.setServerApproveBillingEvent(
serverApproveEntities.stream()
.filter(BillingEvent.OneTime.class::isInstance)
.map(BillingEvent.OneTime.class::cast)
.filter(BillingEvent.class::isInstance)
.map(BillingEvent.class::cast)
.collect(onlyElement())
.createVKey());
}
@@ -75,8 +76,8 @@ public final class DomainTransferUtils {
.setTransferStatus(TransferStatus.PENDING)
.setServerApproveAutorenewEvent(
serverApproveEntities.stream()
.filter(BillingEvent.Recurring.class::isInstance)
.map(BillingEvent.Recurring.class::cast)
.filter(BillingRecurrence.class::isInstance)
.map(BillingRecurrence.class::cast)
.collect(onlyElement())
.createVKey())
.setServerApproveAutorenewPollMessage(
@@ -110,7 +111,7 @@ public final class DomainTransferUtils {
DateTime serverApproveNewExpirationTime,
HistoryEntryId domainHistoryId,
Domain existingDomain,
Recurring existingRecurring,
BillingRecurrence existingBillingRecurrence,
Trid trid,
String gainingRegistrarId,
Optional<Money> transferCost,
@@ -146,12 +147,12 @@ public final class DomainTransferUtils {
.add(
createGainingClientAutorenewEvent(
existingDomain.getCurrentPackageToken().isPresent()
? existingRecurring
? existingBillingRecurrence
.asBuilder()
.setRenewalPriceBehavior(RenewalPriceBehavior.DEFAULT)
.setRenewalPrice(null)
.build()
: existingRecurring,
: existingBillingRecurrence,
serverApproveNewExpirationTime,
domainHistoryId,
targetId,
@@ -246,21 +247,21 @@ public final class DomainTransferUtils {
.build();
}
private static BillingEvent.Recurring createGainingClientAutorenewEvent(
Recurring existingRecurring,
private static BillingRecurrence createGainingClientAutorenewEvent(
BillingRecurrence existingBillingRecurrence,
DateTime serverApproveNewExpirationTime,
HistoryEntryId domainHistoryId,
String targetId,
String gainingRegistrarId) {
return new BillingEvent.Recurring.Builder()
return new BillingRecurrence.Builder()
.setReason(Reason.RENEW)
.setFlags(ImmutableSet.of(Flag.AUTO_RENEW))
.setTargetId(targetId)
.setRegistrarId(gainingRegistrarId)
.setEventTime(serverApproveNewExpirationTime)
.setRecurrenceEndTime(END_OF_TIME)
.setRenewalPriceBehavior(existingRecurring.getRenewalPriceBehavior())
.setRenewalPrice(existingRecurring.getRenewalPrice().orElse(null))
.setRenewalPriceBehavior(existingBillingRecurrence.getRenewalPriceBehavior())
.setRenewalPrice(existingBillingRecurrence.getRenewalPrice().orElse(null))
.setDomainHistoryId(domainHistoryId)
.build();
}
@@ -281,7 +282,7 @@ public final class DomainTransferUtils {
* <p>For details on the policy justification, see b/19430703#comment17 and <a
* href="https://www.icann.org/news/advisory-2002-06-06-en">this ICANN advisory</a>.
*/
private static Optional<BillingEvent.Cancellation> createOptionalAutorenewCancellation(
private static Optional<BillingCancellation> createOptionalAutorenewCancellation(
DateTime automaticTransferTime,
DateTime now,
HistoryEntryId domainHistoryId,
@@ -294,8 +295,7 @@ public final class DomainTransferUtils {
domainAtTransferTime.getGracePeriodsOfType(GracePeriodStatus.AUTO_RENEW), null);
if (autorenewGracePeriod != null && transferCost.isPresent()) {
return Optional.of(
BillingEvent.Cancellation.forGracePeriod(
autorenewGracePeriod, now, domainHistoryId, targetId)
BillingCancellation.forGracePeriod(autorenewGracePeriod, now, domainHistoryId, targetId)
.asBuilder()
.setEventTime(automaticTransferTime)
.build());
@@ -303,14 +303,14 @@ public final class DomainTransferUtils {
return Optional.empty();
}
private static BillingEvent.OneTime createTransferBillingEvent(
private static BillingEvent createTransferBillingEvent(
DateTime automaticTransferTime,
HistoryEntryId domainHistoryId,
String targetId,
String gainingRegistrarId,
Tld registry,
Money transferCost) {
return new BillingEvent.OneTime.Builder()
return new BillingEvent.Builder()
.setReason(Reason.TRANSFER)
.setTargetId(targetId)
.setRegistrarId(gainingRegistrarId)
@@ -19,6 +19,7 @@ import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.collect.ImmutableSortedSet.toImmutableSortedSet;
import static com.google.common.collect.Sets.symmetricDifference;
import static com.google.common.collect.Sets.union;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.flows.FlowUtils.persistEntityChanges;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.checkSameValuesNotAddedAndRemoved;
@@ -49,7 +50,6 @@ import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Ordering;
import com.google.common.collect.Sets;
import com.google.common.net.InternetDomainName;
import google.registry.dns.DnsUtils;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -64,8 +64,8 @@ import google.registry.flows.custom.EntityChanges;
import google.registry.flows.domain.DomainFlowUtils.MissingRegistrantException;
import google.registry.flows.domain.DomainFlowUtils.NameserversNotSpecifiedForTldWithNameserverAllowListException;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.domain.DesignatedContact;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Update;
@@ -156,7 +156,6 @@ public final class DomainUpdateFlow implements TransactionalFlow {
@Inject @Superuser boolean isSuperuser;
@Inject Trid trid;
@Inject DomainHistory.Builder historyBuilder;
@Inject DnsUtils dnsUtils;
@Inject EppResponse.Builder responseBuilder;
@Inject DomainUpdateFlowCustomLogic flowCustomLogic;
@Inject DomainPricingLogic pricingLogic;
@@ -183,11 +182,11 @@ public final class DomainUpdateFlow implements TransactionalFlow {
historyBuilder.setType(DOMAIN_UPDATE).setDomain(newDomain).build();
validateNewState(newDomain);
if (requiresDnsUpdate(existingDomain, newDomain)) {
dnsUtils.requestDomainDnsRefresh(targetId);
requestDomainDnsRefresh(targetId);
}
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
entitiesToSave.add(newDomain, domainHistory);
Optional<BillingEvent.OneTime> statusUpdateBillingEvent =
Optional<BillingEvent> statusUpdateBillingEvent =
createBillingEventForStatusUpdates(existingDomain, newDomain, domainHistory, now);
statusUpdateBillingEvent.ifPresent(entitiesToSave::add);
Optional<PollMessage.OneTime> serverStatusUpdatePollMessage =
@@ -207,7 +206,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
}
/** Determines if any of the changes to new domain should trigger DNS update. */
private boolean requiresDnsUpdate(Domain existingDomain, Domain newDomain) {
private static boolean requiresDnsUpdate(Domain existingDomain, Domain newDomain) {
return existingDomain.shouldPublishToDns() != newDomain.shouldPublishToDns()
|| !Objects.equals(newDomain.getDsData(), existingDomain.getDsData())
|| !Objects.equals(newDomain.getNsHosts(), existingDomain.getNsHosts());
@@ -256,7 +255,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
// We have to verify no duplicate contacts _before_ constructing the domain because it is
// illegal to construct a domain with duplicate contacts.
Sets.SetView<DesignatedContact> newContacts =
Sets.union(Sets.difference(domain.getContacts(), remove.getContacts()), add.getContacts());
union(Sets.difference(domain.getContacts(), remove.getContacts()), add.getContacts());
validateNoDuplicateContacts(newContacts);
Domain.Builder domainBuilder =
@@ -301,7 +300,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
return domainBuilder.build();
}
private void validateRegistrantIsntBeingRemoved(Change change) throws EppException {
private static void validateRegistrantIsntBeingRemoved(Change change) throws EppException {
if (change.getRegistrantContactId() != null && change.getRegistrantContactId().isEmpty()) {
throw new MissingRegistrantException();
}
@@ -314,7 +313,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
* compliant with the additions or amendments, otherwise existing data can become invalid and
* cause Domain update failure.
*/
private void validateNewState(Domain newDomain) throws EppException {
private static void validateNewState(Domain newDomain) throws EppException {
validateRequiredContactsPresent(newDomain.getRegistrant(), newDomain.getContacts());
validateDsData(newDomain.getDsData());
validateNameserversCountForTld(
@@ -324,7 +323,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
}
/** Some status updates cost money. Bill only once no matter how many of them are changed. */
private Optional<BillingEvent.OneTime> createBillingEventForStatusUpdates(
private Optional<BillingEvent> createBillingEventForStatusUpdates(
Domain existingDomain, Domain newDomain, DomainHistory historyEntry, DateTime now) {
Optional<MetadataExtension> metadataExtension =
eppInput.getSingleExtension(MetadataExtension.class);
@@ -334,7 +333,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
if (statusValue.isChargedStatus()) {
// Only charge once.
return Optional.of(
new BillingEvent.OneTime.Builder()
new BillingEvent.Builder()
.setReason(Reason.SERVER_STATUS)
.setTargetId(targetId)
.setRegistrarId(registrarId)
@@ -368,17 +367,17 @@ public final class DomainUpdateFlow implements TransactionalFlow {
.collect(toImmutableSortedSet(Ordering.natural()));
String msg;
if (addedServerStatuses.size() > 0 && removedServerStatuses.size() > 0) {
if (!addedServerStatuses.isEmpty() && !removedServerStatuses.isEmpty()) {
msg =
String.format(
"The registry administrator has added the status(es) %s and removed the status(es)"
+ " %s.",
addedServerStatuses, removedServerStatuses);
} else if (addedServerStatuses.size() > 0) {
} else if (!addedServerStatuses.isEmpty()) {
msg =
String.format(
"The registry administrator has added the status(es) %s.", addedServerStatuses);
} else if (removedServerStatuses.size() > 0) {
} else if (!removedServerStatuses.isEmpty()) {
msg =
String.format(
"The registry administrator has removed the status(es) %s.", removedServerStatuses);
@@ -16,6 +16,7 @@ package google.registry.flows.domain.token;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.pricing.PricingEngineProxy.isDomainPremium;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableList;
@@ -26,8 +27,9 @@ import google.registry.flows.EppException;
import google.registry.flows.EppException.AssociationProhibitsOperationException;
import google.registry.flows.EppException.AuthorizationErrorException;
import google.registry.flows.EppException.StatusProhibitsOperationException;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.flows.domain.DomainPricingLogic.AllocationTokenInvalidForPremiumNameException;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand;
import google.registry.model.domain.fee.FeeQueryCommandExtensionItem.CommandName;
@@ -79,7 +81,13 @@ public class AllocationTokenFlowUtils {
ImmutableMap.Builder<InternetDomainName, String> resultsBuilder = new ImmutableMap.Builder<>();
for (InternetDomainName domainName : domainNames) {
try {
validateToken(domainName, tokenEntity, CommandName.CREATE, registrarId, now);
validateToken(
domainName,
tokenEntity,
CommandName.CREATE,
registrarId,
isDomainPremium(domainName.toString(), now),
now);
validDomainNames.add(domainName);
} catch (EppException e) {
resultsBuilder.put(domainName, e.getMessage());
@@ -105,7 +113,7 @@ public class AllocationTokenFlowUtils {
/**
* Validates a given token. The token could be invalid if it has allowed client IDs or TLDs that
* do not include this client ID / TLD, or if the token has a promotion that is not currently
* running.
* running, or the token is not valid for a premium name when necessary.
*
* @throws EppException if the token is invalid in any way
*/
@@ -114,33 +122,48 @@ public class AllocationTokenFlowUtils {
AllocationToken token,
CommandName commandName,
String registrarId,
boolean isPremium,
DateTime now)
throws EppException {
// Only tokens with default behavior require validation
if (TokenBehavior.DEFAULT.equals(token.getTokenBehavior())) {
if (!token.getAllowedEppActions().isEmpty()
&& !token.getAllowedEppActions().contains(commandName)) {
throw new AllocationTokenNotValidForCommandException();
}
if (!token.getAllowedRegistrarIds().isEmpty()
&& !token.getAllowedRegistrarIds().contains(registrarId)) {
throw new AllocationTokenNotValidForRegistrarException();
}
if (!token.getAllowedTlds().isEmpty()
&& !token.getAllowedTlds().contains(domainName.parent().toString())) {
throw new AllocationTokenNotValidForTldException();
}
if (token.getDomainName().isPresent()
&& !token.getDomainName().get().equals(domainName.toString())) {
throw new AllocationTokenNotValidForDomainException();
}
// Tokens without status transitions will just have a single-entry NOT_STARTED map, so only
// check the status transitions map if it's non-trivial.
if (token.getTokenStatusTransitions().size() > 1
&& !TokenStatus.VALID.equals(token.getTokenStatusTransitions().getValueAtTime(now))) {
throw new AllocationTokenNotInPromotionException();
}
if (!TokenBehavior.DEFAULT.equals(token.getTokenBehavior())) {
return;
}
validateTokenForPossiblePremiumName(Optional.of(token), isPremium);
if (!token.getAllowedEppActions().isEmpty()
&& !token.getAllowedEppActions().contains(commandName)) {
throw new AllocationTokenNotValidForCommandException();
}
if (!token.getAllowedRegistrarIds().isEmpty()
&& !token.getAllowedRegistrarIds().contains(registrarId)) {
throw new AllocationTokenNotValidForRegistrarException();
}
if (!token.getAllowedTlds().isEmpty()
&& !token.getAllowedTlds().contains(domainName.parent().toString())) {
throw new AllocationTokenNotValidForTldException();
}
if (token.getDomainName().isPresent()
&& !token.getDomainName().get().equals(domainName.toString())) {
throw new AllocationTokenNotValidForDomainException();
}
// Tokens without status transitions will just have a single-entry NOT_STARTED map, so only
// check the status transitions map if it's non-trivial.
if (token.getTokenStatusTransitions().size() > 1
&& !TokenStatus.VALID.equals(token.getTokenStatusTransitions().getValueAtTime(now))) {
throw new AllocationTokenNotInPromotionException();
}
}
/** Validates that the given token is valid for a premium name if the name is premium. */
public static void validateTokenForPossiblePremiumName(
Optional<AllocationToken> token, boolean isPremium)
throws AllocationTokenInvalidForPremiumNameException {
if (token.isPresent()
&& token.get().getDiscountFraction() != 0.0
&& isPremium
&& !token.get().shouldDiscountPremiums()) {
throw new AllocationTokenInvalidForPremiumNameException();
}
}
@@ -187,6 +210,7 @@ public class AllocationTokenFlowUtils {
tokenEntity,
CommandName.CREATE,
registrarId,
isDomainPremium(command.getDomainName(), now),
now);
return Optional.of(tokenCustomLogic.validateToken(command, tokenEntity, tld, registrarId, now));
}
@@ -209,6 +233,7 @@ public class AllocationTokenFlowUtils {
tokenEntity,
commandName,
registrarId,
isDomainPremium(existingDomain.getDomainName(), now),
now);
return Optional.of(
tokenCustomLogic.validateToken(existingDomain, tokenEntity, tld, registrarId, now));
@@ -236,16 +261,16 @@ public class AllocationTokenFlowUtils {
return domain;
}
Recurring newRecurringBillingEvent =
BillingRecurrence newBillingRecurrence =
tm().loadByKey(domain.getAutorenewBillingEvent())
.asBuilder()
.setRenewalPriceBehavior(RenewalPriceBehavior.DEFAULT)
.setRenewalPrice(null)
.build();
// the Recurring billing event is reloaded later in the renew flow, so we synchronize changed
// RecurringBillingEvent with storage manually
tm().put(newRecurringBillingEvent);
// the Recurrence is reloaded later in the renew flow, so we synchronize changed
// Recurrences with storage manually
tm().put(newBillingRecurrence);
tm().getEntityManager().flush();
tm().getEntityManager().clear();
@@ -253,7 +278,7 @@ public class AllocationTokenFlowUtils {
return domain
.asBuilder()
.setCurrentPackageToken(null)
.setAutorenewBillingEvent(newRecurringBillingEvent.createVKey())
.setAutorenewBillingEvent(newBillingRecurrence.createVKey())
.build();
}
@@ -14,6 +14,7 @@
package google.registry.flows.host;
import static google.registry.dns.DnsUtils.requestHostDnsRefresh;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.verifyResourceDoesNotExist;
import static google.registry.flows.host.HostFlowUtils.lookupSuperordinateDomain;
@@ -28,7 +29,6 @@ import static google.registry.util.CollectionUtils.isNullOrEmpty;
import com.google.common.collect.ImmutableSet;
import google.registry.config.RegistryConfig.Config;
import google.registry.dns.DnsUtils;
import google.registry.flows.EppException;
import google.registry.flows.EppException.ParameterValueRangeErrorException;
import google.registry.flows.EppException.RequiredParameterMissingException;
@@ -85,7 +85,6 @@ public final class HostCreateFlow implements TransactionalFlow {
@Inject @RegistrarId String registrarId;
@Inject @TargetId String targetId;
@Inject HostHistory.Builder historyBuilder;
@Inject DnsUtils dnsUtils;
@Inject EppResponse.Builder responseBuilder;
@Inject
@@ -138,7 +137,7 @@ public final class HostCreateFlow implements TransactionalFlow {
.build());
// Only update DNS if this is a subordinate host. External hosts have no glue to write, so
// they are only written as NS records from the referencing domain.
dnsUtils.requestHostDnsRefresh(targetId);
requestHostDnsRefresh(targetId);
}
tm().insertAll(entitiesToSave);
return responseBuilder.setResData(HostCreateData.create(targetId, now)).build();
@@ -14,6 +14,7 @@
package google.registry.flows.host;
import static google.registry.dns.DnsUtils.requestHostDnsRefresh;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.checkLinkedDomains;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -24,7 +25,6 @@ import static google.registry.model.eppoutput.Result.Code.SUCCESS;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import google.registry.dns.DnsUtils;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -71,7 +71,6 @@ public final class HostDeleteFlow implements TransactionalFlow {
StatusValue.PENDING_DELETE,
StatusValue.SERVER_DELETE_PROHIBITED);
@Inject DnsUtils dnsUtils;
@Inject ExtensionManager extensionManager;
@Inject @RegistrarId String registrarId;
@Inject @TargetId String targetId;
@@ -104,7 +103,7 @@ public final class HostDeleteFlow implements TransactionalFlow {
}
Host newHost = existingHost.asBuilder().setStatusValues(null).setDeletionTime(now).build();
if (existingHost.isSubordinate()) {
dnsUtils.requestHostDnsRefresh(existingHost.getHostName());
requestHostDnsRefresh(existingHost.getHostName());
tm().update(
tm().loadByKey(existingHost.getSuperordinateDomain())
.asBuilder()
@@ -16,6 +16,7 @@ package google.registry.flows.host;
import static com.google.common.base.MoreObjects.firstNonNull;
import static com.google.common.collect.Sets.union;
import static google.registry.dns.DnsUtils.requestHostDnsRefresh;
import static google.registry.dns.RefreshDnsOnHostRenameAction.PARAM_HOST_KEY;
import static google.registry.dns.RefreshDnsOnHostRenameAction.QUEUE_HOST_RENAME;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
@@ -37,7 +38,6 @@ import com.google.common.collect.ImmutableMultimap;
import com.google.common.collect.ImmutableSet;
import google.registry.batch.AsyncTaskEnqueuer;
import google.registry.batch.CloudTasksUtils;
import google.registry.dns.DnsUtils;
import google.registry.dns.RefreshDnsOnHostRenameAction;
import google.registry.flows.EppException;
import google.registry.flows.EppException.ObjectAlreadyExistsException;
@@ -124,7 +124,6 @@ public final class HostUpdateFlow implements TransactionalFlow {
@Inject @Superuser boolean isSuperuser;
@Inject HostHistory.Builder historyBuilder;
@Inject AsyncTaskEnqueuer asyncTaskEnqueuer;
@Inject DnsUtils dnsUtils;
@Inject EppResponse.Builder responseBuilder;
@Inject CloudTasksUtils cloudTasksUtils;
@@ -241,7 +240,7 @@ public final class HostUpdateFlow implements TransactionalFlow {
verifyNoDisallowedStatuses(existingHost, DISALLOWED_STATUSES);
}
private void verifyHasIpsIffIsExternal(Update command, Host existingHost, Host newHost)
private static void verifyHasIpsIffIsExternal(Update command, Host existingHost, Host newHost)
throws EppException {
boolean wasSubordinate = existingHost.isSubordinate();
boolean willBeSubordinate = newHost.isSubordinate();
@@ -266,14 +265,14 @@ public final class HostUpdateFlow implements TransactionalFlow {
// Only update DNS for subordinate hosts. External hosts have no glue to write, so they
// are only written as NS records from the referencing domain.
if (existingHost.isSubordinate()) {
dnsUtils.requestHostDnsRefresh(existingHost.getHostName());
requestHostDnsRefresh(existingHost.getHostName());
}
// In case of a rename, there are many updates we need to queue up.
if (((Update) resourceCommand).getInnerChange().getHostName() != null) {
// If the renamed host is also subordinate, then we must enqueue an update to write the new
// glue.
if (newHost.isSubordinate()) {
dnsUtils.requestHostDnsRefresh(newHost.getHostName());
requestHostDnsRefresh(newHost.getHostName());
}
// We must also enqueue updates for all domains that use this host as their nameserver so
// that their NS records can be updated to point at the new name.
@@ -317,14 +316,14 @@ public final class HostUpdateFlow implements TransactionalFlow {
/** Host with specified name already exists. */
static class HostAlreadyExistsException extends ObjectAlreadyExistsException {
public HostAlreadyExistsException(String hostName) {
HostAlreadyExistsException(String hostName) {
super(String.format("Object with given ID (%s) already exists", hostName));
}
}
/** Cannot add IP addresses to an external host. */
static class CannotAddIpToExternalHostException extends ParameterValueRangeErrorException {
public CannotAddIpToExternalHostException() {
CannotAddIpToExternalHostException() {
super("Cannot add IP addresses to external hosts");
}
}
@@ -332,14 +331,14 @@ public final class HostUpdateFlow implements TransactionalFlow {
/** Cannot remove all IP addresses from a subordinate host. */
static class CannotRemoveSubordinateHostLastIpException
extends StatusProhibitsOperationException {
public CannotRemoveSubordinateHostLastIpException() {
CannotRemoveSubordinateHostLastIpException() {
super("Cannot remove all IP addresses from a subordinate host");
}
}
/** Cannot rename an external host. */
static class CannotRenameExternalHostException extends StatusProhibitsOperationException {
public CannotRenameExternalHostException() {
CannotRenameExternalHostException() {
super("Cannot rename an external host");
}
}
@@ -0,0 +1,264 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.billing;
import static com.google.common.base.Preconditions.checkNotNull;
import static google.registry.util.CollectionUtils.forceEmptyToNull;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableCopy;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.google.common.collect.ImmutableSet;
import google.registry.model.Buildable;
import google.registry.model.ImmutableObject;
import google.registry.model.UnsafeSerializable;
import google.registry.model.annotations.IdAllocation;
import google.registry.model.domain.DomainHistory;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
import google.registry.persistence.VKey;
import java.util.Set;
import javax.annotation.Nullable;
import javax.persistence.Column;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.Id;
import javax.persistence.MappedSuperclass;
import org.joda.time.DateTime;
/** A billable event in a domain's lifecycle. */
@MappedSuperclass
public abstract class BillingBase extends ImmutableObject
implements Buildable, TransferServerApproveEntity, UnsafeSerializable {
/** The reason for the bill, which maps 1:1 to skus in go/registry-billing-skus. */
public enum Reason {
CREATE(true),
@Deprecated // DO NOT USE THIS REASON. IT REMAINS BECAUSE OF HISTORICAL DATA. SEE b/31676071.
ERROR(false),
FEE_EARLY_ACCESS(true),
RENEW(true),
RESTORE(true),
SERVER_STATUS(false),
TRANSFER(true);
private final boolean requiresPeriod;
Reason(boolean requiresPeriod) {
this.requiresPeriod = requiresPeriod;
}
/**
* Returns whether billing events with this reason have a period years associated with them.
*
* <p>Note that this is an "if an only if" condition.
*/
public boolean hasPeriodYears() {
return requiresPeriod;
}
}
/** Set of flags that can be applied to billing events. */
public enum Flag {
ALLOCATION,
ANCHOR_TENANT,
AUTO_RENEW,
/** Landrush billing events are historical only and are no longer created. */
LANDRUSH,
/**
* This flag is used on create {@link BillingEvent} billing events for domains that were
* reserved.
*
* <p>This can happen when allocation tokens are used or superusers override a domain
* reservation. These cases can need special handling in billing/invoicing. Anchor tenants will
* never have this flag applied; they will have ANCHOR_TENANT instead.
*/
RESERVED,
SUNRISE,
/**
* This flag will be added to any {@link BillingEvent} events that are created via, e.g., an
* automated process to expand {@link BillingRecurrence} events.
*/
SYNTHETIC
}
/**
* Sets of renewal price behaviors that can be applied to billing recurrences.
*
* <p>When a client renews a domain, they could be charged differently, depending on factors such
* as the client type and the domain itself.
*/
public enum RenewalPriceBehavior {
/**
* This indicates the renewal price is the default price.
*
* <p>By default, if the domain is premium, then premium price will be used. Otherwise, the
* standard price of the TLD will be used.
*/
DEFAULT,
/**
* This indicates the domain will be renewed at standard price even if it's a premium domain.
*
* <p>We chose to name this "NONPREMIUM" rather than simply "STANDARD" to avoid confusion
* between "STANDARD" and "DEFAULT".
*
* <p>This price behavior is used with anchor tenants.
*/
NONPREMIUM,
/**
* This indicates that the renewalPrice in {@link BillingRecurrence} will be used for domain
* renewal.
*
* <p>The renewalPrice has a non-null value iff the price behavior is set to "SPECIFIED". This
* behavior is used with internal registrations.
*/
SPECIFIED
}
/** Entity id. */
@IdAllocation @Id Long id;
/** The registrar to bill. */
@Column(name = "registrarId", nullable = false)
String clientId;
/** Revision id of the entry in DomainHistory table that ths bill belongs to. */
@Column(nullable = false)
Long domainHistoryRevisionId;
/** ID of the EPP resource that the bill is for. */
@Column(nullable = false)
String domainRepoId;
/** When this event was created. For recurrence events, this is also the recurrence start time. */
@Column(nullable = false)
DateTime eventTime;
/** The reason for the bill. */
@Enumerated(EnumType.STRING)
@Column(nullable = false)
Reason reason;
/** The fully qualified domain name of the domain that the bill is for. */
@Column(name = "domain_name", nullable = false)
String targetId;
@Nullable Set<Flag> flags;
public String getRegistrarId() {
return clientId;
}
public long getDomainHistoryRevisionId() {
return domainHistoryRevisionId;
}
public String getDomainRepoId() {
return domainRepoId;
}
public DateTime getEventTime() {
return eventTime;
}
public long getId() {
return id;
}
public Reason getReason() {
return reason;
}
public String getTargetId() {
return targetId;
}
public HistoryEntryId getHistoryEntryId() {
return new HistoryEntryId(domainRepoId, domainHistoryRevisionId);
}
public ImmutableSet<Flag> getFlags() {
return nullToEmptyImmutableCopy(flags);
}
@Override
public abstract VKey<? extends BillingBase> createVKey();
/** Override Buildable.asBuilder() to give this method stronger typing. */
@Override
public abstract Builder<?, ?> asBuilder();
/** An abstract builder for {@link BillingBase}. */
public abstract static class Builder<T extends BillingBase, B extends Builder<?, ?>>
extends GenericBuilder<T, B> {
protected Builder() {}
protected Builder(T instance) {
super(instance);
}
public B setReason(Reason reason) {
getInstance().reason = reason;
return thisCastToDerived();
}
public B setId(long id) {
getInstance().id = id;
return thisCastToDerived();
}
public B setRegistrarId(String registrarId) {
getInstance().clientId = registrarId;
return thisCastToDerived();
}
public B setEventTime(DateTime eventTime) {
getInstance().eventTime = eventTime;
return thisCastToDerived();
}
public B setTargetId(String targetId) {
getInstance().targetId = targetId;
return thisCastToDerived();
}
public B setFlags(ImmutableSet<Flag> flags) {
getInstance().flags = forceEmptyToNull(checkArgumentNotNull(flags, "flags"));
return thisCastToDerived();
}
public B setDomainHistoryId(HistoryEntryId domainHistoryId) {
getInstance().domainHistoryRevisionId = domainHistoryId.getRevisionId();
getInstance().domainRepoId = domainHistoryId.getRepoId();
return thisCastToDerived();
}
public B setDomainHistory(DomainHistory domainHistory) {
return setDomainHistoryId(domainHistory.getHistoryEntryId());
}
@Override
public T build() {
T instance = getInstance();
checkNotNull(instance.reason, "Reason must be set");
checkNotNull(instance.clientId, "Registrar ID must be set");
checkNotNull(instance.eventTime, "Event time must be set");
checkNotNull(instance.targetId, "Target ID must be set");
checkNotNull(instance.domainHistoryRevisionId, "Domain History Revision ID must be set");
checkNotNull(instance.domainRepoId, "Domain Repo ID must be set");
return super.build();
}
}
}
@@ -0,0 +1,166 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.billing;
import static com.google.common.base.MoreObjects.firstNonNull;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import com.google.common.collect.ImmutableMap;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.persistence.VKey;
import google.registry.persistence.WithVKey;
import javax.persistence.AttributeOverride;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.Index;
import javax.persistence.Table;
import org.joda.time.DateTime;
/**
* An event representing a cancellation of one of the other two billable event types.
*
* <p>This is implemented as a separate event rather than a bit on BillingEvent in order to preserve
* the immutability of billing events.
*/
@Entity
@Table(
indexes = {
@Index(columnList = "registrarId"),
@Index(columnList = "eventTime"),
@Index(columnList = "domainRepoId"),
@Index(columnList = "billingTime"),
@Index(columnList = "billing_event_id"),
@Index(columnList = "billing_recurrence_id")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_cancellation_id"))
@WithVKey(Long.class)
public class BillingCancellation extends BillingBase {
/** The billing time of the charge that is being cancelled. */
DateTime billingTime;
/** The one-time billing event to cancel, or null for autorenew cancellations. */
@Column(name = "billing_event_id")
VKey<BillingEvent> billingEvent;
/** The Recurrence to cancel, or null for non-autorenew cancellations. */
@Column(name = "billing_recurrence_id")
VKey<BillingRecurrence> billingRecurrence;
public DateTime getBillingTime() {
return billingTime;
}
public VKey<? extends BillingBase> getEventKey() {
return firstNonNull(billingEvent, billingRecurrence);
}
/** The mapping from billable grace period types to originating billing event reasons. */
static final ImmutableMap<GracePeriodStatus, Reason> GRACE_PERIOD_TO_REASON =
ImmutableMap.of(
GracePeriodStatus.ADD, Reason.CREATE,
GracePeriodStatus.AUTO_RENEW, Reason.RENEW,
GracePeriodStatus.RENEW, Reason.RENEW,
GracePeriodStatus.TRANSFER, Reason.TRANSFER);
/**
* Creates a cancellation billing event (parented on the provided history key, and with the
* corresponding event time) that will cancel out the provided grace period's billing event, using
* the supplied targetId and deriving other metadata (clientId, billing time, and the cancellation
* reason) from the grace period.
*/
public static google.registry.model.billing.BillingCancellation forGracePeriod(
GracePeriod gracePeriod,
DateTime eventTime,
HistoryEntryId domainHistoryId,
String targetId) {
checkArgument(
gracePeriod.hasBillingEvent(),
"Cannot create cancellation for grace period without billing event");
Builder builder =
new Builder()
.setReason(checkNotNull(GRACE_PERIOD_TO_REASON.get(gracePeriod.getType())))
.setTargetId(targetId)
.setRegistrarId(gracePeriod.getRegistrarId())
.setEventTime(eventTime)
// The charge being cancelled will take place at the grace period's expiration time.
.setBillingTime(gracePeriod.getExpirationTime())
.setDomainHistoryId(domainHistoryId);
// Set the grace period's billing event using the appropriate Cancellation builder method.
if (gracePeriod.getBillingEvent() != null) {
builder.setBillingEvent(gracePeriod.getBillingEvent());
} else if (gracePeriod.getBillingRecurrence() != null) {
builder.setBillingRecurrence(gracePeriod.getBillingRecurrence());
}
return builder.build();
}
@Override
public VKey<google.registry.model.billing.BillingCancellation> createVKey() {
return createVKey(getId());
}
public static VKey<google.registry.model.billing.BillingCancellation> createVKey(long id) {
return VKey.create(google.registry.model.billing.BillingCancellation.class, id);
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
}
/**
* A builder for {@link google.registry.model.billing.BillingCancellation} since it is immutable.
*/
public static class Builder
extends BillingBase.Builder<google.registry.model.billing.BillingCancellation, Builder> {
public Builder() {}
private Builder(google.registry.model.billing.BillingCancellation instance) {
super(instance);
}
public Builder setBillingTime(DateTime billingTime) {
getInstance().billingTime = billingTime;
return this;
}
public Builder setBillingEvent(VKey<BillingEvent> billingEvent) {
getInstance().billingEvent = billingEvent;
return this;
}
public Builder setBillingRecurrence(VKey<BillingRecurrence> billingRecurrence) {
getInstance().billingRecurrence = billingRecurrence;
return this;
}
@Override
public google.registry.model.billing.BillingCancellation build() {
google.registry.model.billing.BillingCancellation instance = getInstance();
checkNotNull(instance.billingTime, "Must set billing time");
checkNotNull(instance.reason, "Must set reason");
checkState(
(instance.billingEvent == null) != (instance.billingRecurrence == null),
"Cancellations must have exactly one billing event key set");
return super.build();
}
}
}
@@ -1,4 +1,4 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -14,740 +14,197 @@
package google.registry.model.billing;
import static com.google.common.base.MoreObjects.firstNonNull;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.util.CollectionUtils.forceEmptyToNull;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableCopy;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import google.registry.model.Buildable;
import google.registry.model.ImmutableObject;
import google.registry.model.UnsafeSerializable;
import google.registry.model.annotations.IdAllocation;
import google.registry.model.common.TimeOfYear;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
import google.registry.persistence.VKey;
import google.registry.persistence.WithVKey;
import google.registry.persistence.converter.JodaMoneyType;
import java.util.Optional;
import java.util.Set;
import javax.annotation.Nullable;
import javax.persistence.AttributeOverride;
import javax.persistence.AttributeOverrides;
import javax.persistence.Column;
import javax.persistence.Embedded;
import javax.persistence.Entity;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.Id;
import javax.persistence.Index;
import javax.persistence.MappedSuperclass;
import javax.persistence.Table;
import org.hibernate.annotations.Columns;
import org.hibernate.annotations.Type;
import org.joda.money.Money;
import org.joda.time.DateTime;
/** A billable event in a domain's lifecycle. */
@MappedSuperclass
public abstract class BillingEvent extends ImmutableObject
implements Buildable, TransferServerApproveEntity, UnsafeSerializable {
/** A one-time billable event. */
@Entity
@Table(
indexes = {
@Index(columnList = "registrarId"),
@Index(columnList = "eventTime"),
@Index(columnList = "billingTime"),
@Index(columnList = "syntheticCreationTime"),
@Index(columnList = "domainRepoId"),
@Index(columnList = "allocationToken"),
@Index(columnList = "cancellation_matching_billing_recurrence_id")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_event_id"))
@WithVKey(Long.class)
public class BillingEvent extends BillingBase {
/** The reason for the bill, which maps 1:1 to skus in go/registry-billing-skus. */
public enum Reason {
CREATE(true),
@Deprecated // DO NOT USE THIS REASON. IT REMAINS BECAUSE OF HISTORICAL DATA. SEE b/31676071.
ERROR(false),
FEE_EARLY_ACCESS(true),
RENEW(true),
RESTORE(true),
SERVER_STATUS(false),
TRANSFER(true);
/** The billable value. */
@Type(type = JodaMoneyType.TYPE_NAME)
@Columns(columns = {@Column(name = "cost_amount"), @Column(name = "cost_currency")})
Money cost;
private final boolean requiresPeriod;
Reason(boolean requiresPeriod) {
this.requiresPeriod = requiresPeriod;
}
/**
* Returns whether billing events with this reason have a period years associated with them.
*
* <p>Note that this is an "if an only if" condition.
*/
public boolean hasPeriodYears() {
return requiresPeriod;
}
}
/** Set of flags that can be applied to billing events. */
public enum Flag {
ALLOCATION,
ANCHOR_TENANT,
AUTO_RENEW,
/** Landrush billing events are historical only and are no longer created. */
LANDRUSH,
/**
* This flag is used on create {@link OneTime} billing events for domains that were reserved.
*
* <p>This can happen when allocation tokens are used or superusers override a domain
* reservation. These cases can need special handling in billing/invoicing. Anchor tenants will
* never have this flag applied; they will have ANCHOR_TENANT instead.
*/
RESERVED,
SUNRISE,
/**
* This flag will be added to any {@link OneTime} events that are created via, e.g., an
* automated process to expand {@link Recurring} events.
*/
SYNTHETIC
}
/** When the cost should be billed. */
DateTime billingTime;
/**
* Sets of renewal price behaviors that can be applied to billing recurrences.
*
* <p>When a client renews a domain, they could be charged differently, depending on factors such
* as the client type and the domain itself.
* The period in years of the action being billed for, if applicable, otherwise null. Used for
* financial reporting.
*/
public enum RenewalPriceBehavior {
/**
* This indicates the renewal price is the default price.
*
* <p>By default, if the domain is premium, then premium price will be used. Otherwise, the
* standard price of the TLD will be used.
*/
DEFAULT,
/**
* This indicates the domain will be renewed at standard price even if it's a premium domain.
*
* <p>We chose to name this "NONPREMIUM" rather than simply "STANDARD" to avoid confusion
* between "STANDARD" and "DEFAULT".
*
* <p>This price behavior is used with anchor tenants.
*/
NONPREMIUM,
/**
* This indicates that the renewalPrice in {@link Recurring} will be used for domain renewal.
*
* <p>The renewalPrice has a non-null value iff the price behavior is set to "SPECIFIED". This
* behavior is used with internal registrations.
*/
SPECIFIED
Integer periodYears;
/**
* For {@link Flag#SYNTHETIC} events, when this event was persisted to the database (i.e. the
* cursor position at the time the recurrence expansion job was last run). In the event a job
* needs to be undone, a query on this field will return the complete set of potentially bad
* events.
*/
DateTime syntheticCreationTime;
/**
* For {@link Flag#SYNTHETIC} events, a {@link VKey} to the {@link BillingRecurrence} from which
* this {@link google.registry.model.billing.BillingEvent} was created. This is needed in order to
* properly match billing events against {@link BillingCancellation}s.
*/
@Column(name = "cancellation_matching_billing_recurrence_id")
VKey<BillingRecurrence> cancellationMatchingBillingEvent;
/**
* For {@link Flag#SYNTHETIC} events, the {@link DomainHistory} revision ID of the {@link
* BillingRecurrence} from which this {@link google.registry.model.billing.BillingEvent} was
* created. This is needed in order to recreate the {@link VKey} when reading from SQL.
*/
@Column(name = "recurrence_history_revision_id")
Long recurrenceHistoryRevisionId;
/**
* The {@link AllocationToken} used in the creation of this event, or null if one was not used.
*/
@Nullable VKey<AllocationToken> allocationToken;
public Money getCost() {
return cost;
}
/** Entity id. */
@IdAllocation @Id Long id;
/** The registrar to bill. */
@Column(name = "registrarId", nullable = false)
String clientId;
/** Revision id of the entry in DomainHistory table that ths bill belongs to. */
@Column(nullable = false)
Long domainHistoryRevisionId;
/** ID of the EPP resource that the bill is for. */
@Column(nullable = false)
String domainRepoId;
/** When this event was created. For recurring events, this is also the recurrence start time. */
@Column(nullable = false)
DateTime eventTime;
/** The reason for the bill. */
@Enumerated(EnumType.STRING)
@Column(nullable = false)
Reason reason;
/** The fully qualified domain name of the domain that the bill is for. */
@Column(name = "domain_name", nullable = false)
String targetId;
@Nullable Set<Flag> flags;
public String getRegistrarId() {
return clientId;
public DateTime getBillingTime() {
return billingTime;
}
public long getDomainHistoryRevisionId() {
return domainHistoryRevisionId;
public Integer getPeriodYears() {
return periodYears;
}
public String getDomainRepoId() {
return domainRepoId;
public DateTime getSyntheticCreationTime() {
return syntheticCreationTime;
}
public DateTime getEventTime() {
return eventTime;
public VKey<BillingRecurrence> getCancellationMatchingBillingEvent() {
return cancellationMatchingBillingEvent;
}
public long getId() {
return id;
public Long getRecurrenceHistoryRevisionId() {
return recurrenceHistoryRevisionId;
}
public Reason getReason() {
return reason;
}
public String getTargetId() {
return targetId;
}
public HistoryEntryId getHistoryEntryId() {
return new HistoryEntryId(domainRepoId, domainHistoryRevisionId);
}
public ImmutableSet<Flag> getFlags() {
return nullToEmptyImmutableCopy(flags);
public Optional<VKey<AllocationToken>> getAllocationToken() {
return Optional.ofNullable(allocationToken);
}
@Override
public abstract VKey<? extends BillingEvent> createVKey();
public VKey<google.registry.model.billing.BillingEvent> createVKey() {
return createVKey(getId());
}
public static VKey<google.registry.model.billing.BillingEvent> createVKey(long id) {
return VKey.create(google.registry.model.billing.BillingEvent.class, id);
}
/** Override Buildable.asBuilder() to give this method stronger typing. */
@Override
public abstract Builder<?, ?> asBuilder();
public Builder asBuilder() {
return new Builder(clone(this));
}
/** An abstract builder for {@link BillingEvent}. */
public abstract static class Builder<T extends BillingEvent, B extends Builder<?, ?>>
extends GenericBuilder<T, B> {
/** A builder for {@link google.registry.model.billing.BillingEvent} since it is immutable. */
public static class Builder
extends BillingBase.Builder<google.registry.model.billing.BillingEvent, Builder> {
protected Builder() {}
public Builder() {}
protected Builder(T instance) {
private Builder(google.registry.model.billing.BillingEvent instance) {
super(instance);
}
public B setReason(Reason reason) {
getInstance().reason = reason;
return thisCastToDerived();
public Builder setCost(Money cost) {
getInstance().cost = cost;
return this;
}
public B setId(long id) {
getInstance().id = id;
return thisCastToDerived();
public Builder setPeriodYears(Integer periodYears) {
checkNotNull(periodYears);
checkArgument(periodYears > 0);
getInstance().periodYears = periodYears;
return this;
}
public B setRegistrarId(String registrarId) {
getInstance().clientId = registrarId;
return thisCastToDerived();
public Builder setBillingTime(DateTime billingTime) {
getInstance().billingTime = billingTime;
return this;
}
public B setEventTime(DateTime eventTime) {
getInstance().eventTime = eventTime;
return thisCastToDerived();
public Builder setSyntheticCreationTime(DateTime syntheticCreationTime) {
getInstance().syntheticCreationTime = syntheticCreationTime;
return this;
}
public B setTargetId(String targetId) {
getInstance().targetId = targetId;
return thisCastToDerived();
public Builder setCancellationMatchingBillingEvent(
BillingRecurrence cancellationMatchingBillingEvent) {
getInstance().cancellationMatchingBillingEvent =
cancellationMatchingBillingEvent.createVKey();
getInstance().recurrenceHistoryRevisionId =
cancellationMatchingBillingEvent.getDomainHistoryRevisionId();
return this;
}
public B setFlags(ImmutableSet<Flag> flags) {
getInstance().flags = forceEmptyToNull(checkArgumentNotNull(flags, "flags"));
return thisCastToDerived();
}
public B setDomainHistoryId(HistoryEntryId domainHistoryId) {
getInstance().domainHistoryRevisionId = domainHistoryId.getRevisionId();
getInstance().domainRepoId = domainHistoryId.getRepoId();
return thisCastToDerived();
}
public B setDomainHistory(DomainHistory domainHistory) {
return setDomainHistoryId(domainHistory.getHistoryEntryId());
public Builder setAllocationToken(@Nullable VKey<AllocationToken> allocationToken) {
getInstance().allocationToken = allocationToken;
return this;
}
@Override
public T build() {
T instance = getInstance();
checkNotNull(instance.reason, "Reason must be set");
checkNotNull(instance.clientId, "Registrar ID must be set");
checkNotNull(instance.eventTime, "Event time must be set");
checkNotNull(instance.targetId, "Target ID must be set");
checkNotNull(instance.domainHistoryRevisionId, "Domain History Revision ID must be set");
checkNotNull(instance.domainRepoId, "Domain Repo ID must be set");
public google.registry.model.billing.BillingEvent build() {
google.registry.model.billing.BillingEvent instance = getInstance();
checkNotNull(instance.billingTime);
checkNotNull(instance.cost);
checkState(!instance.cost.isNegative(), "Costs should be non-negative.");
// TODO(mcilwain): Enforce this check on all billing events (not just more recent ones)
// post-migration after we add the missing period years values in SQL.
if (instance.eventTime.isAfter(DateTime.parse("2019-01-01T00:00:00Z"))) {
checkState(
instance.reason.hasPeriodYears() == (instance.periodYears != null),
"Period years must be set if and only if reason is "
+ "CREATE, FEE_EARLY_ACCESS, RENEW, RESTORE or TRANSFER.");
}
checkState(
instance.getFlags().contains(Flag.SYNTHETIC) == (instance.syntheticCreationTime != null),
"Synthetic creation time must be set if and only if the SYNTHETIC flag is set.");
checkState(
instance.getFlags().contains(Flag.SYNTHETIC)
== (instance.cancellationMatchingBillingEvent != null),
"Cancellation matching billing event must be set if and only if the SYNTHETIC flag "
+ "is set.");
return super.build();
}
}
/** A one-time billable event. */
@Entity(name = "BillingEvent")
@Table(
indexes = {
@Index(columnList = "registrarId"),
@Index(columnList = "eventTime"),
@Index(columnList = "billingTime"),
@Index(columnList = "syntheticCreationTime"),
@Index(columnList = "domainRepoId"),
@Index(columnList = "allocationToken"),
@Index(columnList = "cancellation_matching_billing_recurrence_id")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_event_id"))
@WithVKey(Long.class)
public static class OneTime extends BillingEvent {
/** The billable value. */
@Type(type = JodaMoneyType.TYPE_NAME)
@Columns(columns = {@Column(name = "cost_amount"), @Column(name = "cost_currency")})
Money cost;
/** When the cost should be billed. */
DateTime billingTime;
/**
* The period in years of the action being billed for, if applicable, otherwise null. Used for
* financial reporting.
*/
Integer periodYears;
/**
* For {@link Flag#SYNTHETIC} events, when this event was persisted to the database (i.e. the
* cursor position at the time the recurrence expansion job was last run). In the event a job
* needs to be undone, a query on this field will return the complete set of potentially bad
* events.
*/
DateTime syntheticCreationTime;
/**
* For {@link Flag#SYNTHETIC} events, a {@link VKey} to the {@link Recurring} from which this
* {@link OneTime} was created. This is needed in order to properly match billing events against
* {@link Cancellation}s.
*/
@Column(name = "cancellation_matching_billing_recurrence_id")
VKey<Recurring> cancellationMatchingBillingEvent;
/**
* For {@link Flag#SYNTHETIC} events, the {@link DomainHistory} revision ID of the {@link
* Recurring} from which this {@link OneTime} was created. This is needed in order to recreate
* the {@link VKey} when reading from SQL.
*/
@Column(name = "recurrence_history_revision_id")
Long recurringEventHistoryRevisionId;
/**
* The {@link AllocationToken} used in the creation of this event, or null if one was not used.
*/
@Nullable VKey<AllocationToken> allocationToken;
public Money getCost() {
return cost;
}
public DateTime getBillingTime() {
return billingTime;
}
public Integer getPeriodYears() {
return periodYears;
}
public DateTime getSyntheticCreationTime() {
return syntheticCreationTime;
}
public VKey<Recurring> getCancellationMatchingBillingEvent() {
return cancellationMatchingBillingEvent;
}
public Long getRecurringEventHistoryRevisionId() {
return recurringEventHistoryRevisionId;
}
public Optional<VKey<AllocationToken>> getAllocationToken() {
return Optional.ofNullable(allocationToken);
}
@Override
public VKey<OneTime> createVKey() {
return createVKey(getId());
}
public static VKey<OneTime> createVKey(long id) {
return VKey.create(OneTime.class, id);
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
}
/** A builder for {@link OneTime} since it is immutable. */
public static class Builder extends BillingEvent.Builder<OneTime, Builder> {
public Builder() {}
private Builder(OneTime instance) {
super(instance);
}
public Builder setCost(Money cost) {
getInstance().cost = cost;
return this;
}
public Builder setPeriodYears(Integer periodYears) {
checkNotNull(periodYears);
checkArgument(periodYears > 0);
getInstance().periodYears = periodYears;
return this;
}
public Builder setBillingTime(DateTime billingTime) {
getInstance().billingTime = billingTime;
return this;
}
public Builder setSyntheticCreationTime(DateTime syntheticCreationTime) {
getInstance().syntheticCreationTime = syntheticCreationTime;
return this;
}
public Builder setCancellationMatchingBillingEvent(
Recurring cancellationMatchingBillingEvent) {
getInstance().cancellationMatchingBillingEvent =
cancellationMatchingBillingEvent.createVKey();
getInstance().recurringEventHistoryRevisionId =
cancellationMatchingBillingEvent.getDomainHistoryRevisionId();
return this;
}
public Builder setAllocationToken(@Nullable VKey<AllocationToken> allocationToken) {
getInstance().allocationToken = allocationToken;
return this;
}
@Override
public OneTime build() {
OneTime instance = getInstance();
checkNotNull(instance.billingTime);
checkNotNull(instance.cost);
checkState(!instance.cost.isNegative(), "Costs should be non-negative.");
// TODO(mcilwain): Enforce this check on all billing events (not just more recent ones)
// post-migration after we add the missing period years values in SQL.
if (instance.eventTime.isAfter(DateTime.parse("2019-01-01T00:00:00Z"))) {
checkState(
instance.reason.hasPeriodYears() == (instance.periodYears != null),
"Period years must be set if and only if reason is "
+ "CREATE, FEE_EARLY_ACCESS, RENEW, RESTORE or TRANSFER.");
}
checkState(
instance.getFlags().contains(Flag.SYNTHETIC)
== (instance.syntheticCreationTime != null),
"Synthetic creation time must be set if and only if the SYNTHETIC flag is set.");
checkState(
instance.getFlags().contains(Flag.SYNTHETIC)
== (instance.cancellationMatchingBillingEvent != null),
"Cancellation matching billing event must be set if and only if the SYNTHETIC flag "
+ "is set.");
return super.build();
}
}
}
/**
* A recurring billable event.
*
* <p>Unlike {@link OneTime} events, these do not store an explicit cost, since the cost of the
* recurring event might change and each time we bill for it, we need to bill at the current cost,
* not the value that was in use at the time the recurrence was created.
*/
@Entity(name = "BillingRecurrence")
@Table(
indexes = {
@Index(columnList = "registrarId"),
@Index(columnList = "eventTime"),
@Index(columnList = "domainRepoId"),
@Index(columnList = "recurrenceEndTime"),
@Index(columnList = "recurrenceLastExpansion"),
@Index(columnList = "recurrence_time_of_year")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_recurrence_id"))
@WithVKey(Long.class)
public static class Recurring extends BillingEvent {
/**
* The billing event recurs every year between {@link #eventTime} and this time on the [month,
* day, time] specified in {@link #recurrenceTimeOfYear}.
*/
DateTime recurrenceEndTime;
/**
* The most recent {@link DateTime} when this recurrence was expanded.
*
* <p>We only bother checking recurrences for potential expansion if this is at least one year
* in the past. If it's more recent than that, it means that the recurrence was already expanded
* too recently to need to be checked again (as domains autorenew each year).
*/
@Column(nullable = false)
DateTime recurrenceLastExpansion;
/**
* The eventTime recurs every year on this [month, day, time] between {@link #eventTime} and
* {@link #recurrenceEndTime}, inclusive of the start but not of the end.
*
* <p>This field is denormalized from {@link #eventTime} to allow for an efficient index, but it
* always has the same data as that field.
*
* <p>Note that this is a recurrence of the event time, not the billing time. The billing time
* can be calculated by adding the relevant grace period length to this date. The reason for
* this requirement is that the event time recurs on a {@link org.joda.time.Period} schedule
* (same day of year, which can be 365 or 366 days later) which is what {@link TimeOfYear} can
* model, whereas the billing time is a fixed {@link org.joda.time.Duration} later.
*/
@Embedded
@AttributeOverrides(
@AttributeOverride(name = "timeString", column = @Column(name = "recurrence_time_of_year")))
TimeOfYear recurrenceTimeOfYear;
/**
* The renewal price for domain renewal if and only if it's specified.
*
* <p>This price column remains null except when the renewal price behavior of the billing is
* SPECIFIED. This column is used for internal registrations.
*/
@Nullable
@Type(type = JodaMoneyType.TYPE_NAME)
@Columns(
columns = {@Column(name = "renewalPriceAmount"), @Column(name = "renewalPriceCurrency")})
Money renewalPrice;
@Enumerated(EnumType.STRING)
@Column(name = "renewalPriceBehavior", nullable = false)
RenewalPriceBehavior renewalPriceBehavior = RenewalPriceBehavior.DEFAULT;
public DateTime getRecurrenceEndTime() {
return recurrenceEndTime;
}
public DateTime getRecurrenceLastExpansion() {
return recurrenceLastExpansion;
}
public TimeOfYear getRecurrenceTimeOfYear() {
return recurrenceTimeOfYear;
}
public RenewalPriceBehavior getRenewalPriceBehavior() {
return renewalPriceBehavior;
}
public Optional<Money> getRenewalPrice() {
return Optional.ofNullable(renewalPrice);
}
@Override
public VKey<Recurring> createVKey() {
return createVKey(getId());
}
public static VKey<Recurring> createVKey(Long id) {
return VKey.create(Recurring.class, id);
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
}
/** A builder for {@link Recurring} since it is immutable. */
public static class Builder extends BillingEvent.Builder<Recurring, Builder> {
public Builder() {}
private Builder(Recurring instance) {
super(instance);
}
public Builder setRecurrenceEndTime(DateTime recurrenceEndTime) {
getInstance().recurrenceEndTime = recurrenceEndTime;
return this;
}
public Builder setRecurrenceLastExpansion(DateTime recurrenceLastExpansion) {
getInstance().recurrenceLastExpansion = recurrenceLastExpansion;
return this;
}
public Builder setRenewalPriceBehavior(RenewalPriceBehavior renewalPriceBehavior) {
getInstance().renewalPriceBehavior = renewalPriceBehavior;
return this;
}
public Builder setRenewalPrice(@Nullable Money renewalPrice) {
getInstance().renewalPrice = renewalPrice;
return this;
}
@Override
public Recurring build() {
Recurring instance = getInstance();
checkNotNull(instance.eventTime);
checkNotNull(instance.reason);
// Don't require recurrenceLastExpansion to be individually set on every new Recurrence.
// The correct default value if not otherwise set is the event time of the recurrence minus
// 1 year.
instance.recurrenceLastExpansion =
Optional.ofNullable(instance.recurrenceLastExpansion)
.orElse(instance.eventTime.minusYears(1));
checkArgument(
instance.renewalPriceBehavior == RenewalPriceBehavior.SPECIFIED
^ instance.renewalPrice == null,
"Renewal price can have a value if and only if the renewal price behavior is"
+ " SPECIFIED");
instance.recurrenceTimeOfYear = TimeOfYear.fromDateTime(instance.eventTime);
instance.recurrenceEndTime =
Optional.ofNullable(instance.recurrenceEndTime).orElse(END_OF_TIME);
return super.build();
}
}
}
/**
* An event representing a cancellation of one of the other two billable event types.
*
* <p>This is implemented as a separate event rather than a bit on BillingEvent in order to
* preserve the immutability of billing events.
*/
@Entity(name = "BillingCancellation")
@Table(
indexes = {
@Index(columnList = "registrarId"),
@Index(columnList = "eventTime"),
@Index(columnList = "domainRepoId"),
@Index(columnList = "billingTime"),
@Index(columnList = "billing_event_id"),
@Index(columnList = "billing_recurrence_id")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_cancellation_id"))
@WithVKey(Long.class)
public static class Cancellation extends BillingEvent {
/** The billing time of the charge that is being cancelled. */
DateTime billingTime;
/**
* The one-time billing event to cancel, or null for autorenew cancellations.
*
* <p>Although the type is {@link VKey} the name "ref" is preserved for historical reasons.
*/
@Column(name = "billing_event_id")
VKey<OneTime> refOneTime;
/**
* The recurring billing event to cancel, or null for non-autorenew cancellations.
*
* <p>Although the type is {@link VKey} the name "ref" is preserved for historical reasons.
*/
@Column(name = "billing_recurrence_id")
VKey<Recurring> refRecurring;
public DateTime getBillingTime() {
return billingTime;
}
public VKey<? extends BillingEvent> getEventKey() {
return firstNonNull(refOneTime, refRecurring);
}
/** The mapping from billable grace period types to originating billing event reasons. */
static final ImmutableMap<GracePeriodStatus, Reason> GRACE_PERIOD_TO_REASON =
ImmutableMap.of(
GracePeriodStatus.ADD, Reason.CREATE,
GracePeriodStatus.AUTO_RENEW, Reason.RENEW,
GracePeriodStatus.RENEW, Reason.RENEW,
GracePeriodStatus.TRANSFER, Reason.TRANSFER);
/**
* Creates a cancellation billing event (parented on the provided history key, and with the
* corresponding event time) that will cancel out the provided grace period's billing event,
* using the supplied targetId and deriving other metadata (clientId, billing time, and the
* cancellation reason) from the grace period.
*/
public static Cancellation forGracePeriod(
GracePeriod gracePeriod,
DateTime eventTime,
HistoryEntryId domainHistoryId,
String targetId) {
checkArgument(
gracePeriod.hasBillingEvent(),
"Cannot create cancellation for grace period without billing event");
Builder builder =
new Builder()
.setReason(checkNotNull(GRACE_PERIOD_TO_REASON.get(gracePeriod.getType())))
.setTargetId(targetId)
.setRegistrarId(gracePeriod.getRegistrarId())
.setEventTime(eventTime)
// The charge being cancelled will take place at the grace period's expiration time.
.setBillingTime(gracePeriod.getExpirationTime())
.setDomainHistoryId(domainHistoryId);
// Set the grace period's billing event using the appropriate Cancellation builder method.
if (gracePeriod.getOneTimeBillingEvent() != null) {
builder.setOneTimeEventKey(gracePeriod.getOneTimeBillingEvent());
} else if (gracePeriod.getRecurringBillingEvent() != null) {
builder.setRecurringEventKey(gracePeriod.getRecurringBillingEvent());
}
return builder.build();
}
@Override
public VKey<Cancellation> createVKey() {
return createVKey(getId());
}
public static VKey<Cancellation> createVKey(long id) {
return VKey.create(Cancellation.class, id);
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
}
/** A builder for {@link Cancellation} since it is immutable. */
public static class Builder extends BillingEvent.Builder<Cancellation, Builder> {
public Builder() {}
private Builder(Cancellation instance) {
super(instance);
}
public Builder setBillingTime(DateTime billingTime) {
getInstance().billingTime = billingTime;
return this;
}
public Builder setOneTimeEventKey(VKey<OneTime> eventKey) {
getInstance().refOneTime = eventKey;
return this;
}
public Builder setRecurringEventKey(VKey<Recurring> eventKey) {
getInstance().refRecurring = eventKey;
return this;
}
@Override
public Cancellation build() {
Cancellation instance = getInstance();
checkNotNull(instance.billingTime, "Must set billing time");
checkNotNull(instance.reason, "Must set reason");
checkState(
(instance.refOneTime == null) != (instance.refRecurring == null),
"Cancellations must have exactly one billing event key set");
return super.build();
}
}
}
}
@@ -0,0 +1,199 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.billing;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import google.registry.model.common.TimeOfYear;
import google.registry.persistence.VKey;
import google.registry.persistence.WithVKey;
import google.registry.persistence.converter.JodaMoneyType;
import java.util.Optional;
import javax.annotation.Nullable;
import javax.persistence.AttributeOverride;
import javax.persistence.AttributeOverrides;
import javax.persistence.Column;
import javax.persistence.Embedded;
import javax.persistence.Entity;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.Index;
import javax.persistence.Table;
import org.hibernate.annotations.Columns;
import org.hibernate.annotations.Type;
import org.joda.money.Money;
import org.joda.time.DateTime;
/**
* A recurring billable event.
*
* <p>Unlike {@link BillingEvent} events, these do not store an explicit cost, since the cost of the
* recurring event might change and each time we bill for it, we need to bill at the current cost,
* not the value that was in use at the time the recurrence was created.
*/
@Entity
@Table(
indexes = {
@Index(columnList = "registrarId"),
@Index(columnList = "eventTime"),
@Index(columnList = "domainRepoId"),
@Index(columnList = "recurrenceEndTime"),
@Index(columnList = "recurrenceLastExpansion"),
@Index(columnList = "recurrence_time_of_year")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_recurrence_id"))
@WithVKey(Long.class)
public class BillingRecurrence extends BillingBase {
/**
* The billing event recurs every year between {@link #eventTime} and this time on the [month,
* day, time] specified in {@link #recurrenceTimeOfYear}.
*/
DateTime recurrenceEndTime;
/**
* The most recent {@link DateTime} when this recurrence was expanded.
*
* <p>We only bother checking recurrences for potential expansion if this is at least one year in
* the past. If it's more recent than that, it means that the recurrence was already expanded too
* recently to need to be checked again (as domains autorenew each year).
*/
@Column(nullable = false)
DateTime recurrenceLastExpansion;
/**
* The eventTime recurs every year on this [month, day, time] between {@link #eventTime} and
* {@link #recurrenceEndTime}, inclusive of the start but not of the end.
*
* <p>This field is denormalized from {@link #eventTime} to allow for an efficient index, but it
* always has the same data as that field.
*
* <p>Note that this is a recurrence of the event time, not the billing time. The billing time can
* be calculated by adding the relevant grace period length to this date. The reason for this
* requirement is that the event time recurs on a {@link org.joda.time.Period} schedule (same day
* of year, which can be 365 or 366 days later) which is what {@link TimeOfYear} can model,
* whereas the billing time is a fixed {@link org.joda.time.Duration} later.
*/
@Embedded
@AttributeOverrides(
@AttributeOverride(name = "timeString", column = @Column(name = "recurrence_time_of_year")))
TimeOfYear recurrenceTimeOfYear;
/**
* The renewal price for domain renewal if and only if it's specified.
*
* <p>This price column remains null except when the renewal price behavior of the billing is
* SPECIFIED. This column is used for internal registrations.
*/
@Nullable
@Type(type = JodaMoneyType.TYPE_NAME)
@Columns(columns = {@Column(name = "renewalPriceAmount"), @Column(name = "renewalPriceCurrency")})
Money renewalPrice;
@Enumerated(EnumType.STRING)
@Column(name = "renewalPriceBehavior", nullable = false)
RenewalPriceBehavior renewalPriceBehavior = RenewalPriceBehavior.DEFAULT;
public DateTime getRecurrenceEndTime() {
return recurrenceEndTime;
}
public DateTime getRecurrenceLastExpansion() {
return recurrenceLastExpansion;
}
public TimeOfYear getRecurrenceTimeOfYear() {
return recurrenceTimeOfYear;
}
public RenewalPriceBehavior getRenewalPriceBehavior() {
return renewalPriceBehavior;
}
public Optional<Money> getRenewalPrice() {
return Optional.ofNullable(renewalPrice);
}
@Override
public VKey<google.registry.model.billing.BillingRecurrence> createVKey() {
return createVKey(getId());
}
public static VKey<google.registry.model.billing.BillingRecurrence> createVKey(Long id) {
return VKey.create(google.registry.model.billing.BillingRecurrence.class, id);
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
}
/**
* A builder for {@link google.registry.model.billing.BillingRecurrence} since it is immutable.
*/
public static class Builder
extends BillingBase.Builder<google.registry.model.billing.BillingRecurrence, Builder> {
public Builder() {}
private Builder(google.registry.model.billing.BillingRecurrence instance) {
super(instance);
}
public Builder setRecurrenceEndTime(DateTime recurrenceEndTime) {
getInstance().recurrenceEndTime = recurrenceEndTime;
return this;
}
public Builder setRecurrenceLastExpansion(DateTime recurrenceLastExpansion) {
getInstance().recurrenceLastExpansion = recurrenceLastExpansion;
return this;
}
public Builder setRenewalPriceBehavior(RenewalPriceBehavior renewalPriceBehavior) {
getInstance().renewalPriceBehavior = renewalPriceBehavior;
return this;
}
public Builder setRenewalPrice(@Nullable Money renewalPrice) {
getInstance().renewalPrice = renewalPrice;
return this;
}
@Override
public google.registry.model.billing.BillingRecurrence build() {
google.registry.model.billing.BillingRecurrence instance = getInstance();
checkNotNull(instance.eventTime);
checkNotNull(instance.reason);
// Don't require recurrenceLastExpansion to be individually set on every new Recurrence.
// The correct default value if not otherwise set is the event time of the recurrence minus
// 1 year.
instance.recurrenceLastExpansion =
Optional.ofNullable(instance.recurrenceLastExpansion)
.orElse(instance.eventTime.minusYears(1));
checkArgument(
instance.renewalPriceBehavior == RenewalPriceBehavior.SPECIFIED
^ instance.renewalPrice == null,
"Renewal price can have a value if and only if the renewal price behavior is"
+ " SPECIFIED");
instance.recurrenceTimeOfYear = TimeOfYear.fromDateTime(instance.eventTime);
instance.recurrenceEndTime =
Optional.ofNullable(instance.recurrenceEndTime).orElse(END_OF_TIME);
return super.build();
}
}
}
@@ -18,7 +18,7 @@ import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.dns.DnsUtils.TargetType;
import google.registry.dns.PublishDnsUpdatesAction;
import google.registry.model.ImmutableObject;
import google.registry.persistence.VKey;
@@ -44,7 +44,7 @@ import com.google.gson.annotations.Expose;
import google.registry.flows.ResourceFlowUtils;
import google.registry.model.EppResource;
import google.registry.model.EppResource.ResourceWithTransferData;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.contact.Contact;
import google.registry.model.domain.launch.LaunchNotice;
import google.registry.model.domain.rgp.GracePeriodStatus;
@@ -200,7 +200,7 @@ public class DomainBase extends EppResource
* should be created, and this field should be updated to point to the new one.
*/
@Column(name = "billing_recurrence_id")
VKey<Recurring> autorenewBillingEvent;
VKey<BillingRecurrence> autorenewBillingEvent;
/**
* The recurring poll message associated with this domain's autorenewals.
@@ -286,7 +286,7 @@ public class DomainBase extends EppResource
return deletePollMessage;
}
public VKey<Recurring> getAutorenewBillingEvent() {
public VKey<BillingRecurrence> getAutorenewBillingEvent() {
return autorenewBillingEvent;
}
@@ -520,7 +520,7 @@ public class DomainBase extends EppResource
builder
.setRegistrationExpirationTime(newExpirationTime)
.addGracePeriod(
GracePeriod.createForRecurring(
GracePeriod.createForRecurrence(
GracePeriodStatus.AUTO_RENEW,
domain.getRepoId(),
lastAutorenewTime.plus(Tld.get(domain.getTld()).getAutoRenewGracePeriodLength()),
@@ -847,7 +847,7 @@ public class DomainBase extends EppResource
return thisCastToDerived();
}
public B setAutorenewBillingEvent(VKey<Recurring> autorenewBillingEvent) {
public B setAutorenewBillingEvent(VKey<BillingRecurrence> autorenewBillingEvent) {
getInstance().autorenewBillingEvent = autorenewBillingEvent;
return thisCastToDerived();
}
@@ -20,7 +20,7 @@ import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.google.common.annotations.VisibleForTesting;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.persistence.VKey;
@@ -60,23 +60,23 @@ public class GracePeriod extends GracePeriodBase {
String domainRepoId,
DateTime expirationTime,
String registrarId,
@Nullable VKey<BillingEvent.OneTime> billingEventOneTime,
@Nullable VKey<BillingEvent.Recurring> billingEventRecurring,
@Nullable VKey<BillingEvent> billingEvent,
@Nullable VKey<BillingRecurrence> billingRecurrence,
@Nullable Long gracePeriodId) {
checkArgument(
billingEventOneTime == null || billingEventRecurring == null,
billingEvent == null || billingRecurrence == null,
"A grace period can have at most one billing event");
checkArgument(
(billingEventRecurring != null) == GracePeriodStatus.AUTO_RENEW.equals(type),
"Recurring billing events must be present on (and only on) autorenew grace periods");
(billingRecurrence != null) == GracePeriodStatus.AUTO_RENEW.equals(type),
"BillingRecurrences must be present on (and only on) autorenew grace periods");
GracePeriod instance = new GracePeriod();
instance.gracePeriodId = gracePeriodId == null ? allocateId() : gracePeriodId;
instance.type = checkArgumentNotNull(type);
instance.domainRepoId = checkArgumentNotNull(domainRepoId);
instance.expirationTime = checkArgumentNotNull(expirationTime);
instance.clientId = checkArgumentNotNull(registrarId);
instance.billingEventOneTime = billingEventOneTime;
instance.billingEventRecurring = billingEventRecurring;
instance.billingEvent = billingEvent;
instance.billingRecurrence = billingRecurrence;
return instance;
}
@@ -92,7 +92,7 @@ public class GracePeriod extends GracePeriodBase {
String domainRepoId,
DateTime expirationTime,
String registrarId,
@Nullable VKey<BillingEvent.OneTime> billingEventOneTime) {
@Nullable VKey<BillingEvent> billingEventOneTime) {
return createInternal(
type, domainRepoId, expirationTime, registrarId, billingEventOneTime, null, null);
}
@@ -111,7 +111,7 @@ public class GracePeriod extends GracePeriodBase {
String domainRepoId,
DateTime expirationTime,
String registrarId,
@Nullable VKey<BillingEvent.OneTime> billingEventOneTime,
@Nullable VKey<BillingEvent> billingEventOneTime,
@Nullable Long gracePeriodId) {
return createInternal(
type, domainRepoId, expirationTime, registrarId, billingEventOneTime, null, gracePeriodId);
@@ -123,40 +123,40 @@ public class GracePeriod extends GracePeriodBase {
history.domainRepoId,
history.expirationTime,
history.clientId,
history.billingEventOneTime,
history.billingEventRecurring,
history.billingEvent,
history.billingRecurrence,
history.gracePeriodId);
}
/** Creates a GracePeriod for a Recurring billing event. */
public static GracePeriod createForRecurring(
/** Creates a GracePeriod for a Recurrence billing event. */
public static GracePeriod createForRecurrence(
GracePeriodStatus type,
String domainRepoId,
DateTime expirationTime,
String registrarId,
VKey<Recurring> billingEventRecurring) {
checkArgumentNotNull(billingEventRecurring, "billingEventRecurring cannot be null");
VKey<BillingRecurrence> billingEventRecurrence) {
checkArgumentNotNull(billingEventRecurrence, "billingEventRecurrence cannot be null");
return createInternal(
type, domainRepoId, expirationTime, registrarId, null, billingEventRecurring, null);
type, domainRepoId, expirationTime, registrarId, null, billingEventRecurrence, null);
}
/** Creates a GracePeriod for a Recurring billing event and a given {@link #gracePeriodId}. */
/** Creates a GracePeriod for a Recurrence billing event and a given {@link #gracePeriodId}. */
@VisibleForTesting
public static GracePeriod createForRecurring(
public static GracePeriod createForRecurrence(
GracePeriodStatus type,
String domainRepoId,
DateTime expirationTime,
String registrarId,
VKey<Recurring> billingEventRecurring,
VKey<BillingRecurrence> billingEventRecurrence,
@Nullable Long gracePeriodId) {
checkArgumentNotNull(billingEventRecurring, "billingEventRecurring cannot be null");
checkArgumentNotNull(billingEventRecurrence, "billingEventRecurrence cannot be null");
return createInternal(
type,
domainRepoId,
expirationTime,
registrarId,
null,
billingEventRecurring,
billingEventRecurrence,
gracePeriodId);
}
@@ -168,7 +168,7 @@ public class GracePeriod extends GracePeriodBase {
/** Constructs a GracePeriod of the given type from the provided one-time BillingEvent. */
public static GracePeriod forBillingEvent(
GracePeriodStatus type, String domainRepoId, BillingEvent.OneTime billingEvent) {
GracePeriodStatus type, String domainRepoId, BillingEvent billingEvent) {
return create(
type,
domainRepoId,
@@ -205,8 +205,8 @@ public class GracePeriod extends GracePeriodBase {
instance.domainRepoId = gracePeriod.domainRepoId;
instance.expirationTime = gracePeriod.expirationTime;
instance.clientId = gracePeriod.clientId;
instance.billingEventOneTime = gracePeriod.billingEventOneTime;
instance.billingEventRecurring = gracePeriod.billingEventRecurring;
instance.billingEvent = gracePeriod.billingEvent;
instance.billingRecurrence = gracePeriod.billingRecurrence;
return instance;
}
}
@@ -17,6 +17,7 @@ package google.registry.model.domain;
import google.registry.model.ImmutableObject;
import google.registry.model.UnsafeSerializable;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.persistence.VKey;
import javax.persistence.Access;
@@ -56,21 +57,21 @@ public class GracePeriodBase extends ImmutableObject implements UnsafeSerializab
/**
* The one-time billing event corresponding to the action that triggered this grace period, or
* null if not applicable. Not set for autorenew grace periods (which instead use the field {@code
* billingEventRecurring}) or for redemption grace periods (since deletes have no cost).
* billingEventRecurrence}) or for redemption grace periods (since deletes have no cost).
*/
// NB: Would @IgnoreSave(IfNull.class), but not allowed for @Embed collections.
@Access(AccessType.FIELD)
@Column(name = "billing_event_id")
VKey<BillingEvent.OneTime> billingEventOneTime = null;
VKey<BillingEvent> billingEvent = null;
/**
* The recurring billing event corresponding to the action that triggered this grace period, if
* applicable - i.e. if the action was an autorenew - or null in all other cases.
* The Recurrence corresponding to the action that triggered this grace period, if applicable -
* i.e. if the action was an autorenew - or null in all other cases.
*/
// NB: Would @IgnoreSave(IfNull.class), but not allowed for @Embed collections.
@Access(AccessType.FIELD)
@Column(name = "billing_recurrence_id")
VKey<BillingEvent.Recurring> billingEventRecurring = null;
VKey<BillingRecurrence> billingRecurrence = null;
public long getGracePeriodId() {
return gracePeriodId;
@@ -100,22 +101,22 @@ public class GracePeriodBase extends ImmutableObject implements UnsafeSerializab
/** Returns true if this GracePeriod has an associated BillingEvent; i.e. if it's refundable. */
public boolean hasBillingEvent() {
return billingEventOneTime != null || billingEventRecurring != null;
return billingEvent != null || billingRecurrence != null;
}
/**
* Returns the one time billing event. The value will only be non-null if the type of this grace
* period is not AUTO_RENEW.
*/
public VKey<BillingEvent.OneTime> getOneTimeBillingEvent() {
return billingEventOneTime;
public VKey<BillingEvent> getBillingEvent() {
return billingEvent;
}
/**
* Returns the recurring billing event. The value will only be non-null if the type of this grace
* period is AUTO_RENEW.
* Returns the Recurrence. The value will only be non-null if the type of this grace period is
* AUTO_RENEW.
*/
public VKey<BillingEvent.Recurring> getRecurringBillingEvent() {
return billingEventRecurring;
public VKey<BillingRecurrence> getBillingRecurrence() {
return billingRecurrence;
}
}
@@ -43,7 +43,7 @@ import google.registry.model.Buildable;
import google.registry.model.CacheUtils;
import google.registry.model.CreateAutoTimestamp;
import google.registry.model.UpdateAutoTimestampEntity;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.common.TimedTransitionProperty;
import google.registry.model.domain.fee.FeeQueryCommandExtensionItem.CommandName;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
@@ -17,6 +17,7 @@ package google.registry.model.reporting;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import google.registry.batch.ExpandBillingRecurrencesAction;
import google.registry.model.Buildable;
import google.registry.model.EppResource;
import google.registry.model.ImmutableObject;
@@ -81,7 +82,7 @@ public abstract class HistoryEntry extends ImmutableObject
DOMAIN_ALLOCATE,
/**
* Used for domain registration autorenews explicitly logged by {@link
* google.registry.batch.ExpandRecurringBillingEventsAction}.
* ExpandBillingRecurrencesAction}.
*/
DOMAIN_AUTORENEW,
DOMAIN_CREATE,
@@ -17,7 +17,9 @@ package google.registry.model.transfer;
import static google.registry.util.CollectionUtils.isNullOrEmpty;
import com.google.common.collect.ImmutableSet;
import google.registry.model.billing.BillingCancellation;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Period;
import google.registry.model.domain.Period.Unit;
import google.registry.model.poll.PollMessage;
@@ -69,7 +71,7 @@ public class DomainTransferData extends TransferData {
DateTime transferredRegistrationExpirationTime;
@Column(name = "transfer_billing_cancellation_id")
public VKey<BillingEvent.Cancellation> billingCancellationId;
public VKey<BillingCancellation> billingCancellationId;
/**
* The regular one-time billing event that will be charged for a server-approved transfer.
@@ -78,7 +80,7 @@ public class DomainTransferData extends TransferData {
* being transferred is not a domain.
*/
@Column(name = "transfer_billing_event_id")
VKey<BillingEvent.OneTime> serverApproveBillingEvent;
VKey<BillingEvent> serverApproveBillingEvent;
/**
* The autorenew billing event that should be associated with this resource after the transfer.
@@ -87,7 +89,7 @@ public class DomainTransferData extends TransferData {
* being transferred is not a domain.
*/
@Column(name = "transfer_billing_recurrence_id")
VKey<BillingEvent.Recurring> serverApproveAutorenewEvent;
VKey<BillingRecurrence> serverApproveAutorenewEvent;
/**
* The autorenew poll message that should be associated with this resource after the transfer.
@@ -120,12 +122,12 @@ public class DomainTransferData extends TransferData {
}
@Nullable
public VKey<BillingEvent.OneTime> getServerApproveBillingEvent() {
public VKey<BillingEvent> getServerApproveBillingEvent() {
return serverApproveBillingEvent;
}
@Nullable
public VKey<BillingEvent.Recurring> getServerApproveAutorenewEvent() {
public VKey<BillingRecurrence> getServerApproveAutorenewEvent() {
return serverApproveAutorenewEvent;
}
@@ -176,9 +178,9 @@ public class DomainTransferData extends TransferData {
domainTransferData.billingCancellationId = null;
} else {
domainTransferData.billingCancellationId =
(VKey<BillingEvent.Cancellation>)
(VKey<BillingCancellation>)
serverApproveEntities.stream()
.filter(k -> k.getKind().equals(BillingEvent.Cancellation.class))
.filter(k -> k.getKind().equals(BillingCancellation.class))
.findFirst()
.orElse(null);
}
@@ -204,14 +206,13 @@ public class DomainTransferData extends TransferData {
return this;
}
public Builder setServerApproveBillingEvent(
VKey<BillingEvent.OneTime> serverApproveBillingEvent) {
public Builder setServerApproveBillingEvent(VKey<BillingEvent> serverApproveBillingEvent) {
getInstance().serverApproveBillingEvent = serverApproveBillingEvent;
return this;
}
public Builder setServerApproveAutorenewEvent(
VKey<BillingEvent.Recurring> serverApproveAutorenewEvent) {
VKey<BillingRecurrence> serverApproveAutorenewEvent) {
getInstance().serverApproveAutorenewEvent = serverApproveAutorenewEvent;
return this;
}
@@ -21,7 +21,7 @@ import google.registry.batch.CannedScriptExecutionAction;
import google.registry.batch.DeleteExpiredDomainsAction;
import google.registry.batch.DeleteLoadTestDataAction;
import google.registry.batch.DeleteProberDataAction;
import google.registry.batch.ExpandRecurringBillingEventsAction;
import google.registry.batch.ExpandBillingRecurrencesAction;
import google.registry.batch.RelockDomainAction;
import google.registry.batch.ResaveAllEppResourcesPipelineAction;
import google.registry.batch.ResaveEntityAction;
@@ -32,7 +32,6 @@ import google.registry.cron.CronModule;
import google.registry.cron.TldFanoutAction;
import google.registry.dns.DnsModule;
import google.registry.dns.PublishDnsUpdatesAction;
import google.registry.dns.ReadDnsQueueAction;
import google.registry.dns.ReadDnsRefreshRequestsAction;
import google.registry.dns.RefreshDnsAction;
import google.registry.dns.RefreshDnsOnHostRenameAction;
@@ -115,7 +114,7 @@ interface BackendRequestComponent {
DeleteProberDataAction deleteProberDataAction();
ExpandRecurringBillingEventsAction expandRecurringBillingEventsAction();
ExpandBillingRecurrencesAction expandBillingRecurrencesAction();
ExportDomainListsAction exportDomainListsAction();
@@ -143,8 +142,6 @@ interface BackendRequestComponent {
PublishSpec11ReportAction publishSpec11ReportAction();
ReadDnsQueueAction readDnsQueueAction();
ReadDnsRefreshRequestsAction readDnsRefreshRequestsAction();
RdeReportAction rdeReportAction();
@@ -14,7 +14,7 @@
package google.registry.persistence.converter;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingBase.Flag;
import java.util.Set;
import javax.persistence.AttributeConverter;
import javax.persistence.Converter;
@@ -21,6 +21,7 @@ import com.beust.jcommander.Parameter;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableSet;
import google.registry.tools.params.NameserversParameter;
import google.registry.tools.params.StringListParameter;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
@@ -53,15 +54,15 @@ abstract class CreateOrUpdateDomainCommand extends MutatingEppToolCommand {
String registrant;
@Parameter(
names = {"-a", "--admins"},
description = "Comma-separated list of admin contacts."
)
names = {"-a", "--admins"},
description = "Comma-separated list of admin contacts.",
listConverter = StringListParameter.class)
List<String> admins = new ArrayList<>();
@Parameter(
names = {"-t", "--techs"},
description = "Comma-separated list of technical contacts."
)
names = {"-t", "--techs"},
description = "Comma-separated list of technical contacts.",
listConverter = StringListParameter.class)
List<String> techs = new ArrayList<>();
@Parameter(
@@ -17,6 +17,8 @@ package google.registry.tools;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Predicates.isNull;
import static com.google.common.base.Strings.isNullOrEmpty;
import static com.google.common.base.Verify.verify;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.util.DomainNameUtils.canonicalizeHostname;
import static google.registry.util.RegistrarUtils.normalizeRegistrarName;
@@ -34,6 +36,7 @@ import google.registry.tools.params.OptionalLongParameter;
import google.registry.tools.params.OptionalPhoneNumberParameter;
import google.registry.tools.params.OptionalStringParameter;
import google.registry.tools.params.PathParameter;
import google.registry.tools.params.StringListParameter;
import google.registry.util.CidrAddressBlock;
import java.nio.file.Files;
import java.nio.file.Path;
@@ -69,12 +72,14 @@ abstract class CreateOrUpdateRegistrarCommand extends MutatingCommand {
@Parameter(
names = "--allowed_tlds",
description = "Comma-delimited list of TLDs which the registrar is allowed to use")
description = "Comma-delimited list of TLDs which the registrar is allowed to use",
listConverter = StringListParameter.class)
List<String> allowedTlds = new ArrayList<>();
@Parameter(
names = "--add_allowed_tlds",
description = "Comma-delimited list of TLDs to add to TLDs a registrar is allowed to use")
description = "Comma-delimited list of TLDs to add to TLDs a registrar is allowed to use",
listConverter = StringListParameter.class)
List<String> addAllowedTlds = new ArrayList<>();
@Nullable
@@ -130,6 +135,13 @@ abstract class CreateOrUpdateRegistrarCommand extends MutatingCommand {
validateWith = PathParameter.InputFile.class)
Path clientCertificateFilename;
@Parameter(
names = "--rotate_primary_cert",
description =
"Used together with --cert_file when updating an registrar. "
+ "If set, current cert is saved as failover.")
private Boolean rotatePrimaryCert = Boolean.FALSE;
@Nullable
@Parameter(
names = "--failover_cert_file",
@@ -139,8 +151,9 @@ abstract class CreateOrUpdateRegistrarCommand extends MutatingCommand {
@Parameter(
names = "--ip_allow_list",
description = "Comma-delimited list of IP ranges. An empty string clears the allow list.")
List<String> ipAllowList = new ArrayList<>();
description = "Comma-delimited list of IP ranges. An empty string clears the allow list.",
listConverter = StringListParameter.class)
List<String> ipAllowList;
@Nullable
@Parameter(
@@ -248,7 +261,8 @@ abstract class CreateOrUpdateRegistrarCommand extends MutatingCommand {
description =
"Comma-delimited list of RDAP servers. An empty argument clears the list."
+ " Note that for real registrars this could get overridden periodically by"
+ " ICANN-registered values.")
+ " ICANN-registered values.",
listConverter = StringListParameter.class)
List<String> rdapServers = new ArrayList<>();
/** Returns the existing registrar (for update) or null (for creates). */
@@ -322,16 +336,9 @@ abstract class CreateOrUpdateRegistrarCommand extends MutatingCommand {
}
builder.setAllowedTlds(allowedTldsBuilder.build());
}
if (!ipAllowList.isEmpty()) {
ImmutableList.Builder<CidrAddressBlock> ipAllowListBuilder = new ImmutableList.Builder<>();
if (!(ipAllowList.size() == 1 && ipAllowList.get(0).contains("null"))) {
for (String ipRange : ipAllowList) {
if (!ipRange.isEmpty()) {
ipAllowListBuilder.add(CidrAddressBlock.create(ipRange));
}
}
}
builder.setIpAddressAllowList(ipAllowListBuilder.build());
if (ipAllowList != null) {
builder.setIpAddressAllowList(
ipAllowList.stream().map(CidrAddressBlock::create).collect(toImmutableList()));
}
if (clientCertificateFilename != null) {
String asciiCert = new String(Files.readAllBytes(clientCertificateFilename), US_ASCII);
@@ -341,8 +348,20 @@ abstract class CreateOrUpdateRegistrarCommand extends MutatingCommand {
if (!asciiCert.equals("")) {
certificateChecker.validateCertificate(asciiCert);
}
if (rotatePrimaryCert) {
verify(
oldRegistrar != null,
"--rotate_primary_cert cannot be set by create_registrar command.");
verify(
oldRegistrar.getClientCertificate().isPresent(),
"Primary cert is absent. Rotation may remove a failover certificate still in use.");
builder.setFailoverClientCertificate(oldRegistrar.getClientCertificate().get(), now);
}
builder.setClientCertificate(asciiCert, now);
}
if (rotatePrimaryCert && clientCertificateFilename == null) {
throw new IllegalArgumentException("--rotate_primary_cert must be used with --cert_file.");
}
if (failoverClientCertificateFilename != null) {
String asciiCert =
@@ -20,6 +20,7 @@ import google.registry.model.tld.label.ReservedList;
import google.registry.model.tld.label.ReservedListDao;
import google.registry.tools.params.PathParameter;
import java.nio.file.Path;
import java.util.stream.Collectors;
import javax.annotation.Nullable;
/**
@@ -69,4 +70,12 @@ public abstract class CreateOrUpdateReservedListCommand extends ConfirmingComman
}
return message;
}
String outputReservedListEntries(ReservedList rl) {
return "["
+ rl.getReservedListEntries().values().stream()
.map(rle -> String.format("(%s)", rle.toString()))
.collect(Collectors.joining(", "))
+ "]";
}
}
@@ -18,7 +18,6 @@ import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.tools.UpdateOrDeleteAllocationTokensCommand.getTokenKeys;
import static google.registry.util.CollectionUtils.findDuplicates;
import static google.registry.util.CollectionUtils.isNullOrEmpty;
import static google.registry.util.DomainNameUtils.canonicalizeHostname;
import com.beust.jcommander.Parameter;
@@ -37,6 +36,7 @@ import google.registry.model.tld.label.PremiumList;
import google.registry.model.tld.label.PremiumListDao;
import google.registry.tldconfig.idn.IdnTableEnum;
import google.registry.tools.params.OptionalStringParameter;
import google.registry.tools.params.StringListParameter;
import google.registry.tools.params.TransitionListParameter.BillingCostTransitions;
import google.registry.tools.params.TransitionListParameter.TldStateTransitions;
import java.util.Arrays;
@@ -195,19 +195,22 @@ abstract class CreateOrUpdateTldCommand extends MutatingCommand {
@Nullable
@Parameter(
names = "--reserved_lists",
description = "A comma-separated list of reserved list names to be applied to the TLD")
description = "A comma-separated list of reserved list names to be applied to the TLD",
listConverter = StringListParameter.class)
List<String> reservedListNames;
@Nullable
@Parameter(
names = "--allowed_registrants",
description = "A comma-separated list of allowed registrants for the TLD")
description = "A comma-separated list of allowed registrants for the TLD",
listConverter = StringListParameter.class)
List<String> allowedRegistrants;
@Nullable
@Parameter(
names = "--allowed_nameservers",
description = "A comma-separated list of allowed nameservers for the TLD")
description = "A comma-separated list of allowed nameservers for the TLD",
listConverter = StringListParameter.class)
List<String> allowedNameservers;
@Parameter(
@@ -223,8 +226,9 @@ abstract class CreateOrUpdateTldCommand extends MutatingCommand {
@Nullable
@Parameter(
names = "--dns_writers",
description = "A comma-separated list of DnsWriter implementations to use")
names = "--dns_writers",
description = "A comma-separated list of DnsWriter implementations to use",
listConverter = StringListParameter.class)
List<String> dnsWriters;
@Nullable
@@ -262,7 +266,8 @@ abstract class CreateOrUpdateTldCommand extends MutatingCommand {
"A comma-separated list of default allocation tokens to be applied to the TLD. The"
+ " ordering of this list will determine which token is used in the case where"
+ " multiple tokens are valid for a registration. Use an empty string to clear all"
+ " present default tokens.")
+ " present default tokens.",
listConverter = StringListParameter.class)
List<String> defaultTokens;
@Nullable
@@ -271,7 +276,8 @@ abstract class CreateOrUpdateTldCommand extends MutatingCommand {
description =
"A comma-separated list of the IDN tables to use for this TLD. Specify an empty list to"
+ " remove any previously-set tables and to use the default. All elements must be"
+ " IdnTableEnum values")
+ " IdnTableEnum values",
listConverter = StringListParameter.class)
List<String> idnTables;
/** Returns the existing tld (for update) or null (for creates). */
@@ -420,12 +426,8 @@ abstract class CreateOrUpdateTldCommand extends MutatingCommand {
builder.setAllowedFullyQualifiedHostNames(getAllowedNameservers(oldTld));
if (!isNullOrEmpty(defaultTokens)) {
if (defaultTokens.equals(ImmutableList.of(""))) {
builder.setDefaultPromoTokens(ImmutableList.of());
} else {
builder.setDefaultPromoTokens(getTokenKeys(defaultTokens, null));
}
if (defaultTokens != null) {
builder.setDefaultPromoTokens(getTokenKeys(defaultTokens, null));
}
if (idnTables != null) {
if (idnTables.equals(ImmutableList.of(""))) {
@@ -28,7 +28,6 @@ import com.google.common.base.Strings;
import google.registry.model.tld.label.ReservedList;
import java.nio.file.Files;
import java.util.List;
import java.util.stream.Collectors;
import org.joda.time.DateTime;
/** Command to create a {@link ReservedList}. */
@@ -64,11 +63,8 @@ final class CreateReservedListCommand extends CreateOrUpdateReservedListCommand
.setCreationTimestamp(now)
.build();
String entries =
reservedList.getReservedListEntries().entrySet().stream()
.map(entry -> String.format("%s=%s", entry.getKey(), entry.getValue()))
.collect(Collectors.joining(", "));
return String.format("%s\nreservedListMap={%s}\n", reservedList, entries);
return String.format(
"%s\nreservedListMap=%s\n", reservedList, outputReservedListEntries(reservedList));
}
private static void validateListName(String name) {
@@ -26,8 +26,8 @@ import com.google.common.collect.Sets;
import google.registry.batch.CloudTasksUtils;
import google.registry.batch.RelockDomainAction;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.RegistryLock;
@@ -397,8 +397,8 @@ public final class DomainLockUtils {
tm().update(domain);
tm().insert(domainHistory);
if (!lock.isSuperuser()) { // admin actions shouldn't affect billing
BillingEvent.OneTime oneTime =
new BillingEvent.OneTime.Builder()
BillingEvent billingEvent =
new BillingEvent.Builder()
.setReason(Reason.SERVER_STATUS)
.setTargetId(domain.getForeignKey())
.setRegistrarId(domain.getCurrentSponsorRegistrarId())
@@ -407,7 +407,7 @@ public final class DomainLockUtils {
.setBillingTime(now)
.setDomainHistory(domainHistory)
.build();
tm().insert(oneTime);
tm().insert(billingEvent);
}
}
}
@@ -17,7 +17,7 @@ package google.registry.tools;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.collect.Sets.difference;
import static google.registry.model.billing.BillingEvent.RenewalPriceBehavior.DEFAULT;
import static google.registry.model.billing.BillingBase.RenewalPriceBehavior.DEFAULT;
import static google.registry.model.domain.token.AllocationToken.TokenType.DEFAULT_PROMO;
import static google.registry.model.domain.token.AllocationToken.TokenType.PACKAGE;
import static google.registry.model.domain.token.AllocationToken.TokenType.SINGLE_USE;
@@ -39,7 +39,7 @@ import com.google.common.collect.ImmutableSortedMap;
import com.google.common.collect.Iterables;
import com.google.common.collect.Streams;
import com.google.common.io.Files;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.domain.fee.FeeQueryCommandExtensionItem.CommandName;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.domain.token.AllocationToken.RegistrationBehavior;
@@ -32,6 +32,7 @@ import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.tools.params.OptionalPhoneNumberParameter;
import google.registry.tools.params.PathParameter;
import google.registry.tools.params.StringListParameter;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
@@ -72,8 +73,10 @@ final class RegistrarPocCommand extends MutatingCommand {
@Nullable
@Parameter(
names = "--contact_type",
description = "Type of communications for this contact; separate multiple with commas."
+ " Allowed values are ABUSE, ADMIN, BILLING, LEGAL, MARKETING, TECH, WHOIS.")
description =
"Type of communications for this contact; separate multiple with commas."
+ " Allowed values are ABUSE, ADMIN, BILLING, LEGAL, MARKETING, TECH, WHOIS.",
listConverter = StringListParameter.class)
private List<String> contactTypeNames;
@Nullable
@@ -177,15 +180,6 @@ final class RegistrarPocCommand extends MutatingCommand {
// If the contact_type parameter is not specified, we should not make any changes.
if (contactTypeNames == null) {
contactTypes = null;
// It appears that when the user specifies "--contact_type=" with no types following,
// JCommander sets contactTypeNames to a one-element list containing the empty string. This is
// strange, but we need to handle this by setting the contact types to the empty set. Also do
// this if contactTypeNames is empty, which is what I would hope JCommander would return in
// some future, better world.
} else //noinspection UnnecessaryParentheses
if (contactTypeNames.isEmpty()
|| (contactTypeNames.size() == 1 && contactTypeNames.get(0).isEmpty())) {
contactTypes = ImmutableSet.of();
} else {
contactTypes =
contactTypeNames.stream()
@@ -15,7 +15,7 @@
package google.registry.tools;
import com.google.common.collect.ImmutableMap;
import google.registry.tools.javascrap.CreateCancellationsForOneTimesCommand;
import google.registry.tools.javascrap.CreateCancellationsForBillingEventsCommand;
/** Container class to create and run remote commands against a server instance. */
public final class RegistryTool {
@@ -35,7 +35,9 @@ public final class RegistryTool {
.put("convert_idn", ConvertIdnCommand.class)
.put("count_domains", CountDomainsCommand.class)
.put("create_anchor_tenant", CreateAnchorTenantCommand.class)
.put("create_cancellations_for_one_times", CreateCancellationsForOneTimesCommand.class)
.put(
"create_cancellations_for_billing_events",
CreateCancellationsForBillingEventsCommand.class)
.put("create_cdns_tld", CreateCdnsTld.class)
.put("create_contact", CreateContactCommand.class)
.put("create_domain", CreateDomainCommand.class)
@@ -40,7 +40,6 @@ import google.registry.request.Modules.UrlConnectionServiceModule;
import google.registry.request.Modules.UrlFetchServiceModule;
import google.registry.request.Modules.UserServiceModule;
import google.registry.tools.AuthModule.LocalCredentialModule;
import google.registry.tools.javascrap.CreateCancellationsForOneTimesCommand;
import google.registry.util.UtilsModule;
import google.registry.whois.NonCachingWhoisModule;
import javax.annotation.Nullable;
@@ -92,8 +91,6 @@ interface RegistryToolComponent {
void inject(CreateAnchorTenantCommand command);
void inject(CreateCancellationsForOneTimesCommand command);
void inject(CreateCdnsTld command);
void inject(CreateContactCommand command);
@@ -24,7 +24,7 @@ import com.google.api.client.util.GenericData;
import com.google.auth.oauth2.UserCredentials;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule.DefaultCredential;
import google.registry.config.CredentialModule.ApplicationDefaultCredential;
import google.registry.config.RegistryConfig;
import google.registry.config.RegistryConfig.Config;
import google.registry.util.GoogleCredentialsBundle;
@@ -56,7 +56,7 @@ class RequestFactoryModule {
@Provides
static HttpRequestFactory provideHttpRequestFactory(
@DefaultCredential GoogleCredentialsBundle credentialsBundle,
@ApplicationDefaultCredential GoogleCredentialsBundle credentialsBundle,
@Config("iapClientId") Optional<String> iapClientId) {
if (RegistryConfig.areServersLocal()) {
return new NetHttpTransport()
@@ -32,8 +32,7 @@ import com.google.common.collect.ImmutableMultimap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Sets;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.Period;
@@ -205,7 +204,7 @@ class UnrenewDomainCommand extends ConfirmingCommand {
.setEventTime(now)
.build();
// Create a new autorenew billing event and poll message starting at the new expiration time.
BillingEvent.Recurring newAutorenewEvent =
BillingRecurrence newAutorenewEvent =
newAutorenewBillingEvent(domain)
.setEventTime(newExpirationTime)
.setDomainHistory(domainHistory)
@@ -216,9 +215,9 @@ class UnrenewDomainCommand extends ConfirmingCommand {
.setHistoryEntry(domainHistory)
.build();
// End the old autorenew billing event and poll message now.
Recurring existingRecurring = tm().loadByKey(domain.getAutorenewBillingEvent());
BillingRecurrence existingBillingRecurrence = tm().loadByKey(domain.getAutorenewBillingEvent());
updateAutorenewRecurrenceEndTime(
domain, existingRecurring, now, domainHistory.getHistoryEntryId());
domain, existingBillingRecurrence, now, domainHistory.getHistoryEntryId());
Domain newDomain =
domain
.asBuilder()
@@ -28,12 +28,13 @@ import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedMap;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.domain.fee.FeeQueryCommandExtensionItem.CommandName;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.domain.token.AllocationToken.RegistrationBehavior;
import google.registry.model.domain.token.AllocationToken.TokenStatus;
import google.registry.model.domain.token.AllocationToken.TokenType;
import google.registry.tools.params.StringListParameter;
import google.registry.tools.params.TransitionListParameter.TokenStatusTransitions;
import java.util.List;
import java.util.Map;
@@ -54,21 +55,24 @@ final class UpdateAllocationTokensCommand extends UpdateOrDeleteAllocationTokens
names = {"--allowed_client_ids"},
description =
"Comma-separated list of allowed client IDs. Use the empty string to clear the "
+ "existing list.")
+ "existing list.",
listConverter = StringListParameter.class)
private List<String> allowedClientIds;
@Parameter(
names = {"--allowed_tlds"},
description =
"Comma-separated list of allowed TLDs. Use the empty string to clear the "
+ "existing list.")
+ "existing list.",
listConverter = StringListParameter.class)
private List<String> allowedTlds;
@Parameter(
names = {"--allowed_epp_actions"},
description =
"Comma-separated list of allowed EPP actions. Use an empty string to clear the existing"
+ " list.")
+ " list.",
listConverter = StringListParameter.class)
private List<String> allowedEppActions;
@Parameter(
@@ -128,18 +132,6 @@ final class UpdateAllocationTokensCommand extends UpdateOrDeleteAllocationTokens
@Override
public void init() {
// A single entry with the empty string means that the user passed an empty argument to the
// lists, so we should clear them
if (ImmutableList.of("").equals(allowedClientIds)) {
allowedClientIds = ImmutableList.of();
}
if (ImmutableList.of("").equals(allowedTlds)) {
allowedTlds = ImmutableList.of();
}
if (ImmutableList.of("").equals(allowedEppActions)) {
allowedEppActions = ImmutableList.of();
}
if (tokenStatusTransitions != null
&& (tokenStatusTransitions.containsValue(TokenStatus.ENDED)
|| tokenStatusTransitions.containsValue(TokenStatus.CANCELLED))) {
@@ -25,8 +25,8 @@ import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import google.registry.model.EppResourceUtils;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.reporting.HistoryEntry;
@@ -38,7 +38,7 @@ import org.joda.money.Money;
import org.joda.time.DateTime;
/**
* Command to update {@link Recurring} billing events with a new behavior and/or price.
* Command to update {@link BillingRecurrence} billing events with a new behavior and/or price.
*
* <p>More specifically, this closes the existing recurrence object and creates a new, similar,
* object as well as a corresponding synthetic {@link DomainHistory} object. This is done to
@@ -85,11 +85,11 @@ public class UpdateRecurrenceCommand extends ConfirmingCommand {
+ " SPECIFIED");
}
}
ImmutableMap<Domain, Recurring> domainsAndRecurrings =
tm().transact(this::loadDomainsAndRecurrings);
ImmutableMap<Domain, BillingRecurrence> domainsAndRecurrences =
tm().transact(this::loadDomainsAndRecurrences);
if (renewalPriceBehavior == null) {
// Allow users to specify only a price only if all renewals are already SPECIFIED
domainsAndRecurrings.forEach(
domainsAndRecurrences.forEach(
(d, r) ->
checkArgument(
r.getRenewalPriceBehavior().equals(RenewalPriceBehavior.SPECIFIED),
@@ -103,41 +103,45 @@ public class UpdateRecurrenceCommand extends ConfirmingCommand {
"Update the following with behavior %s%s?\n%s",
renewalPriceBehavior,
specifiedPriceString,
Joiner.on('\n').withKeyValueSeparator(':').join(domainsAndRecurrings));
Joiner.on('\n').withKeyValueSeparator(':').join(domainsAndRecurrences));
}
@Override
protected String execute() throws Exception {
ImmutableList<Recurring> newRecurrings = tm().transact(this::internalExecute);
return "Updated new recurring(s): " + newRecurrings;
ImmutableList<BillingRecurrence> newBillingRecurrences = tm().transact(this::internalExecute);
return "Updated new recurrence(s): " + newBillingRecurrences;
}
private ImmutableList<Recurring> internalExecute() {
ImmutableMap<Domain, Recurring> domainsAndRecurrings = loadDomainsAndRecurrings();
private ImmutableList<BillingRecurrence> internalExecute() {
ImmutableMap<Domain, BillingRecurrence> domainsAndRecurrences = loadDomainsAndRecurrences();
DateTime now = tm().getTransactionTime();
ImmutableList.Builder<Recurring> resultBuilder = new ImmutableList.Builder<>();
domainsAndRecurrings.forEach(
(domain, existingRecurring) -> {
// Make a new history ID to break the (recurring, history, domain) circular dep chain
ImmutableList.Builder<BillingRecurrence> resultBuilder = new ImmutableList.Builder<>();
domainsAndRecurrences.forEach(
(domain, existingRecurrence) -> {
// Make a new history ID to break the (recurrence, history, domain) circular dep chain
long newHistoryId = allocateId();
HistoryEntryId newDomainHistoryId = new HistoryEntryId(domain.getRepoId(), newHistoryId);
Recurring endingNow = existingRecurring.asBuilder().setRecurrenceEndTime(now).build();
Recurring.Builder newRecurringBuilder =
existingRecurring
BillingRecurrence endingNow =
existingRecurrence.asBuilder().setRecurrenceEndTime(now).build();
BillingRecurrence.Builder newRecurrenceBuilder =
existingRecurrence
.asBuilder()
// set the ID to be 0 (null) to create a new object
.setId(0)
.setDomainHistoryId(newDomainHistoryId);
if (renewalPriceBehavior != null) {
newRecurringBuilder.setRenewalPriceBehavior(renewalPriceBehavior);
newRecurringBuilder.setRenewalPrice(null);
newRecurrenceBuilder.setRenewalPriceBehavior(renewalPriceBehavior);
newRecurrenceBuilder.setRenewalPrice(null);
}
if (specifiedRenewalPrice != null) {
newRecurringBuilder.setRenewalPrice(specifiedRenewalPrice);
newRecurrenceBuilder.setRenewalPrice(specifiedRenewalPrice);
}
Recurring newRecurring = newRecurringBuilder.build();
BillingRecurrence newBillingRecurrence = newRecurrenceBuilder.build();
Domain newDomain =
domain.asBuilder().setAutorenewBillingEvent(newRecurring.createVKey()).build();
domain
.asBuilder()
.setAutorenewBillingEvent(newBillingRecurrence.createVKey())
.build();
DomainHistory newDomainHistory =
new DomainHistory.Builder()
.setRevisionId(newDomainHistoryId.getRevisionId())
@@ -149,14 +153,14 @@ public class UpdateRecurrenceCommand extends ConfirmingCommand {
.setType(HistoryEntry.Type.SYNTHETIC)
.setModificationTime(now)
.build();
tm().putAll(endingNow, newRecurring, newDomain, newDomainHistory);
resultBuilder.add(newRecurring);
tm().putAll(endingNow, newBillingRecurrence, newDomain, newDomainHistory);
resultBuilder.add(newBillingRecurrence);
});
return resultBuilder.build();
}
private ImmutableMap<Domain, Recurring> loadDomainsAndRecurrings() {
ImmutableMap.Builder<Domain, Recurring> result = new ImmutableMap.Builder<>();
private ImmutableMap<Domain, BillingRecurrence> loadDomainsAndRecurrences() {
ImmutableMap.Builder<Domain, BillingRecurrence> result = new ImmutableMap.Builder<>();
DateTime now = tm().getTransactionTime();
for (String domainName : mainParameters) {
Domain domain =
@@ -183,12 +187,12 @@ public class UpdateRecurrenceCommand extends ConfirmingCommand {
"Domain %s autorenew ended prior to now at %s",
domainName,
endTime));
Recurring recurring = tm().loadByKey(domain.getAutorenewBillingEvent());
BillingRecurrence billingRecurrence = tm().loadByKey(domain.getAutorenewBillingEvent());
checkArgument(
recurring.getRecurrenceEndTime().equals(END_OF_TIME),
billingRecurrence.getRecurrenceEndTime().equals(END_OF_TIME),
"Domain %s's recurrence's end date is not END_OF_TIME",
domainName);
result.put(domain, recurring);
result.put(domain, billingRecurrence);
}
return result.build();
}
@@ -22,7 +22,6 @@ import com.google.common.base.Strings;
import google.registry.model.tld.label.ReservedList;
import java.nio.file.Files;
import java.util.List;
import java.util.stream.Collectors;
/** Command to safely update {@link ReservedList}. */
@Parameters(separators = " =", commandDescription = "Update a ReservedList.")
@@ -52,17 +51,11 @@ final class UpdateReservedListCommand extends CreateOrUpdateReservedListCommand
if (!existingReservedList
.getReservedListEntries()
.equals(reservedList.getReservedListEntries())) {
String oldEntries =
existingReservedList.getReservedListEntries().entrySet().stream()
.map(entry -> String.format("%s=%s", entry.getKey(), entry.getValue()))
.collect(Collectors.joining(", "));
String newEntries =
reservedList.getReservedListEntries().entrySet().stream()
.map(entry -> String.format("%s=%s", entry.getKey(), entry.getValue()))
.collect(Collectors.joining(", "));
return String.format(
"Update reserved list for %s?\nOld List: %s\n New List: %s",
name, oldEntries, newEntries);
"Update reserved list for %s?\nOld list: %s\n New list: %s",
name,
outputReservedListEntries(existingReservedList),
outputReservedListEntries(reservedList));
}
return "No entity changes to apply.";
}
@@ -28,6 +28,7 @@ import com.google.common.collect.Maps;
import google.registry.config.RegistryEnvironment;
import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldState;
import google.registry.tools.params.StringListParameter;
import java.util.List;
import java.util.Map;
import java.util.Optional;
@@ -39,40 +40,47 @@ import org.joda.time.DateTimeZone;
/** Command to update a TLD. */
@Parameters(separators = " =", commandDescription = "Update existing TLD(s)")
public class UpdateTldCommand extends CreateOrUpdateTldCommand {
@Nullable
@Parameter(
names = "--add_reserved_lists",
description = "A comma-separated list of reserved list names to be added to the TLD")
description = "A comma-separated list of reserved list names to be added to the TLD",
listConverter = StringListParameter.class)
List<String> reservedListsAdd;
@Nullable
@Parameter(
names = "--remove_reserved_lists",
description = "A comma-separated list of reserved list names to be removed from the TLD")
description = "A comma-separated list of reserved list names to be removed from the TLD",
listConverter = StringListParameter.class)
List<String> reservedListsRemove;
@Nullable
@Parameter(
names = "--add_allowed_registrants",
description = "A comma-separated list of allowed registrants to be added to the TLD")
description = "A comma-separated list of allowed registrants to be added to the TLD",
listConverter = StringListParameter.class)
List<String> allowedRegistrantsAdd;
@Nullable
@Parameter(
names = "--remove_allowed_registrants",
description = "A comma-separated list of allowed registrants to be removed from the TLD")
description = "A comma-separated list of allowed registrants to be removed from the TLD",
listConverter = StringListParameter.class)
List<String> allowedRegistrantsRemove;
@Nullable
@Parameter(
names = "--add_allowed_nameservers",
description = "A comma-separated list of allowed nameservers to be added to the TLD")
description = "A comma-separated list of allowed nameservers to be added to the TLD",
listConverter = StringListParameter.class)
List<String> allowedNameserversAdd;
@Nullable
@Parameter(
names = "--remove_allowed_nameservers",
description = "A comma-separated list of allowed nameservers to be removed from the TLD")
description = "A comma-separated list of allowed nameservers to be removed from the TLD",
listConverter = StringListParameter.class)
List<String> allowedNameserversRemove;
@Nullable
@@ -19,9 +19,9 @@ import static google.registry.persistence.transaction.TransactionManagerFactory.
import com.beust.jcommander.Parameter;
import com.beust.jcommander.Parameters;
import com.google.common.collect.ImmutableSet;
import google.registry.model.billing.BillingBase;
import google.registry.model.billing.BillingCancellation;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Cancellation;
import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.persistence.VKey;
import google.registry.persistence.transaction.QueryComposer.Comparator;
import google.registry.tools.ConfirmingCommand;
@@ -29,14 +29,16 @@ import google.registry.tools.params.LongParameter;
import java.util.List;
/**
* Command to create {@link Cancellation}s for specified {@link OneTime} billing events.
* Command to create {@link BillingCancellation}s for specified {@link BillingEvent} billing events.
*
* <p>This can be used to fix situations where we've inadvertently billed registrars. It's generally
* easier and better to issue cancellations within the Nomulus system than to attempt to issue
* refunds after the fact.
*/
@Parameters(separators = " =", commandDescription = "Manually create Cancellations for OneTimes.")
public class CreateCancellationsForOneTimesCommand extends ConfirmingCommand {
@Parameters(
separators = " =",
commandDescription = "Manually create Cancellations for BillingEvents.")
public class CreateCancellationsForBillingEventsCommand extends ConfirmingCommand {
@Parameter(
description = "Space-delimited billing event ID(s) to cancel",
@@ -44,80 +46,83 @@ public class CreateCancellationsForOneTimesCommand extends ConfirmingCommand {
validateWith = LongParameter.class)
private List<Long> mainParameters;
private ImmutableSet<OneTime> oneTimesToCancel;
private ImmutableSet<BillingEvent> billingEventsToCancel;
@Override
protected void init() {
ImmutableSet.Builder<Long> missingIdsBuilder = new ImmutableSet.Builder<>();
ImmutableSet.Builder<Long> alreadyCancelledIdsBuilder = new ImmutableSet.Builder<>();
ImmutableSet.Builder<OneTime> oneTimesBuilder = new ImmutableSet.Builder<>();
ImmutableSet.Builder<BillingEvent> billingEventsBuilder = new ImmutableSet.Builder<>();
tm().transact(
() -> {
for (Long billingEventId : ImmutableSet.copyOf(mainParameters)) {
VKey<OneTime> key = VKey.create(OneTime.class, billingEventId);
VKey<BillingEvent> key = VKey.create(BillingEvent.class, billingEventId);
if (tm().exists(key)) {
OneTime oneTime = tm().loadByKey(key);
if (alreadyCancelled(oneTime)) {
BillingEvent billingEvent = tm().loadByKey(key);
if (alreadyCancelled(billingEvent)) {
alreadyCancelledIdsBuilder.add(billingEventId);
} else {
oneTimesBuilder.add(oneTime);
billingEventsBuilder.add(billingEvent);
}
} else {
missingIdsBuilder.add(billingEventId);
}
}
});
oneTimesToCancel = oneTimesBuilder.build();
System.out.printf("Found %d OneTime(s) to cancel\n", oneTimesToCancel.size());
billingEventsToCancel = billingEventsBuilder.build();
System.out.printf("Found %d BillingEvent(s) to cancel\n", billingEventsToCancel.size());
ImmutableSet<Long> missingIds = missingIdsBuilder.build();
if (!missingIds.isEmpty()) {
System.out.printf("Missing OneTime(s) for IDs %s\n", missingIds);
System.out.printf("Missing BillingEvent(s) for IDs %s\n", missingIds);
}
ImmutableSet<Long> alreadyCancelledIds = alreadyCancelledIdsBuilder.build();
if (!alreadyCancelledIds.isEmpty()) {
System.out.printf(
"The following OneTime IDs were already cancelled: %s\n", alreadyCancelledIds);
"The following BillingEvent IDs were already cancelled: %s\n", alreadyCancelledIds);
}
}
@Override
protected String prompt() {
return String.format("Create cancellations for %d OneTime(s)?", oneTimesToCancel.size());
return String.format(
"Create cancellations for %d BillingEvent(s)?", billingEventsToCancel.size());
}
@Override
protected String execute() throws Exception {
int cancelledOneTimes = 0;
for (OneTime oneTime : oneTimesToCancel) {
cancelledOneTimes +=
int cancelledBillingEvents = 0;
for (BillingEvent billingEvent : billingEventsToCancel) {
cancelledBillingEvents +=
tm().transact(
() -> {
if (alreadyCancelled(oneTime)) {
if (alreadyCancelled(billingEvent)) {
System.out.printf(
"OneTime %d already cancelled, this is unexpected.\n", oneTime.getId());
"BillingEvent %d already cancelled, this is unexpected.\n",
billingEvent.getId());
return 0;
}
tm().put(
new Cancellation.Builder()
.setOneTimeEventKey(oneTime.createVKey())
.setBillingTime(oneTime.getBillingTime())
.setDomainHistoryId(oneTime.getHistoryEntryId())
.setRegistrarId(oneTime.getRegistrarId())
.setEventTime(oneTime.getEventTime())
.setReason(BillingEvent.Reason.ERROR)
.setTargetId(oneTime.getTargetId())
new BillingCancellation.Builder()
.setBillingEvent(billingEvent.createVKey())
.setBillingTime(billingEvent.getBillingTime())
.setDomainHistoryId(billingEvent.getHistoryEntryId())
.setRegistrarId(billingEvent.getRegistrarId())
.setEventTime(billingEvent.getEventTime())
.setReason(BillingBase.Reason.ERROR)
.setTargetId(billingEvent.getTargetId())
.build());
System.out.printf(
"Added Cancellation for OneTime with ID %d\n", oneTime.getId());
"Added BillingCancellation for BillingEvent with ID %d\n",
billingEvent.getId());
return 1;
});
}
return String.format("Created %d Cancellation event(s)", cancelledOneTimes);
return String.format("Created %d BillingCancellation event(s)", cancelledBillingEvents);
}
private boolean alreadyCancelled(OneTime oneTime) {
return tm().createQueryComposer(Cancellation.class)
.where("refOneTime", Comparator.EQ, oneTime.getId())
private boolean alreadyCancelled(BillingEvent billingEvent) {
return tm().createQueryComposer(BillingCancellation.class)
.where("billingEvent", Comparator.EQ, billingEvent.getId())
.first()
.isPresent();
}
@@ -0,0 +1,37 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.tools.params;
import com.google.common.base.Splitter;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableList;
import java.util.List;
/**
* Converter for lists of String params that omits any empty strings.
*
* <p>JCommander automatically parses a comma-separated list well enough, but it parses the empty
* list, e.g. "--foo=" as a list consisting solely of the empty string, which is not what we want.
*/
public class StringListParameter extends ParameterConverterValidator<List<String>> {
@Override
public List<String> convert(String value) {
if (Strings.isNullOrEmpty(value)) {
return ImmutableList.of();
}
return Splitter.on(',').trimResults().omitEmptyStrings().splitToList(value);
}
}
@@ -15,13 +15,13 @@
package google.registry.tools.server;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.model.tld.Registries.assertTldsExist;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.RequestParameters.PARAM_TLDS;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import google.registry.dns.DnsUtils;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.Response;
@@ -43,10 +43,10 @@ import org.joda.time.Duration;
* run internally, or by pretending to be internal by setting the X-AppEngine-QueueName header,
* which only admin users can do.
*
* <p>You must pass in a number of <code>smearMinutes</code> as a URL parameter so that the DNS
* queue doesn't get overloaded. A rough rule of thumb for Cloud DNS is 1 minute per every 1,000
* domains. This smears the updates out over the next N minutes. For small TLDs consisting of fewer
* than 1,000 domains, passing in 1 is fine (which will execute all the updates immediately).
* <p>You must pass in a number of {@code smearMinutes} as a URL parameter so that the DNS queue
* doesn't get overloaded. A rough rule of thumb for Cloud DNS is 1 minute per every 1,000 domains.
* This smears the updates out over the next N minutes. For small TLDs consisting of fewer than
* 1,000 domains, passing in 1 is fine (which will execute all the updates immediately).
*/
@Action(
service = Action.Service.TOOLS,
@@ -66,7 +66,6 @@ public class RefreshDnsForAllDomainsAction implements Runnable {
@Parameter("smearMinutes")
int smearMinutes;
@Inject DnsUtils dnsUtils;
@Inject Clock clock;
@Inject Random random;
@@ -91,7 +90,7 @@ public class RefreshDnsForAllDomainsAction implements Runnable {
domainName -> {
try {
// Smear the task execution time over the next N minutes.
dnsUtils.requestDomainDnsRefresh(
requestDomainDnsRefresh(
domainName, Duration.standardMinutes(random.nextInt(smearMinutes)));
} catch (Throwable t) {
logger.atSevere().withCause(t).log(
@@ -38,9 +38,9 @@
<mapping-file>META-INF/orm.xml</mapping-file>
<class>google.registry.model.billing.BillingEvent$Cancellation</class>
<class>google.registry.model.billing.BillingEvent$OneTime</class>
<class>google.registry.model.billing.BillingEvent$Recurring</class>
<class>google.registry.model.billing.BillingCancellation</class>
<class>google.registry.model.billing.BillingEvent</class>
<class>google.registry.model.billing.BillingRecurrence</class>
<class>google.registry.model.common.Cursor</class>
<class>google.registry.model.common.DatabaseMigrationStateSchedule</class>
<class>google.registry.model.common.DnsRefreshRequest</class>
@@ -105,9 +105,9 @@
<class>google.registry.persistence.converter.TransferServerApproveEntitySetConverter</class>
<!-- Generated converters for VKey -->
<class>google.registry.model.billing.VKeyConverter_Cancellation</class>
<class>google.registry.model.billing.VKeyConverter_OneTime</class>
<class>google.registry.model.billing.VKeyConverter_Recurring</class>
<class>google.registry.model.billing.VKeyConverter_BillingCancellation</class>
<class>google.registry.model.billing.VKeyConverter_BillingEvent</class>
<class>google.registry.model.billing.VKeyConverter_BillingRecurrence</class>
<class>google.registry.model.contact.VKeyConverter_Contact</class>
<class>google.registry.model.domain.VKeyConverter_Domain</class>
<class>google.registry.model.domain.token.VKeyConverter_AllocationToken</class>
@@ -29,8 +29,8 @@ SELECT b, r FROM BillingEvent b
JOIN Registrar r ON b.clientId = r.registrarId
JOIN Domain d ON b.domainRepoId = d.repoId
JOIN Tld t ON t.tldStr = d.tld
LEFT JOIN BillingCancellation c ON b.id = c.refOneTime
LEFT JOIN BillingCancellation cr ON b.cancellationMatchingBillingEvent = cr.refRecurring
LEFT JOIN BillingCancellation c ON b.id = c.billingEvent
LEFT JOIN BillingCancellation cr ON b.cancellationMatchingBillingEvent = cr.billingRecurrence
WHERE r.billingAccountMap IS NOT NULL
AND r.type = 'REAL'
AND t.invoicingEnabled IS TRUE
@@ -28,7 +28,7 @@ import static org.mockito.Mockito.verifyNoInteractions;
import com.google.common.collect.ImmutableList;
import com.google.common.testing.TestLogHandler;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.contact.Contact;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.domain.token.AllocationToken.TokenType;
@@ -29,9 +29,9 @@ import com.google.common.collect.ImmutableSet;
import google.registry.flows.DaggerEppTestComponent;
import google.registry.flows.EppController;
import google.registry.flows.EppTestComponent.FakesAndMocksModule;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingBase.Flag;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.poll.PollMessage;
@@ -43,7 +43,6 @@ import google.registry.testing.DatabaseHelper;
import google.registry.testing.FakeClock;
import google.registry.testing.FakeLockHandler;
import google.registry.testing.FakeResponse;
import google.registry.testing.TaskQueueExtension;
import java.util.Optional;
import org.joda.time.DateTime;
import org.junit.jupiter.api.BeforeEach;
@@ -59,8 +58,6 @@ class DeleteExpiredDomainsActionTest {
final JpaIntegrationTestExtension jpa =
new JpaTestExtensions.Builder().withClock(clock).buildIntegrationTestExtension();
@RegisterExtension final TaskQueueExtension taskQueue = new TaskQueueExtension();
private final FakeResponse response = new FakeResponse();
private DeleteExpiredDomainsAction action;
@@ -173,7 +170,7 @@ class DeleteExpiredDomainsActionTest {
.setModificationTime(clock.nowUtc().minusMonths(9))
.setRegistrarId(pendingExpirationDomain.getCreationRegistrarId())
.build());
BillingEvent.Recurring autorenewBillingEvent =
BillingRecurrence autorenewBillingEvent =
persistResource(createAutorenewBillingEvent(createHistoryEntry).build());
PollMessage.Autorenew autorenewPollMessage =
persistResource(createAutorenewPollMessage(createHistoryEntry).build());
@@ -189,9 +186,8 @@ class DeleteExpiredDomainsActionTest {
return pendingExpirationDomain;
}
private BillingEvent.Recurring.Builder createAutorenewBillingEvent(
DomainHistory createHistoryEntry) {
return new BillingEvent.Recurring.Builder()
private BillingRecurrence.Builder createAutorenewBillingEvent(DomainHistory createHistoryEntry) {
return new BillingRecurrence.Builder()
.setReason(Reason.RENEW)
.setFlags(ImmutableSet.of(Flag.AUTO_RENEW))
.setTargetId("fizz.tld")
@@ -18,6 +18,7 @@ import static com.google.common.truth.Truth.assertThat;
import static com.google.common.truth.Truth.assertWithMessage;
import static com.google.common.truth.Truth8.assertThat;
import static google.registry.model.EppResourceUtils.loadByForeignKey;
import static google.registry.testing.DatabaseHelper.assertDomainDnsRequests;
import static google.registry.testing.DatabaseHelper.createTld;
import static google.registry.testing.DatabaseHelper.loadByEntitiesIfPresent;
import static google.registry.testing.DatabaseHelper.loadByEntity;
@@ -30,14 +31,12 @@ import static google.registry.testing.DatabaseHelper.persistSimpleResource;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static org.joda.time.DateTimeZone.UTC;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.mockito.Mockito.mock;
import com.google.common.collect.ImmutableSet;
import google.registry.config.RegistryEnvironment;
import google.registry.dns.DnsUtils;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.poll.PollMessage;
@@ -47,7 +46,6 @@ import google.registry.model.tld.Tld.TldType;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
import google.registry.testing.DatabaseHelper;
import google.registry.testing.DnsUtilsHelper;
import google.registry.testing.SystemPropertyExtension;
import java.util.Optional;
import java.util.Set;
@@ -72,9 +70,6 @@ class DeleteProberDataActionTest {
private DeleteProberDataAction action;
private final DnsUtils dnsUtils = mock(DnsUtils.class);
private final DnsUtilsHelper dnsUtilsHelper = new DnsUtilsHelper(dnsUtils);
@BeforeEach
void beforeEach() {
// Entities in these two should not be touched.
@@ -99,7 +94,6 @@ class DeleteProberDataActionTest {
private void resetAction() {
action = new DeleteProberDataAction();
action.dnsUtils = dnsUtils;
action.isDryRun = false;
action.tlds = ImmutableSet.of();
action.registryAdminRegistrarId = "TheRegistrar";
@@ -201,7 +195,7 @@ class DeleteProberDataActionTest {
DateTime timeAfterDeletion = DateTime.now(UTC);
assertThat(loadByForeignKey(Domain.class, "blah.ib-any.test", timeAfterDeletion)).isEmpty();
assertThat(loadByEntity(domain).getDeletionTime()).isLessThan(timeAfterDeletion);
dnsUtilsHelper.assertDomainDnsRequests("blah.ib-any.test");
assertDomainDnsRequests("blah.ib-any.test");
}
@Test
@@ -218,7 +212,7 @@ class DeleteProberDataActionTest {
action.run();
assertThat(loadByForeignKey(Domain.class, "blah.ib-any.test", timeAfterDeletion)).isEmpty();
assertThat(loadByEntity(domain).getDeletionTime()).isLessThan(timeAfterDeletion);
dnsUtilsHelper.assertDomainDnsRequests("blah.ib-any.test");
assertDomainDnsRequests("blah.ib-any.test");
}
@Test
@@ -290,9 +284,9 @@ class DeleteProberDataActionTest {
.setRegistrarId("TheRegistrar")
.setModificationTime(DELETION_TIME.minusYears(3))
.build());
BillingEvent.OneTime billingEvent =
BillingEvent billingEvent =
persistSimpleResource(
new BillingEvent.OneTime.Builder()
new BillingEvent.Builder()
.setDomainHistory(historyEntry)
.setBillingTime(DELETION_TIME.plusYears(1))
.setCost(Money.parse("USD 10"))
@@ -39,15 +39,14 @@ import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
import org.mockito.ArgumentCaptor;
/** Unit tests for {@link ExpandRecurringBillingEventsAction}. */
public class ExpandRecurringBillingEventsActionTest extends BeamActionTestBase {
/** Unit tests for {@link ExpandBillingRecurrencesAction}. */
public class ExpandBillingRecurrencesActionTest extends BeamActionTestBase {
private final DateTime cursorTime = DateTime.parse("2020-02-01T00:00:00Z");
private final DateTime now = DateTime.parse("2020-02-02T00:00:00Z");
private final FakeClock clock = new FakeClock(now);
private final ExpandRecurringBillingEventsAction action =
new ExpandRecurringBillingEventsAction();
private final ExpandBillingRecurrencesAction action = new ExpandBillingRecurrencesAction();
private final HashMap<String, String> expectedParameters = new HashMap<>();
private final ArgumentCaptor<LaunchFlexTemplateRequest> launchRequest =
@@ -82,7 +81,7 @@ public class ExpandRecurringBillingEventsActionTest extends BeamActionTestBase {
action.run();
assertThat(response.getStatus()).isEqualTo(200);
assertThat(response.getPayload())
.isEqualTo("Launched recurring billing event expansion pipeline: jobid");
.isEqualTo("Launched billing recurrence expansion pipeline: jobid");
verify(templates, times(1)).launch(eq("projectId"), eq("jobRegion"), launchRequest.capture());
assertThat(launchRequest.getValue().getLaunchParameter().getParameters())
.containsExactlyEntriesIn(expectedParameters);
@@ -95,7 +94,7 @@ public class ExpandRecurringBillingEventsActionTest extends BeamActionTestBase {
action.run();
assertThat(response.getStatus()).isEqualTo(200);
assertThat(response.getPayload())
.isEqualTo("Launched recurring billing event expansion pipeline: jobid");
.isEqualTo("Launched billing recurrence expansion pipeline: jobid");
verify(templates, times(1)).launch(eq("projectId"), eq("jobRegion"), launchRequest.capture());
assertThat(launchRequest.getValue().getLaunchParameter().getParameters())
.containsExactlyEntriesIn(expectedParameters);
@@ -108,7 +107,7 @@ public class ExpandRecurringBillingEventsActionTest extends BeamActionTestBase {
action.run();
assertThat(response.getStatus()).isEqualTo(200);
assertThat(response.getPayload())
.isEqualTo("Launched recurring billing event expansion pipeline: jobid");
.isEqualTo("Launched billing recurrence expansion pipeline: jobid");
verify(templates, times(1)).launch(eq("projectId"), eq("jobRegion"), launchRequest.capture());
assertThat(launchRequest.getValue().getLaunchParameter().getParameters())
.containsExactlyEntriesIn(expectedParameters);
@@ -121,7 +120,7 @@ public class ExpandRecurringBillingEventsActionTest extends BeamActionTestBase {
action.run();
assertThat(response.getStatus()).isEqualTo(200);
assertThat(response.getPayload())
.isEqualTo("Launched recurring billing event expansion pipeline: jobid");
.isEqualTo("Launched billing recurrence expansion pipeline: jobid");
verify(templates, times(1)).launch(eq("projectId"), eq("jobRegion"), launchRequest.capture());
assertThat(launchRequest.getValue().getLaunchParameter().getParameters())
.containsExactlyEntriesIn(expectedParameters);
@@ -136,7 +135,7 @@ public class ExpandRecurringBillingEventsActionTest extends BeamActionTestBase {
action.run();
assertThat(response.getStatus()).isEqualTo(200);
assertThat(response.getPayload())
.isEqualTo("Launched recurring billing event expansion pipeline: jobid");
.isEqualTo("Launched billing recurrence expansion pipeline: jobid");
verify(templates, times(1)).launch(eq("projectId"), eq("jobRegion"), launchRequest.capture());
assertThat(launchRequest.getValue().getLaunchParameter().getParameters())
.containsExactlyEntriesIn(expectedParameters);
@@ -37,11 +37,10 @@ import static org.junit.jupiter.api.Assertions.assertThrows;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import google.registry.beam.TestPipelineExtension;
import google.registry.model.billing.BillingBase.Flag;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.common.Cursor;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
@@ -71,8 +70,8 @@ import org.junit.jupiter.api.extension.RegisterExtension;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
/** Unit tests for {@link ExpandRecurringBillingEventsPipeline}. */
public class ExpandRecurringBillingEventsPipelineTest {
/** Unit tests for {@link ExpandBillingRecurrencesPipeline}. */
public class ExpandBillingRecurrencesPipelineTest {
private static final DateTimeFormatter DATE_TIME_FORMATTER =
DateTimeFormat.forPattern("yyyy-MM-dd'T'HH:mm:ss.SSSZ");
@@ -87,7 +86,7 @@ public class ExpandRecurringBillingEventsPipelineTest {
private Domain domain;
private Recurring recurring;
private BillingRecurrence billingRecurrence;
private final TestOptions options = PipelineOptionsFactory.create().as(TestOptions.class);
@@ -115,7 +114,7 @@ public class ExpandRecurringBillingEventsPipelineTest {
// Set up the database.
createTld("tld");
recurring = createDomainAtTime("example.tld", startTime.minusYears(1).plusHours(12));
billingRecurrence = createDomainAtTime("example.tld", startTime.minusYears(1).plusHours(12));
domain = loadByForeignKey(Domain.class, "example.tld", clock.nowUtc()).get();
}
@@ -150,7 +149,7 @@ public class ExpandRecurringBillingEventsPipelineTest {
assertBillingEventsForResource(
domain,
defaultOneTime(getOnlyAutoRenewHistory()),
recurring
billingRecurrence
.asBuilder()
.setRecurrenceLastExpansion(domain.getCreationTime().plusYears(1))
.build());
@@ -162,8 +161,9 @@ public class ExpandRecurringBillingEventsPipelineTest {
@Test
void testSuccess_expandSingleEvent_deletedDuringGracePeriod() {
domain = persistResource(domain.asBuilder().setDeletionTime(endTime.minusHours(2)).build());
recurring =
persistResource(recurring.asBuilder().setRecurrenceEndTime(endTime.minusHours(2)).build());
billingRecurrence =
persistResource(
billingRecurrence.asBuilder().setRecurrenceEndTime(endTime.minusHours(2)).build());
runPipeline();
// Assert about DomainHistory, no transaction record should have been written.
@@ -174,7 +174,7 @@ public class ExpandRecurringBillingEventsPipelineTest {
assertBillingEventsForResource(
domain,
defaultOneTime(getOnlyAutoRenewHistory()),
recurring
billingRecurrence
.asBuilder()
.setRecurrenceLastExpansion(domain.getCreationTime().plusYears(1))
.build());
@@ -199,7 +199,7 @@ public class ExpandRecurringBillingEventsPipelineTest {
assertBillingEventsForResource(
domain,
defaultOneTime(getOnlyAutoRenewHistory()),
recurring
billingRecurrence
.asBuilder()
.setRecurrenceLastExpansion(domain.getCreationTime().plusYears(1))
.build());
@@ -210,11 +210,11 @@ public class ExpandRecurringBillingEventsPipelineTest {
@Test
void testSuccess_noExpansion_recurrenceClosedBeforeEventTime() {
recurring =
billingRecurrence =
persistResource(
recurring
billingRecurrence
.asBuilder()
.setRecurrenceEndTime(recurring.getEventTime().minusDays(1))
.setRecurrenceEndTime(billingRecurrence.getEventTime().minusDays(1))
.build());
runPipeline();
assertNoExpansionsHappened();
@@ -222,19 +222,20 @@ public class ExpandRecurringBillingEventsPipelineTest {
@Test
void testSuccess_noExpansion_recurrenceClosedBeforeStartTime() {
recurring =
persistResource(recurring.asBuilder().setRecurrenceEndTime(startTime.minusDays(1)).build());
billingRecurrence =
persistResource(
billingRecurrence.asBuilder().setRecurrenceEndTime(startTime.minusDays(1)).build());
runPipeline();
assertNoExpansionsHappened();
}
@Test
void testSuccess_noExpansion_recurrenceClosedBeforeNextExpansion() {
recurring =
billingRecurrence =
persistResource(
recurring
billingRecurrence
.asBuilder()
.setEventTime(recurring.getEventTime().minusYears(1))
.setEventTime(billingRecurrence.getEventTime().minusYears(1))
.setRecurrenceEndTime(startTime.plusHours(6))
.build());
runPipeline();
@@ -243,16 +244,17 @@ public class ExpandRecurringBillingEventsPipelineTest {
@Test
void testSuccess_noExpansion_eventTimeAfterEndTime() {
recurring = persistResource(recurring.asBuilder().setEventTime(endTime.plusDays(1)).build());
billingRecurrence =
persistResource(billingRecurrence.asBuilder().setEventTime(endTime.plusDays(1)).build());
runPipeline();
assertNoExpansionsHappened();
}
@Test
void testSuccess_noExpansion_LastExpansionLessThanAYearAgo() {
recurring =
billingRecurrence =
persistResource(
recurring
billingRecurrence
.asBuilder()
.setRecurrenceLastExpansion(startTime.minusYears(1).plusDays(1))
.build());
@@ -263,7 +265,7 @@ public class ExpandRecurringBillingEventsPipelineTest {
@Test
void testSuccess_noExpansion_oneTimeAlreadyExists() {
DomainHistory history = persistResource(defaultDomainHistory());
OneTime oneTime = persistResource(defaultOneTime(history));
BillingEvent billingEvent = persistResource(defaultOneTime(history));
runPipeline();
// Assert about DomainHistory.
@@ -271,7 +273,7 @@ public class ExpandRecurringBillingEventsPipelineTest {
// Assert about BillingEvents. No expansion happened, so last recurrence expansion time is
// unchanged.
assertBillingEventsForResource(domain, oneTime, recurring);
assertBillingEventsForResource(domain, billingEvent, billingRecurrence);
// Assert about Cursor.
assertCursorAt(endTime);
@@ -296,7 +298,7 @@ public class ExpandRecurringBillingEventsPipelineTest {
assertBillingEventsForResource(
domain,
defaultOneTime(getOnlyAutoRenewHistory()),
recurring
billingRecurrence
.asBuilder()
.setRecurrenceLastExpansion(domain.getCreationTime().plusYears(1))
.build());
@@ -306,7 +308,7 @@ public class ExpandRecurringBillingEventsPipelineTest {
}
// We control the number of threads used in the pipeline to test if the batching behavior works
// properly. When two threads are used, the two recurrings are processed in different workers and
// properly. When two threads are used, the two recurrences are processed in different workers and
// should be processed in parallel.
@ParameterizedTest
@ValueSource(ints = {1, 2})
@@ -318,7 +320,7 @@ public class ExpandRecurringBillingEventsPipelineTest {
.setPremiumList(persistPremiumList("premium", USD, "other,USD 100"))
.build());
DateTime otherCreateTime = startTime.minusYears(1).plusHours(5);
Recurring otherRecurring = createDomainAtTime("other.test", otherCreateTime);
BillingRecurrence otherBillingRecurrence = createDomainAtTime("other.test", otherCreateTime);
Domain otherDomain = loadByForeignKey(Domain.class, "other.test", clock.nowUtc()).get();
options.setTargetParallelism(numOfThreads);
@@ -334,14 +336,15 @@ public class ExpandRecurringBillingEventsPipelineTest {
assertBillingEventsForResource(
domain,
defaultOneTime(getOnlyAutoRenewHistory()),
recurring
billingRecurrence
.asBuilder()
.setRecurrenceLastExpansion(domain.getCreationTime().plusYears(1))
.build());
assertBillingEventsForResource(
otherDomain,
defaultOneTime(otherDomain, getOnlyAutoRenewHistory(otherDomain), otherRecurring, 100),
otherRecurring
defaultOneTime(
otherDomain, getOnlyAutoRenewHistory(otherDomain), otherBillingRecurrence, 100),
otherBillingRecurrence
.asBuilder()
.setRecurrenceLastExpansion(otherDomain.getCreationTime().plusYears(1))
.build());
@@ -396,7 +399,7 @@ public class ExpandRecurringBillingEventsPipelineTest {
.setBillingTime(
domain.getCreationTime().plusYears(2).plus(Tld.DEFAULT_AUTO_RENEW_GRACE_PERIOD))
.build(),
recurring
billingRecurrence
.asBuilder()
.setRecurrenceLastExpansion(domain.getCreationTime().plusYears(2))
.build());
@@ -406,9 +409,9 @@ public class ExpandRecurringBillingEventsPipelineTest {
}
private void runPipeline() {
ExpandRecurringBillingEventsPipeline expandRecurringBillingEventsPipeline =
new ExpandRecurringBillingEventsPipeline(options, clock);
expandRecurringBillingEventsPipeline.setupPipeline(pipeline);
ExpandBillingRecurrencesPipeline expandBillingRecurrencesPipeline =
new ExpandBillingRecurrencesPipeline(options, clock);
expandBillingRecurrencesPipeline.setupPipeline(pipeline);
pipeline.run(options).waitUntilFinish();
}
@@ -424,7 +427,7 @@ public class ExpandRecurringBillingEventsPipelineTest {
assertThat(persistedHistory.get(0).getType()).isEqualTo(DOMAIN_CREATE);
// Only the original recurrence is present.
assertBillingEventsForResource(domain, recurring);
assertBillingEventsForResource(domain, billingRecurrence);
// If this is not a dry run, the cursor should still be moved even though expansions happened,
// because we still successfully processed all the needed expansions (none in this case) in the
@@ -458,13 +461,13 @@ public class ExpandRecurringBillingEventsPipelineTest {
.build();
}
private OneTime defaultOneTime(DomainHistory history) {
return defaultOneTime(domain, history, recurring, 11);
private BillingEvent defaultOneTime(DomainHistory history) {
return defaultOneTime(domain, history, billingRecurrence, 11);
}
private OneTime defaultOneTime(
Domain domain, DomainHistory history, Recurring recurring, int cost) {
return new BillingEvent.OneTime.Builder()
private BillingEvent defaultOneTime(
Domain domain, DomainHistory history, BillingRecurrence billingRecurrence, int cost) {
return new BillingEvent.Builder()
.setBillingTime(
domain.getCreationTime().plusYears(1).plus(Tld.DEFAULT_AUTO_RENEW_GRACE_PERIOD))
.setRegistrarId("TheRegistrar")
@@ -474,7 +477,7 @@ public class ExpandRecurringBillingEventsPipelineTest {
.setPeriodYears(1)
.setReason(Reason.RENEW)
.setSyntheticCreationTime(endTime)
.setCancellationMatchingBillingEvent(recurring)
.setCancellationMatchingBillingEvent(billingRecurrence)
.setTargetId(domain.getDomainName())
.setDomainHistory(history)
.build();
@@ -517,7 +520,7 @@ public class ExpandRecurringBillingEventsPipelineTest {
assertThat(cursor.getCursorTime()).isEqualTo(expectedCursorTime);
}
private static Recurring createDomainAtTime(String domainName, DateTime createTime) {
private static BillingRecurrence createDomainAtTime(String domainName, DateTime createTime) {
Domain domain = persistActiveDomain(domainName, createTime);
DomainHistory domainHistory =
persistResource(
@@ -528,7 +531,7 @@ public class ExpandRecurringBillingEventsPipelineTest {
.setDomain(domain)
.build());
return persistResource(
new Recurring.Builder()
new BillingRecurrence.Builder()
.setDomainHistory(domainHistory)
.setRegistrarId(domain.getCreationRegistrarId())
.setEventTime(createTime.plusYears(1))
@@ -539,5 +542,5 @@ public class ExpandRecurringBillingEventsPipelineTest {
.build());
}
public interface TestOptions extends ExpandRecurringBillingEventsPipelineOptions, DirectOptions {}
public interface TestOptions extends ExpandBillingRecurrencesPipelineOptions, DirectOptions {}
}
@@ -36,11 +36,11 @@ import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedMap;
import com.google.common.testing.TestLogHandler;
import google.registry.beam.TestPipelineExtension;
import google.registry.model.billing.BillingEvent.Cancellation;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingBase.Flag;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingCancellation;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.registrar.Registrar;
@@ -89,9 +89,9 @@ class InvoicingPipelineTest {
private static final String YEAR_MONTH = "2017-10";
private static final String INVOICE_FILE_PREFIX = "REG-INV";
private static final ImmutableList<BillingEvent> INPUT_EVENTS =
private static final ImmutableList<google.registry.beam.billing.BillingEvent> INPUT_EVENTS =
ImmutableList.of(
BillingEvent.create(
google.registry.beam.billing.BillingEvent.create(
1,
DateTime.parse("2017-10-04T00:00:00Z"),
DateTime.parse("2017-10-04T00:00:00Z"),
@@ -106,7 +106,7 @@ class InvoicingPipelineTest {
"USD",
20.5,
""),
BillingEvent.create(
google.registry.beam.billing.BillingEvent.create(
2,
DateTime.parse("2017-10-04T00:00:00Z"),
DateTime.parse("2017-10-04T00:00:00Z"),
@@ -121,7 +121,7 @@ class InvoicingPipelineTest {
"USD",
20.5,
""),
BillingEvent.create(
google.registry.beam.billing.BillingEvent.create(
3,
DateTime.parse("2017-10-02T00:00:00Z"),
DateTime.parse("2017-09-29T00:00:00Z"),
@@ -136,7 +136,7 @@ class InvoicingPipelineTest {
"JPY",
70.0,
""),
BillingEvent.create(
google.registry.beam.billing.BillingEvent.create(
4,
DateTime.parse("2017-10-04T00:00:00Z"),
DateTime.parse("2017-10-04T00:00:00Z"),
@@ -151,7 +151,7 @@ class InvoicingPipelineTest {
"USD",
20.5,
""),
BillingEvent.create(
google.registry.beam.billing.BillingEvent.create(
5,
DateTime.parse("2017-10-04T00:00:00Z"),
DateTime.parse("2017-10-04T00:00:00Z"),
@@ -166,7 +166,7 @@ class InvoicingPipelineTest {
"USD",
0.0,
"SUNRISE ANCHOR_TENANT"),
BillingEvent.create(
google.registry.beam.billing.BillingEvent.create(
6,
DateTime.parse("2017-10-04T00:00:00Z"),
DateTime.parse("2017-10-04T00:00:00Z"),
@@ -181,7 +181,7 @@ class InvoicingPipelineTest {
"USD",
0.0,
""),
BillingEvent.create(
google.registry.beam.billing.BillingEvent.create(
7,
DateTime.parse("2017-10-04T00:00:00Z"),
DateTime.parse("2017-10-04T00:00:00Z"),
@@ -237,7 +237,7 @@ class InvoicingPipelineTest {
PipelineOptionsFactory.create().as(InvoicingPipelineOptions.class);
private File billingBucketUrl;
private PCollection<BillingEvent> billingEvents;
private PCollection<google.registry.beam.billing.BillingEvent> billingEvents;
private final TestLogHandler logHandler = new TestLogHandler();
private final Logger loggerToIntercept =
@@ -251,7 +251,9 @@ class InvoicingPipelineTest {
options.setYearMonth(YEAR_MONTH);
options.setInvoiceFilePrefix(INVOICE_FILE_PREFIX);
billingEvents =
pipeline.apply(Create.of(INPUT_EVENTS).withCoder(SerializableCoder.of(BillingEvent.class)));
pipeline.apply(
Create.of(INPUT_EVENTS)
.withCoder(SerializableCoder.of(google.registry.beam.billing.BillingEvent.class)));
}
@Test
@@ -274,7 +276,8 @@ class InvoicingPipelineTest {
@Test
void testSuccess_readFromCloudSql() throws Exception {
setupCloudSql();
PCollection<BillingEvent> billingEvents = InvoicingPipeline.readFromCloudSql(options, pipeline);
PCollection<google.registry.beam.billing.BillingEvent> billingEvents =
InvoicingPipeline.readFromCloudSql(options, pipeline);
billingEvents = billingEvents.apply(new ChangeDomainRepo());
PAssert.that(billingEvents).containsInAnyOrder(INPUT_EVENTS);
pipeline.run().waitUntilFinish();
@@ -299,8 +302,9 @@ class InvoicingPipelineTest {
persistResource(test);
Domain domain = persistActiveDomain("mycanadiandomain.test");
persistOneTimeBillingEvent(25, domain, registrar, Reason.RENEW, 3, Money.of(CAD, 20.5));
PCollection<BillingEvent> billingEvents = InvoicingPipeline.readFromCloudSql(options, pipeline);
persistBillingEvent(25, domain, registrar, Reason.RENEW, 3, Money.of(CAD, 20.5));
PCollection<google.registry.beam.billing.BillingEvent> billingEvents =
InvoicingPipeline.readFromCloudSql(options, pipeline);
billingEvents = billingEvents.apply(new ChangeDomainRepo());
PAssert.that(billingEvents).containsInAnyOrder(INPUT_EVENTS);
pipeline.run().waitUntilFinish();
@@ -352,9 +356,9 @@ class InvoicingPipelineTest {
+ "JOIN Registrar r ON b.clientId = r.registrarId\n"
+ "JOIN Domain d ON b.domainRepoId = d.repoId\n"
+ "JOIN Tld t ON t.tldStr = d.tld\n"
+ "LEFT JOIN BillingCancellation c ON b.id = c.refOneTime\n"
+ "LEFT JOIN BillingCancellation c ON b.id = c.billingEvent\n"
+ "LEFT JOIN BillingCancellation cr ON b.cancellationMatchingBillingEvent ="
+ " cr.refRecurring\n"
+ " cr.billingRecurrence\n"
+ "WHERE r.billingAccountMap IS NOT NULL\n"
+ "AND r.type = 'REAL'\n"
+ "AND t.invoicingEnabled IS TRUE\n"
@@ -416,9 +420,9 @@ class InvoicingPipelineTest {
Domain domain6 = persistActiveDomain("locked.test");
Domain domain7 = persistActiveDomain("update-prohibited.test");
persistOneTimeBillingEvent(1, domain1, registrar1, Reason.RENEW, 3, Money.of(USD, 20.5));
persistOneTimeBillingEvent(2, domain2, registrar1, Reason.RENEW, 3, Money.of(USD, 20.5));
persistOneTimeBillingEvent(
persistBillingEvent(1, domain1, registrar1, Reason.RENEW, 3, Money.of(USD, 20.5));
persistBillingEvent(2, domain2, registrar1, Reason.RENEW, 3, Money.of(USD, 20.5));
persistBillingEvent(
3,
domain3,
registrar1,
@@ -427,8 +431,8 @@ class InvoicingPipelineTest {
Money.ofMajor(JPY, 70),
DateTime.parse("2017-09-29T00:00:00.0Z"),
DateTime.parse("2017-10-02T00:00:00.0Z"));
persistOneTimeBillingEvent(4, domain4, registrar2, Reason.RENEW, 1, Money.of(USD, 20.5));
persistOneTimeBillingEvent(
persistBillingEvent(4, domain4, registrar2, Reason.RENEW, 1, Money.of(USD, 20.5));
persistBillingEvent(
5,
domain5,
registrar3,
@@ -439,15 +443,15 @@ class InvoicingPipelineTest {
DateTime.parse("2017-10-04T00:00:00.0Z"),
Flag.SUNRISE,
Flag.ANCHOR_TENANT);
persistOneTimeBillingEvent(6, domain6, registrar1, Reason.SERVER_STATUS, 0, Money.of(USD, 0));
persistOneTimeBillingEvent(7, domain7, registrar1, Reason.SERVER_STATUS, 0, Money.of(USD, 20));
persistBillingEvent(6, domain6, registrar1, Reason.SERVER_STATUS, 0, Money.of(USD, 0));
persistBillingEvent(7, domain7, registrar1, Reason.SERVER_STATUS, 0, Money.of(USD, 20));
// Add billing event for a non-billable registrar
Registrar registrar4 = persistNewRegistrar("noBillRegistrar");
registrar4 = registrar4.asBuilder().setBillingAccountMap(null).build();
persistResource(registrar4);
Domain domain8 = persistActiveDomain("non-billable.test");
persistOneTimeBillingEvent(8, domain8, registrar4, Reason.RENEW, 3, Money.of(USD, 20.5));
persistBillingEvent(8, domain8, registrar4, Reason.RENEW, 3, Money.of(USD, 20.5));
// Add billing event for a non-real registrar
Registrar registrar5 = persistNewRegistrar("notRealRegistrar");
@@ -460,16 +464,16 @@ class InvoicingPipelineTest {
.build();
persistResource(registrar5);
Domain domain9 = persistActiveDomain("not-real.test");
persistOneTimeBillingEvent(9, domain9, registrar5, Reason.RENEW, 3, Money.of(USD, 20.5));
persistBillingEvent(9, domain9, registrar5, Reason.RENEW, 3, Money.of(USD, 20.5));
// Add billing event for a non-invoicing TLD
createTld("nobill");
Domain domain10 = persistActiveDomain("test.nobill");
persistOneTimeBillingEvent(10, domain10, registrar1, Reason.RENEW, 3, Money.of(USD, 20.5));
persistBillingEvent(10, domain10, registrar1, Reason.RENEW, 3, Money.of(USD, 20.5));
// Add billing event before October 2017
Domain domain11 = persistActiveDomain("july.test");
persistOneTimeBillingEvent(
persistBillingEvent(
11,
domain11,
registrar1,
@@ -481,64 +485,64 @@ class InvoicingPipelineTest {
// Add a billing event with a corresponding cancellation
Domain domain12 = persistActiveDomain("cancel.test");
OneTime oneTime =
persistOneTimeBillingEvent(12, domain12, registrar1, Reason.RENEW, 3, Money.of(USD, 20.5));
BillingEvent billingEvent =
persistBillingEvent(12, domain12, registrar1, Reason.RENEW, 3, Money.of(USD, 20.5));
DomainHistory domainHistory = persistDomainHistory(domain12, registrar1);
Cancellation cancellation =
new Cancellation()
BillingCancellation cancellation =
new BillingCancellation()
.asBuilder()
.setId(1)
.setRegistrarId(registrar1.getRegistrarId())
.setEventTime(DateTime.parse("2017-10-05T00:00:00.0Z"))
.setBillingTime(DateTime.parse("2017-10-04T00:00:00.0Z"))
.setOneTimeEventKey(oneTime.createVKey())
.setBillingEvent(billingEvent.createVKey())
.setTargetId(domain12.getDomainName())
.setReason(Reason.RENEW)
.setDomainHistory(domainHistory)
.build();
persistResource(cancellation);
// Add billing event with a corresponding recurring billing event and cancellation
Domain domain13 = persistActiveDomain("cancel-recurring.test");
DomainHistory domainHistoryRecurring = persistDomainHistory(domain13, registrar1);
// Add billing event with a corresponding recurrence billing event and cancellation
Domain domain13 = persistActiveDomain("cancel-recurrence.test");
DomainHistory domainHistoryRecurrence = persistDomainHistory(domain13, registrar1);
Recurring recurring =
new Recurring()
BillingRecurrence billingRecurrence =
new BillingRecurrence()
.asBuilder()
.setRegistrarId(registrar1.getRegistrarId())
.setRecurrenceEndTime(END_OF_TIME)
.setId(1)
.setDomainHistory(domainHistoryRecurring)
.setDomainHistory(domainHistoryRecurrence)
.setTargetId(domain13.getDomainName())
.setEventTime(DateTime.parse("2017-10-04T00:00:00.0Z"))
.setReason(Reason.RENEW)
.build();
persistResource(recurring);
OneTime oneTimeRecurring =
persistOneTimeBillingEvent(13, domain13, registrar1, Reason.RENEW, 3, Money.of(USD, 20.5));
oneTimeRecurring =
oneTimeRecurring
persistResource(billingRecurrence);
BillingEvent billingEventRecurrence =
persistBillingEvent(13, domain13, registrar1, Reason.RENEW, 3, Money.of(USD, 20.5));
billingEventRecurrence =
billingEventRecurrence
.asBuilder()
.setCancellationMatchingBillingEvent(recurring)
.setCancellationMatchingBillingEvent(billingRecurrence)
.setFlags(ImmutableSet.of(Flag.SYNTHETIC))
.setSyntheticCreationTime(DateTime.parse("2017-10-03T00:00:00.0Z"))
.build();
persistResource(oneTimeRecurring);
persistResource(billingEventRecurrence);
Cancellation cancellationRecurring =
new Cancellation()
BillingCancellation cancellationRecurrence =
new BillingCancellation()
.asBuilder()
.setId(2)
.setRegistrarId(registrar1.getRegistrarId())
.setEventTime(DateTime.parse("2017-10-05T00:00:00.0Z"))
.setBillingTime(DateTime.parse("2017-10-04T00:00:00.0Z"))
.setRecurringEventKey(recurring.createVKey())
.setBillingRecurrence(billingRecurrence.createVKey())
.setTargetId(domain13.getDomainName())
.setReason(Reason.RENEW)
.setDomainHistory(domainHistoryRecurring)
.setDomainHistory(domainHistoryRecurrence)
.build();
persistResource(cancellationRecurring);
persistResource(cancellationRecurrence);
}
private static DomainHistory persistDomainHistory(Domain domain, Registrar registrar) {
@@ -552,9 +556,9 @@ class InvoicingPipelineTest {
return persistResource(domainHistory);
}
private static OneTime persistOneTimeBillingEvent(
private static BillingEvent persistBillingEvent(
int id, Domain domain, Registrar registrar, Reason reason, int years, Money money) {
return persistOneTimeBillingEvent(
return persistBillingEvent(
id,
domain,
registrar,
@@ -565,7 +569,7 @@ class InvoicingPipelineTest {
DateTime.parse("2017-10-04T00:00:00.0Z"));
}
private static OneTime persistOneTimeBillingEvent(
private static BillingEvent persistBillingEvent(
int id,
Domain domain,
Registrar registrar,
@@ -575,8 +579,8 @@ class InvoicingPipelineTest {
DateTime eventTime,
DateTime billingTime,
Flag... flags) {
OneTime.Builder billingEventBuilder =
new OneTime()
BillingEvent.Builder billingEventBuilder =
new BillingEvent()
.asBuilder()
.setId(id)
.setBillingTime(billingTime)
@@ -596,18 +600,21 @@ class InvoicingPipelineTest {
}
private static class ChangeDomainRepo
extends PTransform<PCollection<BillingEvent>, PCollection<BillingEvent>> {
extends PTransform<
PCollection<google.registry.beam.billing.BillingEvent>,
PCollection<google.registry.beam.billing.BillingEvent>> {
private static final long serialVersionUID = 2695033474967615250L;
@Override
public PCollection<BillingEvent> expand(PCollection<BillingEvent> input) {
public PCollection<google.registry.beam.billing.BillingEvent> expand(
PCollection<google.registry.beam.billing.BillingEvent> input) {
return input.apply(
"Map to invoicing key",
MapElements.into(TypeDescriptor.of(BillingEvent.class))
MapElements.into(TypeDescriptor.of(google.registry.beam.billing.BillingEvent.class))
.via(
billingEvent ->
BillingEvent.create(
google.registry.beam.billing.BillingEvent.create(
billingEvent.id(),
billingEvent.billingTime(),
billingEvent.eventTime(),
@@ -15,11 +15,12 @@
package google.registry.dns;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.testing.DatabaseHelper.assertHostDnsRequests;
import static google.registry.testing.DatabaseHelper.assertNoDnsRequests;
import static google.registry.testing.DatabaseHelper.assertNoDnsRequestsExcept;
import static google.registry.testing.DatabaseHelper.createTld;
import static google.registry.testing.DatabaseHelper.persistActiveDomain;
import static google.registry.testing.DatabaseHelper.persistActiveSubordinateHost;
import static google.registry.testing.TaskQueueHelper.assertDnsTasksEnqueued;
import static google.registry.testing.TaskQueueHelper.assertNoDnsTasksEnqueued;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
@@ -30,7 +31,6 @@ import google.registry.request.HttpException.NotFoundException;
import google.registry.request.RequestModule;
import google.registry.testing.CloudTasksHelper.CloudTasksHelperModule;
import google.registry.testing.FakeClock;
import google.registry.testing.TaskQueueExtension;
import java.io.PrintWriter;
import java.io.StringWriter;
import javax.servlet.http.HttpServletRequest;
@@ -43,18 +43,15 @@ import org.junit.jupiter.api.extension.RegisterExtension;
/** Unit tests for Dagger injection of the DNS package. */
public final class DnsInjectionTest {
@RegisterExtension
final JpaIntegrationTestExtension jpa =
new JpaTestExtensions.Builder().buildIntegrationTestExtension();
@RegisterExtension final TaskQueueExtension taskQueue = new TaskQueueExtension();
private final HttpServletRequest req = mock(HttpServletRequest.class);
private final HttpServletResponse rsp = mock(HttpServletResponse.class);
private final StringWriter httpOutput = new StringWriter();
private final FakeClock clock = new FakeClock(DateTime.parse("2014-01-01TZ"));
private DnsTestComponent component;
private DnsQueue dnsQueue;
@RegisterExtension
final JpaIntegrationTestExtension jpa =
new JpaTestExtensions.Builder().withClock(clock).buildIntegrationTestExtension();
@BeforeEach
void beforeEach() throws Exception {
@@ -64,18 +61,18 @@ public final class DnsInjectionTest {
.requestModule(new RequestModule(req, rsp))
.cloudTasksHelperModule(new CloudTasksHelperModule(clock))
.build();
dnsQueue = component.dnsQueue();
createTld("lol");
}
@Test
void testReadDnsQueueAction_injectsAndWorks() {
void testReadDnsRefreshRequestsAction_injectsAndWorks() {
persistActiveSubordinateHost("ns1.example.lol", persistActiveDomain("example.lol"));
clock.advanceOneMilli();
dnsQueue.addDomainRefreshTask("example.lol");
DnsUtils.requestDomainDnsRefresh("example.lol");
when(req.getParameter("tld")).thenReturn("lol");
component.readDnsQueueAction().run();
assertNoDnsTasksEnqueued();
clock.advanceOneMilli();
component.readDnsRefreshRequestsAction().run();
assertNoDnsRequests();
}
@Test
@@ -84,7 +81,7 @@ public final class DnsInjectionTest {
when(req.getParameter("type")).thenReturn("domain");
when(req.getParameter("name")).thenReturn("example.lol");
component.refreshDns().run();
assertDnsTasksEnqueued("example.lol");
assertNoDnsRequestsExcept("example.lol");
}
@Test
@@ -102,7 +99,7 @@ public final class DnsInjectionTest {
when(req.getParameter("type")).thenReturn("host");
when(req.getParameter("name")).thenReturn("ns1.example.lol");
component.refreshDns().run();
assertDnsTasksEnqueued("ns1.example.lol");
assertHostDnsRequests("ns1.example.lol");
}
@Test
@@ -1,108 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.dns;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.testing.DatabaseHelper.createTld;
import static google.registry.testing.TaskQueueHelper.assertNoTasksEnqueued;
import static google.registry.testing.TaskQueueHelper.assertTasksEnqueued;
import static org.junit.jupiter.api.Assertions.assertThrows;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
import google.registry.testing.FakeClock;
import google.registry.testing.TaskQueueExtension;
import google.registry.testing.TaskQueueHelper.TaskMatcher;
import org.joda.time.DateTime;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
/** Unit tests for {@link DnsQueue}. */
public class DnsQueueTest {
@RegisterExtension
final JpaIntegrationTestExtension jpa =
new JpaTestExtensions.Builder().buildIntegrationTestExtension();
@RegisterExtension final TaskQueueExtension taskQueue = new TaskQueueExtension();
private DnsQueue dnsQueue;
private final FakeClock clock = new FakeClock(DateTime.parse("2010-01-01T10:00:00Z"));
@BeforeEach
void beforeEach() {
dnsQueue = DnsQueue.createForTesting(clock);
dnsQueue.leaseTasksBatchSize = 10;
}
@Test
void test_addHostRefreshTask_success() {
createTld("tld");
dnsQueue.addHostRefreshTask("octopus.tld");
assertTasksEnqueued(
"dns-pull",
new TaskMatcher()
.param("Target-Type", "HOST")
.param("Target-Name", "octopus.tld")
.param("Create-Time", "2010-01-01T10:00:00.000Z")
.param("tld", "tld"));
}
@Test
void test_addHostRefreshTask_failsOnUnknownTld() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() -> {
try {
dnsQueue.addHostRefreshTask("octopus.notatld");
} finally {
assertNoTasksEnqueued("dns-pull");
}
});
assertThat(thrown)
.hasMessageThat()
.contains("octopus.notatld is not a subordinate host to a known tld");
}
@Test
void test_addDomainRefreshTask_success() {
createTld("tld");
dnsQueue.addDomainRefreshTask("octopus.tld");
assertTasksEnqueued(
"dns-pull",
new TaskMatcher()
.param("Target-Type", "DOMAIN")
.param("Target-Name", "octopus.tld")
.param("Create-Time", "2010-01-01T10:00:00.000Z")
.param("tld", "tld"));
}
@Test
void test_addDomainRefreshTask_failsOnUnknownTld() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() -> {
try {
dnsQueue.addDomainRefreshTask("fake.notatld");
} finally {
assertNoTasksEnqueued("dns-pull");
}
});
assertThat(thrown).hasMessageThat().contains("TLD notatld does not exist");
}
}
@@ -35,7 +35,7 @@ import javax.inject.Singleton;
VoidDnsWriterModule.class,
})
interface DnsTestComponent {
DnsQueue dnsQueue();
RefreshDnsAction refreshDns();
ReadDnsQueueAction readDnsQueueAction();
ReadDnsRefreshRequestsAction readDnsRefreshRequestsAction();
}
@@ -20,20 +20,10 @@ import static google.registry.persistence.transaction.TransactionManagerFactory.
import static google.registry.testing.DatabaseHelper.createTld;
import static google.registry.testing.DatabaseHelper.loadAllOf;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSortedMap;
import com.google.common.collect.Iterables;
import com.google.common.collect.Ordering;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.model.common.DatabaseMigrationStateSchedule;
import google.registry.model.common.DatabaseMigrationStateSchedule.MigrationState;
import google.registry.dns.DnsUtils.TargetType;
import google.registry.model.common.DnsRefreshRequest;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
@@ -42,7 +32,6 @@ import java.util.Comparator;
import org.joda.time.DateTime;
import org.joda.time.Duration;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
@@ -54,52 +43,22 @@ public class DnsUtilsTest {
private static final String domainName = "test.tld";
private static final String hostName = "ns1.test.tld";
private final DnsQueue dnsQueue = mock(DnsQueue.class);
private final DnsUtils dnsUtils = new DnsUtils(dnsQueue);
private final FakeClock clock = new FakeClock(DateTime.parse("2020-02-02T01:23:45Z"));
@RegisterExtension
JpaIntegrationTestExtension jpa =
new JpaTestExtensions.Builder().withClock(clock).buildIntegrationTestExtension();
@BeforeAll
static void beforeAll() {
DatabaseMigrationStateSchedule.useUncachedForTest();
}
@BeforeEach
void beforeEach() {
createTld(tld);
when(dnsQueue.getClock()).thenReturn(clock);
}
@Test
void testSuccess_hostRefresh_pullQueue() {
dnsUtils.requestHostDnsRefresh(hostName);
verify(dnsQueue).addHostRefreshTask(hostName);
assertThat(loadAllOf(DnsRefreshRequest.class)).isEmpty();
}
@Test
void testSuccess_domainRefresh_pullQueue() {
dnsUtils.requestDomainDnsRefresh(domainName);
verify(dnsQueue).addDomainRefreshTask(domainName, Duration.ZERO);
assertThat(loadAllOf(DnsRefreshRequest.class)).isEmpty();
}
@Test
void testSuccess_domainRefreshWithDelay_pullQueue() {
dnsUtils.requestDomainDnsRefresh(domainName, Duration.standardMinutes(3));
verify(dnsQueue).addDomainRefreshTask(domainName, Duration.standardMinutes(3));
assertThat(loadAllOf(DnsRefreshRequest.class)).isEmpty();
}
@Test
void testFailure_hostRefresh_unmanagedHost() {
String unmanagedHostName = "ns1.another.example";
Assertions.assertThrows(
IllegalArgumentException.class, () -> dnsUtils.requestHostDnsRefresh(unmanagedHostName));
verify(dnsQueue, never()).addHostRefreshTask(anyString());
IllegalArgumentException.class, () -> DnsUtils.requestHostDnsRefresh(unmanagedHostName));
assertThat(loadAllOf(DnsRefreshRequest.class)).isEmpty();
}
@@ -108,34 +67,27 @@ public class DnsUtilsTest {
String unmanagedDomainName = "another.example";
Assertions.assertThrows(
IllegalArgumentException.class,
() -> dnsUtils.requestDomainDnsRefresh(unmanagedDomainName));
verify(dnsQueue, never()).addDomainRefreshTask(anyString(), any(Duration.class));
() -> DnsUtils.requestDomainDnsRefresh(unmanagedDomainName));
assertThat(loadAllOf(DnsRefreshRequest.class)).isEmpty();
}
@Test
void testSuccess_hostRefresh() {
useDnsSql();
dnsUtils.requestHostDnsRefresh(hostName);
verify(dnsQueue, never()).addHostRefreshTask(anyString());
DnsUtils.requestHostDnsRefresh(hostName);
DnsRefreshRequest request = Iterables.getOnlyElement(loadAllOf(DnsRefreshRequest.class));
assertRequest(request, TargetType.HOST, hostName, tld, clock.nowUtc());
}
@Test
void testSuccess_domainRefresh() {
useDnsSql();
dnsUtils.requestDomainDnsRefresh(domainName);
verify(dnsQueue, never()).addDomainRefreshTask(anyString(), any(Duration.class));
DnsUtils.requestDomainDnsRefresh(domainName);
DnsRefreshRequest request = Iterables.getOnlyElement(loadAllOf(DnsRefreshRequest.class));
assertRequest(request, TargetType.DOMAIN, domainName, tld, clock.nowUtc());
}
@Test
void testSuccess_domainRefreshWithDelay() {
useDnsSql();
dnsUtils.requestDomainDnsRefresh(domainName, Duration.standardMinutes(3));
verify(dnsQueue, never()).addDomainRefreshTask(anyString(), any(Duration.class));
DnsUtils.requestDomainDnsRefresh(domainName, Duration.standardMinutes(3));
DnsRefreshRequest request = Iterables.getOnlyElement(loadAllOf(DnsRefreshRequest.class));
assertRequest(request, TargetType.DOMAIN, domainName, tld, clock.nowUtc().plusMinutes(3));
}
@@ -182,7 +134,7 @@ public class DnsUtilsTest {
// Requests within cooldown period not included.
requests =
dnsUtils.readAndUpdateRequestsWithLatestProcessTime("tld", Duration.standardMinutes(1), 4);
DnsUtils.readAndUpdateRequestsWithLatestProcessTime("tld", Duration.standardMinutes(1), 4);
assertThat(requests.size()).isEqualTo(1);
assertRequest(
requests.get(0),
@@ -195,7 +147,7 @@ public class DnsUtilsTest {
@Test
void testSuccess_deleteRequests() {
dnsUtils.deleteRequests(processRequests());
DnsUtils.deleteRequests(processRequests());
ImmutableList<DnsRefreshRequest> remainingRequests =
loadAllOf(DnsRefreshRequest.class).stream()
.sorted(Comparator.comparing(DnsRefreshRequest::getRequestTime))
@@ -222,31 +174,30 @@ public class DnsUtilsTest {
tm().transact(() -> tm().delete(remainingRequests.get(2)));
assertThat(loadAllOf(DnsRefreshRequest.class).size()).isEqualTo(2);
// Should not throw even though one of the request is already deleted.
dnsUtils.deleteRequests(remainingRequests);
DnsUtils.deleteRequests(remainingRequests);
assertThat(loadAllOf(DnsRefreshRequest.class).size()).isEqualTo(0);
}
private ImmutableList<DnsRefreshRequest> processRequests() {
useDnsSql();
createTld("example");
// Domain Included.
dnsUtils.requestDomainDnsRefresh("test1.tld", Duration.standardMinutes(1));
DnsUtils.requestDomainDnsRefresh("test1.tld", Duration.standardMinutes(1));
// This one should be returned before test1.tld, even though it's added later, because of
// the delay specified in test1.tld.
dnsUtils.requestDomainDnsRefresh("test2.tld");
DnsUtils.requestDomainDnsRefresh("test2.tld");
// Not included because the TLD is not under management.
dnsUtils.requestDomainDnsRefresh("something.example", Duration.standardMinutes(2));
DnsUtils.requestDomainDnsRefresh("something.example", Duration.standardMinutes(2));
clock.advanceBy(Duration.standardMinutes(3));
// Host included.
dnsUtils.requestHostDnsRefresh("ns1.test2.tld");
DnsUtils.requestHostDnsRefresh("ns1.test2.tld");
// Not included because the request time is in the future
dnsUtils.requestDomainDnsRefresh("test4.tld", Duration.standardMinutes(2));
DnsUtils.requestDomainDnsRefresh("test4.tld", Duration.standardMinutes(2));
// Included after the previous one. Same request time, order by insertion order (i.e. ID);
dnsUtils.requestDomainDnsRefresh("test5.tld");
DnsUtils.requestDomainDnsRefresh("test5.tld");
// Not included because batch size is exceeded;
dnsUtils.requestDomainDnsRefresh("test6.tld");
DnsUtils.requestDomainDnsRefresh("test6.tld");
clock.advanceBy(Duration.standardMinutes(1));
return dnsUtils.readAndUpdateRequestsWithLatestProcessTime(
return DnsUtils.readAndUpdateRequestsWithLatestProcessTime(
"tld", Duration.standardMinutes(1), 4);
}
@@ -268,26 +219,4 @@ public class DnsUtilsTest {
assertThat(request.getRequestTime()).isEqualTo(requestTime);
assertThat(request.getLastProcessTime()).isEqualTo(processTime);
}
private void useDnsSql() {
DateTime currentTime = clock.nowUtc();
clock.setTo(START_OF_TIME);
tm().transact(
() ->
DatabaseMigrationStateSchedule.set(
new ImmutableSortedMap.Builder<DateTime, MigrationState>(Ordering.natural())
.put(START_OF_TIME, MigrationState.DATASTORE_ONLY)
.put(START_OF_TIME.plusMillis(1), MigrationState.DATASTORE_PRIMARY)
.put(START_OF_TIME.plusMillis(2), MigrationState.DATASTORE_PRIMARY_NO_ASYNC)
.put(
START_OF_TIME.plusMillis(3), MigrationState.DATASTORE_PRIMARY_READ_ONLY)
.put(START_OF_TIME.plusMillis(4), MigrationState.SQL_PRIMARY_READ_ONLY)
.put(START_OF_TIME.plusMillis(5), MigrationState.SQL_PRIMARY)
.put(START_OF_TIME.plusMillis(6), MigrationState.SQL_ONLY)
.put(START_OF_TIME.plusMillis(7), MigrationState.SEQUENCE_BASED_ALLOCATE_ID)
.put(START_OF_TIME.plusMillis(8), MigrationState.NORDN_SQL)
.put(START_OF_TIME.plusMillis(9), MigrationState.DNS_SQL)
.build()));
clock.setTo(currentTime);
}
}
@@ -15,7 +15,6 @@
package google.registry.dns;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.dns.DnsConstants.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.DnsModule.PARAM_DNS_WRITER;
import static google.registry.dns.DnsModule.PARAM_DOMAINS;
import static google.registry.dns.DnsModule.PARAM_HOSTS;
@@ -23,8 +22,13 @@ import static google.registry.dns.DnsModule.PARAM_LOCK_INDEX;
import static google.registry.dns.DnsModule.PARAM_NUM_PUBLISH_LOCKS;
import static google.registry.dns.DnsModule.PARAM_PUBLISH_TASK_ENQUEUED;
import static google.registry.dns.DnsModule.PARAM_REFRESH_REQUEST_TIME;
import static google.registry.dns.DnsUtils.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.PublishDnsUpdatesAction.RETRIES_BEFORE_PERMANENT_FAILURE;
import static google.registry.request.RequestParameters.PARAM_TLD;
import static google.registry.testing.DatabaseHelper.assertDomainDnsRequests;
import static google.registry.testing.DatabaseHelper.assertHostDnsRequests;
import static google.registry.testing.DatabaseHelper.assertNoDnsRequests;
import static google.registry.testing.DatabaseHelper.assertNoDnsRequestsExcept;
import static google.registry.testing.DatabaseHelper.createTld;
import static google.registry.testing.DatabaseHelper.persistActiveDomain;
import static google.registry.testing.DatabaseHelper.persistActiveSubordinateHost;
@@ -54,7 +58,6 @@ import google.registry.request.HttpException.ServiceUnavailableException;
import google.registry.request.lock.LockHandler;
import google.registry.testing.CloudTasksHelper;
import google.registry.testing.CloudTasksHelper.TaskMatcher;
import google.registry.testing.DnsUtilsHelper;
import google.registry.testing.FakeClock;
import google.registry.testing.FakeLockHandler;
import google.registry.testing.FakeResponse;
@@ -83,8 +86,6 @@ public class PublishDnsUpdatesActionTest {
private final FakeLockHandler lockHandler = new FakeLockHandler(true);
private final DnsWriter dnsWriter = mock(DnsWriter.class);
private final DnsMetrics dnsMetrics = mock(DnsMetrics.class);
private final DnsUtils dnsUtils = mock(DnsUtils.class);
private final DnsUtilsHelper dnsUtilsHelper = new DnsUtilsHelper(dnsUtils);
private final CloudTasksHelper cloudTasksHelper = new CloudTasksHelper();
private PublishDnsUpdatesAction action;
private InternetAddress outgoingRegistry;
@@ -95,6 +96,7 @@ public class PublishDnsUpdatesActionTest {
@BeforeEach
void beforeEach() throws Exception {
createTld("xn--q9jyb4c");
createTld("com");
outgoingRegistry = new InternetAddress("outgoing@registry.example");
registrySupportEmail = Lazies.of(new InternetAddress("registry@test.com"));
registryCcEmail = Lazies.of(new InternetAddress("registry-cc@test.com"));
@@ -161,7 +163,6 @@ public class PublishDnsUpdatesActionTest {
outgoingRegistry,
Optional.ofNullable(retryCount),
Optional.empty(),
dnsUtils,
new DnsWriterProxy(ImmutableMap.of("correctWriter", dnsWriter)),
dnsMetrics,
lockHandler,
@@ -195,7 +196,7 @@ public class PublishDnsUpdatesActionTest {
Duration.standardHours(2),
Duration.standardHours(1));
verifyNoMoreInteractions(dnsMetrics);
dnsUtilsHelper.assertNoMoreDnsRequests();
assertNoDnsRequests();
assertThat(response.getStatus()).isEqualTo(SC_OK);
}
@@ -222,7 +223,7 @@ public class PublishDnsUpdatesActionTest {
Duration.standardHours(2),
Duration.standardHours(1));
verifyNoMoreInteractions(dnsMetrics);
dnsUtilsHelper.assertNoMoreDnsRequests();
assertNoDnsRequests();
assertThat(response.getStatus()).isEqualTo(SC_OK);
}
@@ -275,7 +276,7 @@ public class PublishDnsUpdatesActionTest {
Duration.standardHours(2),
Duration.standardHours(1));
verifyNoMoreInteractions(dnsMetrics);
dnsUtilsHelper.assertNoMoreDnsRequests();
assertNoDnsRequests();
}
@Test
@@ -495,7 +496,7 @@ public class PublishDnsUpdatesActionTest {
Duration.standardHours(2),
Duration.standardHours(1));
verifyNoMoreInteractions(dnsMetrics);
dnsUtilsHelper.assertNoMoreDnsRequests();
assertNoDnsRequests();
}
@Test
@@ -525,7 +526,7 @@ public class PublishDnsUpdatesActionTest {
Duration.standardHours(2),
Duration.standardHours(1));
verifyNoMoreInteractions(dnsMetrics);
dnsUtilsHelper.assertNoMoreDnsRequests();
assertNoDnsRequests();
}
@Test
@@ -553,7 +554,7 @@ public class PublishDnsUpdatesActionTest {
Duration.standardHours(2),
Duration.standardHours(1));
verifyNoMoreInteractions(dnsMetrics);
dnsUtilsHelper.assertNoMoreDnsRequests();
assertNoDnsRequests();
}
@Test
@@ -579,9 +580,9 @@ public class PublishDnsUpdatesActionTest {
Duration.standardHours(2),
Duration.standardHours(1));
verifyNoMoreInteractions(dnsMetrics);
dnsUtilsHelper.assertDomainDnsRequests("example.com");
dnsUtilsHelper.assertHostDnsRequests("ns1.example.com");
dnsUtilsHelper.assertNoMoreDnsRequests();
assertDomainDnsRequests("example.com");
assertHostDnsRequests("ns1.example.com");
assertNoDnsRequestsExcept("example.com", "ns1.example.com");
}
@Test
@@ -607,9 +608,9 @@ public class PublishDnsUpdatesActionTest {
Duration.standardHours(2),
Duration.standardHours(1));
verifyNoMoreInteractions(dnsMetrics);
dnsUtilsHelper.assertDomainDnsRequests("example.com");
dnsUtilsHelper.assertHostDnsRequests("ns1.example.com");
dnsUtilsHelper.assertNoMoreDnsRequests();
assertDomainDnsRequests("example.com");
assertHostDnsRequests("ns1.example.com");
assertNoDnsRequestsExcept("example.com", "ns1.example.com");
}
@Test
@@ -631,11 +632,12 @@ public class PublishDnsUpdatesActionTest {
Duration.standardHours(2),
Duration.standardHours(1));
verifyNoMoreInteractions(dnsMetrics);
dnsUtilsHelper.assertDomainDnsRequests("example.com");
dnsUtilsHelper.assertDomainDnsRequests("example2.com");
dnsUtilsHelper.assertHostDnsRequests("ns1.example.com");
dnsUtilsHelper.assertHostDnsRequests("ns2.example.com");
dnsUtilsHelper.assertHostDnsRequests("ns1.example2.com");
dnsUtilsHelper.assertNoMoreDnsRequests();
assertDomainDnsRequests("example.com");
assertDomainDnsRequests("example2.com");
assertHostDnsRequests("ns1.example.com");
assertHostDnsRequests("ns2.example.com");
assertHostDnsRequests("ns1.example2.com");
assertNoDnsRequestsExcept(
"example.com", "example2.com", "ns1.example.com", "ns2.example.com", "ns1.example2.com");
}
}
@@ -1,500 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.dns;
import static com.google.appengine.api.taskqueue.QueueFactory.getQueue;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.Lists.transform;
import static com.google.common.truth.Truth.assertThat;
import static com.google.common.truth.Truth8.assertThat;
import static google.registry.dns.DnsConstants.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.DnsConstants.DNS_PULL_QUEUE_NAME;
import static google.registry.dns.DnsConstants.DNS_TARGET_CREATE_TIME_PARAM;
import static google.registry.dns.DnsConstants.DNS_TARGET_NAME_PARAM;
import static google.registry.dns.DnsConstants.DNS_TARGET_TYPE_PARAM;
import static google.registry.request.RequestParameters.PARAM_TLD;
import static google.registry.testing.DatabaseHelper.createTlds;
import static google.registry.testing.DatabaseHelper.persistResource;
import com.google.appengine.api.taskqueue.QueueFactory;
import com.google.appengine.api.taskqueue.TaskOptions;
import com.google.cloud.tasks.v2.HttpMethod;
import com.google.cloud.tasks.v2.Task;
import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMultimap;
import com.google.common.collect.ImmutableSet;
import com.google.common.hash.Hashing;
import com.google.common.net.InternetDomainName;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldType;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
import google.registry.testing.CloudTasksHelper;
import google.registry.testing.CloudTasksHelper.TaskMatcher;
import google.registry.testing.FakeClock;
import google.registry.testing.TaskQueueExtension;
import google.registry.testing.TaskQueueHelper;
import google.registry.testing.UriParameters;
import java.nio.charset.StandardCharsets;
import java.util.Map.Entry;
import java.util.Optional;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
import org.joda.time.DateTime;
import org.joda.time.Duration;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.extension.RegisterExtension;
import org.junitpioneer.jupiter.RetryingTest;
/** Unit tests for {@link ReadDnsQueueAction}. */
public class ReadDnsQueueActionTest {
private static final int TEST_TLD_UPDATE_BATCH_SIZE = 100;
private DnsQueue dnsQueue;
// Because of a bug in the queue test environment - b/73372999 - we must set the fake date of the
// test in the future. Set to year 3000 so it'll remain in the future for a very long time.
private final FakeClock clock = new FakeClock(DateTime.parse("3000-01-01TZ"));
private final CloudTasksHelper cloudTasksHelper = new CloudTasksHelper(clock);
@RegisterExtension
final JpaIntegrationTestExtension jpa =
new JpaTestExtensions.Builder().withClock(clock).buildIntegrationTestExtension();
@RegisterExtension final TaskQueueExtension taskQueue = new TaskQueueExtension();
@BeforeEach
void beforeEach() {
// Because of b/73372999 - the FakeClock can't be in the past, or the TaskQueues stop working.
// To make sure it's never in the past, we set the date far-far into the future
clock.setTo(DateTime.parse("3000-01-01TZ"));
createTlds("com", "net", "example", "multilock.uk");
persistResource(Tld.get("com").asBuilder().setDnsWriters(ImmutableSet.of("comWriter")).build());
persistResource(Tld.get("net").asBuilder().setDnsWriters(ImmutableSet.of("netWriter")).build());
persistResource(
Tld.get("example")
.asBuilder()
.setTldType(TldType.TEST)
.setDnsWriters(ImmutableSet.of("exampleWriter"))
.build());
persistResource(
Tld.get("multilock.uk")
.asBuilder()
.setNumDnsPublishLocks(1000)
.setDnsWriters(ImmutableSet.of("multilockWriter"))
.build());
dnsQueue = DnsQueue.createForTesting(clock);
}
private void run() {
ReadDnsQueueAction action =
new ReadDnsQueueAction(
TEST_TLD_UPDATE_BATCH_SIZE,
Duration.standardSeconds(10),
Optional.empty(),
clock,
dnsQueue,
Hashing.murmur3_32(),
cloudTasksHelper.getTestCloudTasksUtils());
// Advance the time a little, to ensure that leaseTasks() returns all tasks.
clock.advanceBy(Duration.standardHours(1));
action.run();
}
private static TaskOptions createRefreshTask(String name, TargetType type) {
TaskOptions options =
TaskOptions.Builder.withMethod(TaskOptions.Method.PULL)
.param(DNS_TARGET_TYPE_PARAM, type.toString())
.param(DNS_TARGET_NAME_PARAM, name)
.param(DNS_TARGET_CREATE_TIME_PARAM, "3000-01-01TZ");
String tld = InternetDomainName.from(name).parts().reverse().get(0);
return options.param("tld", tld);
}
private static TaskQueueHelper.TaskMatcher createDomainRefreshTaskMatcher(String name) {
return new TaskQueueHelper.TaskMatcher()
.param(DNS_TARGET_NAME_PARAM, name)
.param(DNS_TARGET_TYPE_PARAM, TargetType.DOMAIN.toString());
}
private void assertTldsEnqueuedInPushQueue(ImmutableMultimap<String, String> tldsToDnsWriters) {
// By default, the publishDnsUpdates tasks will be enqueued one hour after the update items were
// created in the pull queue. This is because of the clock.advanceBy in run()
cloudTasksHelper.assertTasksEnqueued(
DNS_PUBLISH_PUSH_QUEUE_NAME,
transform(
tldsToDnsWriters.entries().asList(),
(Entry<String, String> tldToDnsWriter) ->
new TaskMatcher()
.url(PublishDnsUpdatesAction.PATH)
.param("tld", tldToDnsWriter.getKey())
.param("dnsWriter", tldToDnsWriter.getValue())
.param("requestTime", "3000-01-01T00:00:00.000Z")
.param("enqueued", "3000-01-01T01:00:00.000Z")
// Single-lock TLDs should use lock 1 of 1 by default
.param("lockIndex", "1")
.param("numPublishLocks", "1")
.header("content-type", "application/x-www-form-urlencoded")));
}
@RetryingTest(4)
void testSuccess_methodPostIsDefault() {
dnsQueue.addDomainRefreshTask("domain.com");
dnsQueue.addDomainRefreshTask("domain.net");
dnsQueue.addDomainRefreshTask("domain.example");
run();
TaskQueueHelper.assertNoTasksEnqueued(DNS_PULL_QUEUE_NAME);
cloudTasksHelper.assertTasksEnqueued(
DNS_PUBLISH_PUSH_QUEUE_NAME,
new TaskMatcher().method(HttpMethod.POST),
new TaskMatcher().method(HttpMethod.POST),
new TaskMatcher().method(HttpMethod.POST));
}
@RetryingTest(4)
void testSuccess_allSingleLockTlds() {
dnsQueue.addDomainRefreshTask("domain.com");
dnsQueue.addDomainRefreshTask("domain.net");
dnsQueue.addDomainRefreshTask("domain.example");
run();
TaskQueueHelper.assertNoTasksEnqueued(DNS_PULL_QUEUE_NAME);
assertTldsEnqueuedInPushQueue(
ImmutableMultimap.of("com", "comWriter", "net", "netWriter", "example", "exampleWriter"));
}
@RetryingTest(4)
void testSuccess_moreUpdatesThanQueueBatchSize() {
// The task queue has a batch size of 1000 (that's the maximum number of items you can lease at
// once).
ImmutableList<String> domains =
IntStream.range(0, 1500)
.mapToObj(i -> String.format("domain_%04d.com", i))
.collect(toImmutableList());
domains.forEach(dnsQueue::addDomainRefreshTask);
run();
TaskQueueHelper.assertNoTasksEnqueued(DNS_PULL_QUEUE_NAME);
ImmutableList<Task> queuedTasks =
ImmutableList.copyOf(cloudTasksHelper.getTestTasksFor(DNS_PUBLISH_PUSH_QUEUE_NAME));
// ReadDnsQueueAction batches items per TLD in batches of size 100.
// So for 1500 items in the DNS queue, we expect 15 items in the push queue
assertThat(queuedTasks).hasSize(15);
// Check all the expected domains are indeed enqueued
assertThat(
queuedTasks.stream()
.flatMap(
task ->
UriParameters.parse(
task.getAppEngineHttpRequest()
.getBody()
.toString(StandardCharsets.UTF_8))
.get("domains")
.stream())
.flatMap(values -> Splitter.on(',').splitToStream(values)))
.containsExactlyElementsIn(domains);
}
@RetryingTest(4)
void testSuccess_twoDnsWriters() {
persistResource(
Tld.get("com")
.asBuilder()
.setDnsWriters(ImmutableSet.of("comWriter", "otherWriter"))
.build());
dnsQueue.addDomainRefreshTask("domain.com");
run();
TaskQueueHelper.assertNoTasksEnqueued(DNS_PULL_QUEUE_NAME);
assertTldsEnqueuedInPushQueue(ImmutableMultimap.of("com", "comWriter", "com", "otherWriter"));
}
@RetryingTest(4)
void testSuccess_differentUpdateTimes_usesMinimum() {
clock.setTo(DateTime.parse("3000-02-03TZ"));
dnsQueue.addDomainRefreshTask("domain1.com");
clock.setTo(DateTime.parse("3000-02-04TZ"));
dnsQueue.addDomainRefreshTask("domain2.com");
clock.setTo(DateTime.parse("3000-02-05TZ"));
dnsQueue.addDomainRefreshTask("domain3.com");
run();
TaskQueueHelper.assertNoTasksEnqueued(DNS_PULL_QUEUE_NAME);
cloudTasksHelper.assertTasksEnqueued(
DNS_PUBLISH_PUSH_QUEUE_NAME,
new TaskMatcher()
.param("enqueued", "3000-02-05T01:00:00.000Z")
.param("requestTime", "3000-02-03T00:00:00.000Z")
.param("tld", "com")
.param("dnsWriter", "comWriter")
.param("domains", "domain1.com,domain2.com,domain3.com")
.param("hosts", "")
.param("lockIndex", "1")
.param("numPublishLocks", "1"));
}
@RetryingTest(4)
void testSuccess_oneTldPaused_returnedToQueue() {
persistResource(Tld.get("net").asBuilder().setDnsPaused(true).build());
dnsQueue.addDomainRefreshTask("domain.com");
dnsQueue.addDomainRefreshTask("domain.net");
dnsQueue.addDomainRefreshTask("domain.example");
run();
TaskQueueHelper.assertTasksEnqueued(
DNS_PULL_QUEUE_NAME, createDomainRefreshTaskMatcher("domain.net"));
assertTldsEnqueuedInPushQueue(
ImmutableMultimap.of("com", "comWriter", "example", "exampleWriter"));
}
@RetryingTest(4)
void testSuccess_oneTldUnknown_returnedToQueue() {
dnsQueue.addDomainRefreshTask("domain.com");
dnsQueue.addDomainRefreshTask("domain.example");
QueueFactory.getQueue(DNS_PULL_QUEUE_NAME)
.add(
TaskOptions.Builder.withDefaults()
.method(TaskOptions.Method.PULL)
.param(DNS_TARGET_TYPE_PARAM, TargetType.DOMAIN.toString())
.param(DNS_TARGET_NAME_PARAM, "domain.unknown")
.param(DNS_TARGET_CREATE_TIME_PARAM, "3000-01-01TZ")
.param(PARAM_TLD, "unknown"));
run();
TaskQueueHelper.assertTasksEnqueued(
DNS_PULL_QUEUE_NAME, createDomainRefreshTaskMatcher("domain.unknown"));
assertTldsEnqueuedInPushQueue(
ImmutableMultimap.of("com", "comWriter", "example", "exampleWriter"));
}
@RetryingTest(4)
void testSuccess_corruptTaskTldMismatch_published() {
// TODO(mcilwain): what's the correct action to take in this case?
dnsQueue.addDomainRefreshTask("domain.com");
dnsQueue.addDomainRefreshTask("domain.example");
QueueFactory.getQueue(DNS_PULL_QUEUE_NAME)
.add(
TaskOptions.Builder.withDefaults()
.method(TaskOptions.Method.PULL)
.param(DNS_TARGET_TYPE_PARAM, TargetType.DOMAIN.toString())
.param(DNS_TARGET_NAME_PARAM, "domain.wrongtld")
.param(DNS_TARGET_CREATE_TIME_PARAM, "3000-01-01TZ")
.param(PARAM_TLD, "net"));
run();
TaskQueueHelper.assertNoTasksEnqueued(DNS_PULL_QUEUE_NAME);
assertTldsEnqueuedInPushQueue(
ImmutableMultimap.of("com", "comWriter", "example", "exampleWriter", "net", "netWriter"));
}
@RetryingTest(4)
void testSuccess_corruptTaskNoTld_discarded() {
dnsQueue.addDomainRefreshTask("domain.com");
dnsQueue.addDomainRefreshTask("domain.example");
QueueFactory.getQueue(DNS_PULL_QUEUE_NAME)
.add(
TaskOptions.Builder.withDefaults()
.method(TaskOptions.Method.PULL)
.param(DNS_TARGET_TYPE_PARAM, TargetType.DOMAIN.toString())
.param(DNS_TARGET_NAME_PARAM, "domain.net"));
run();
// The corrupt task isn't in the pull queue, but also isn't in the push queue
TaskQueueHelper.assertNoTasksEnqueued(DNS_PULL_QUEUE_NAME);
assertTldsEnqueuedInPushQueue(
ImmutableMultimap.of("com", "comWriter", "example", "exampleWriter"));
}
@RetryingTest(4)
void testSuccess_corruptTaskNoName_discarded() {
dnsQueue.addDomainRefreshTask("domain.com");
dnsQueue.addDomainRefreshTask("domain.example");
QueueFactory.getQueue(DNS_PULL_QUEUE_NAME)
.add(
TaskOptions.Builder.withDefaults()
.method(TaskOptions.Method.PULL)
.param(DNS_TARGET_TYPE_PARAM, TargetType.DOMAIN.toString())
.param(PARAM_TLD, "net"));
run();
// The corrupt task isn't in the pull queue, but also isn't in the push queue
TaskQueueHelper.assertNoTasksEnqueued(DNS_PULL_QUEUE_NAME);
assertTldsEnqueuedInPushQueue(
ImmutableMultimap.of("com", "comWriter", "example", "exampleWriter"));
}
@RetryingTest(4)
void testSuccess_corruptTaskNoType_discarded() {
dnsQueue.addDomainRefreshTask("domain.com");
dnsQueue.addDomainRefreshTask("domain.example");
QueueFactory.getQueue(DNS_PULL_QUEUE_NAME)
.add(
TaskOptions.Builder.withDefaults()
.method(TaskOptions.Method.PULL)
.param(DNS_TARGET_NAME_PARAM, "domain.net")
.param(PARAM_TLD, "net"));
run();
// The corrupt task isn't in the pull queue, but also isn't in the push queue
TaskQueueHelper.assertNoTasksEnqueued(DNS_PULL_QUEUE_NAME);
assertTldsEnqueuedInPushQueue(
ImmutableMultimap.of("com", "comWriter", "example", "exampleWriter"));
}
@RetryingTest(4)
void testSuccess_corruptTaskWrongType_discarded() {
dnsQueue.addDomainRefreshTask("domain.com");
dnsQueue.addDomainRefreshTask("domain.example");
QueueFactory.getQueue(DNS_PULL_QUEUE_NAME)
.add(
TaskOptions.Builder.withDefaults()
.method(TaskOptions.Method.PULL)
.param(DNS_TARGET_TYPE_PARAM, "Wrong type")
.param(DNS_TARGET_NAME_PARAM, "domain.net")
.param(PARAM_TLD, "net"));
run();
// The corrupt task isn't in the pull queue, but also isn't in the push queue
TaskQueueHelper.assertNoTasksEnqueued(DNS_PULL_QUEUE_NAME);
assertTldsEnqueuedInPushQueue(
ImmutableMultimap.of("com", "comWriter", "example", "exampleWriter"));
}
private static String makeCommaSeparatedRange(int from, int to, String format) {
return IntStream.range(from, to)
.mapToObj(i -> String.format(format, i))
.collect(Collectors.joining(","));
}
@RetryingTest(4)
void testSuccess_manyDomainsAndHosts() {
for (int i = 0; i < 150; i++) {
// 0: domain; 1: host 1; 2: host 2
for (int thingType = 0; thingType < 3; thingType++) {
for (String tld : ImmutableList.of("com", "net")) {
String domainName = String.format("domain%04d.%s", i, tld);
switch (thingType) {
case 1:
getQueue(DNS_PULL_QUEUE_NAME)
.add(createRefreshTask("ns1." + domainName, TargetType.HOST));
break;
case 2:
getQueue(DNS_PULL_QUEUE_NAME)
.add(createRefreshTask("ns2." + domainName, TargetType.HOST));
break;
default:
dnsQueue.addDomainRefreshTask(domainName);
break;
}
}
}
}
run();
TaskQueueHelper.assertNoTasksEnqueued(DNS_PULL_QUEUE_NAME);
cloudTasksHelper.assertTasksEnqueued(
DNS_PUBLISH_PUSH_QUEUE_NAME,
new TaskMatcher()
.url(PublishDnsUpdatesAction.PATH)
.param("domains", makeCommaSeparatedRange(0, 100, "domain%04d.com"))
.param("hosts", ""),
new TaskMatcher()
.url(PublishDnsUpdatesAction.PATH)
.param("domains", makeCommaSeparatedRange(100, 150, "domain%04d.com"))
.param("hosts", makeCommaSeparatedRange(0, 50, "ns1.domain%04d.com")),
new TaskMatcher()
.url(PublishDnsUpdatesAction.PATH)
.param("domains", "")
.param("hosts", makeCommaSeparatedRange(50, 150, "ns1.domain%04d.com")),
new TaskMatcher()
.url(PublishDnsUpdatesAction.PATH)
.param("domains", "")
.param("hosts", makeCommaSeparatedRange(0, 100, "ns2.domain%04d.com")),
new TaskMatcher()
.url(PublishDnsUpdatesAction.PATH)
.param("domains", "")
.param("hosts", makeCommaSeparatedRange(100, 150, "ns2.domain%04d.com")),
new TaskMatcher()
.url(PublishDnsUpdatesAction.PATH)
.param("domains", makeCommaSeparatedRange(0, 100, "domain%04d.net"))
.param("hosts", ""),
new TaskMatcher()
.url(PublishDnsUpdatesAction.PATH)
.param("domains", makeCommaSeparatedRange(100, 150, "domain%04d.net"))
.param("hosts", makeCommaSeparatedRange(0, 50, "ns1.domain%04d.net")),
new TaskMatcher()
.url(PublishDnsUpdatesAction.PATH)
.param("domains", "")
.param("hosts", makeCommaSeparatedRange(50, 150, "ns1.domain%04d.net")),
new TaskMatcher()
.url(PublishDnsUpdatesAction.PATH)
.param("domains", "")
.param("hosts", makeCommaSeparatedRange(0, 100, "ns2.domain%04d.net")),
new TaskMatcher()
.url(PublishDnsUpdatesAction.PATH)
.param("domains", "")
.param("hosts", makeCommaSeparatedRange(100, 150, "ns2.domain%04d.net")));
}
@RetryingTest(4)
void testSuccess_lockGroupsHostBySuperordinateDomain() {
dnsQueue.addDomainRefreshTask("hello.multilock.uk");
dnsQueue.addHostRefreshTask("ns1.abc.hello.multilock.uk");
dnsQueue.addHostRefreshTask("ns2.hello.multilock.uk");
dnsQueue.addDomainRefreshTask("another.multilock.uk");
dnsQueue.addHostRefreshTask("ns3.def.another.multilock.uk");
dnsQueue.addHostRefreshTask("ns4.another.multilock.uk");
run();
TaskQueueHelper.assertNoTasksEnqueued(DNS_PULL_QUEUE_NAME);
// Expect two different groups; in-balliwick hosts are locked with their superordinate domains.
cloudTasksHelper.assertTasksEnqueued(
DNS_PUBLISH_PUSH_QUEUE_NAME,
new TaskMatcher()
.url(PublishDnsUpdatesAction.PATH)
.param("tld", "multilock.uk")
.param("dnsWriter", "multilockWriter")
.param("requestTime", "3000-01-01T00:00:00.000Z")
.param("enqueued", "3000-01-01T01:00:00.000Z")
.param("domains", "hello.multilock.uk")
.param("hosts", "ns1.abc.hello.multilock.uk,ns2.hello.multilock.uk")
.header("content-type", "application/x-www-form-urlencoded"),
new TaskMatcher()
.url(PublishDnsUpdatesAction.PATH)
.param("tld", "multilock.uk")
.param("dnsWriter", "multilockWriter")
.param("requestTime", "3000-01-01T00:00:00.000Z")
.param("enqueued", "3000-01-01T01:00:00.000Z")
.param("domains", "another.multilock.uk")
.param("hosts", "ns3.def.another.multilock.uk,ns4.another.multilock.uk")
.header("content-type", "application/x-www-form-urlencoded"));
}
}
@@ -15,10 +15,6 @@
package google.registry.dns;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.model.common.DatabaseMigrationStateSchedule.MigrationState;
import static google.registry.model.common.DatabaseMigrationStateSchedule.set;
import static google.registry.model.common.DatabaseMigrationStateSchedule.useUncachedForTest;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.testing.DatabaseHelper.createTld;
import static google.registry.testing.DatabaseHelper.loadAllOf;
import static google.registry.testing.DatabaseHelper.persistResource;
@@ -38,9 +34,7 @@ import static org.mockito.Mockito.verify;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedMap;
import com.google.common.collect.Ordering;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.dns.DnsUtils.TargetType;
import google.registry.model.common.DnsRefreshRequest;
import google.registry.model.common.DnsRefreshRequestTest;
import google.registry.model.tld.Tld;
@@ -54,7 +48,6 @@ import java.util.Optional;
import org.joda.time.DateTime;
import org.joda.time.DateTimeZone;
import org.joda.time.Duration;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
@@ -65,7 +58,6 @@ public class ReadDnsRefreshRequestsActionTest {
private final FakeClock clock = new FakeClock(DateTime.parse("2020-02-02T01:23:45Z"));
private final CloudTasksHelper cloudTasksHelper = new CloudTasksHelper(clock);
private final DnsUtils dnsUtils = new DnsUtils(null);
private final Optional<Integer> jitterSeconds = Optional.of(5);
@RegisterExtension
@@ -80,20 +72,13 @@ public class ReadDnsRefreshRequestsActionTest {
jitterSeconds,
"tld",
clock,
dnsUtils,
null,
cloudTasksHelper.getTestCloudTasksUtils()));
private ImmutableList<DnsRefreshRequest> requests;
@BeforeAll
static void beforeAll() {
useUncachedForTest();
}
@BeforeEach
void beforeEach() {
useDnsSql();
persistResource(
createTld("tld")
.asBuilder()
@@ -271,26 +256,4 @@ public class ReadDnsRefreshRequestsActionTest {
.isAtMost(Duration.standardSeconds(jitterSeconds.get()));
});
}
private void useDnsSql() {
DateTime currentTime = clock.nowUtc();
clock.setTo(START_OF_TIME);
tm().transact(
() ->
set(
new ImmutableSortedMap.Builder<DateTime, MigrationState>(Ordering.natural())
.put(START_OF_TIME, MigrationState.DATASTORE_ONLY)
.put(START_OF_TIME.plusMillis(1), MigrationState.DATASTORE_PRIMARY)
.put(START_OF_TIME.plusMillis(2), MigrationState.DATASTORE_PRIMARY_NO_ASYNC)
.put(
START_OF_TIME.plusMillis(3), MigrationState.DATASTORE_PRIMARY_READ_ONLY)
.put(START_OF_TIME.plusMillis(4), MigrationState.SQL_PRIMARY_READ_ONLY)
.put(START_OF_TIME.plusMillis(5), MigrationState.SQL_PRIMARY)
.put(START_OF_TIME.plusMillis(6), MigrationState.SQL_ONLY)
.put(START_OF_TIME.plusMillis(7), MigrationState.SEQUENCE_BASED_ALLOCATE_ID)
.put(START_OF_TIME.plusMillis(8), MigrationState.NORDN_SQL)
.put(START_OF_TIME.plusMillis(9), MigrationState.DNS_SQL)
.build()));
clock.setTo(currentTime);
}
}
@@ -15,20 +15,22 @@
package google.registry.dns;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.testing.DatabaseHelper.assertDomainDnsRequests;
import static google.registry.testing.DatabaseHelper.assertHostDnsRequests;
import static google.registry.testing.DatabaseHelper.assertNoDnsRequests;
import static google.registry.testing.DatabaseHelper.assertNoDnsRequestsExcept;
import static google.registry.testing.DatabaseHelper.createTld;
import static google.registry.testing.DatabaseHelper.persistActiveDomain;
import static google.registry.testing.DatabaseHelper.persistActiveHost;
import static google.registry.testing.DatabaseHelper.persistActiveSubordinateHost;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.mockito.Mockito.mock;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.dns.DnsUtils.TargetType;
import google.registry.model.domain.Domain;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
import google.registry.request.HttpException.BadRequestException;
import google.registry.request.HttpException.NotFoundException;
import google.registry.testing.DnsUtilsHelper;
import google.registry.testing.FakeClock;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
@@ -41,12 +43,10 @@ public class RefreshDnsActionTest {
final JpaIntegrationTestExtension jpa =
new JpaTestExtensions.Builder().buildIntegrationTestExtension();
private final DnsUtils dnsUtils = mock(DnsUtils.class);
private final DnsUtilsHelper dnsUtilsHelper = new DnsUtilsHelper(dnsUtils);
private final FakeClock clock = new FakeClock();
private void run(TargetType type, String name) {
new RefreshDnsAction(name, type, clock, dnsUtils).run();
new RefreshDnsAction(name, type, clock).run();
}
@BeforeEach
@@ -58,9 +58,9 @@ public class RefreshDnsActionTest {
void testSuccess_host() {
Domain domain = persistActiveDomain("example.xn--q9jyb4c");
persistActiveSubordinateHost("ns1.example.xn--q9jyb4c", domain);
run(TargetType.HOST, "ns1.example.xn--q9jyb4c");
dnsUtilsHelper.assertHostDnsRequests("ns1.example.xn--q9jyb4c");
dnsUtilsHelper.assertNoMoreDnsRequests();
run(DnsUtils.TargetType.HOST, "ns1.example.xn--q9jyb4c");
assertHostDnsRequests("ns1.example.xn--q9jyb4c");
assertNoDnsRequestsExcept("ns1.example.xn--q9jyb4c");
}
@Test
@@ -72,9 +72,9 @@ public class RefreshDnsActionTest {
BadRequestException.class,
() -> {
try {
run(TargetType.HOST, "ns1.example.xn--q9jyb4c");
run(DnsUtils.TargetType.HOST, "ns1.example.xn--q9jyb4c");
} finally {
dnsUtilsHelper.assertNoMoreDnsRequests();
assertNoDnsRequests();
}
});
assertThat(thrown)
@@ -85,23 +85,25 @@ public class RefreshDnsActionTest {
@Test
void testSuccess_domain() {
persistActiveDomain("example.xn--q9jyb4c");
run(TargetType.DOMAIN, "example.xn--q9jyb4c");
dnsUtilsHelper.assertDomainDnsRequests("example.xn--q9jyb4c");
dnsUtilsHelper.assertNoMoreDnsRequests();
run(DnsUtils.TargetType.DOMAIN, "example.xn--q9jyb4c");
assertDomainDnsRequests("example.xn--q9jyb4c");
assertNoDnsRequestsExcept("example.xn--q9jyb4c");
}
@Test
void testFailure_unqualifiedName() {
assertThrows(BadRequestException.class, () -> run(TargetType.DOMAIN, "example"));
assertThrows(BadRequestException.class, () -> run(DnsUtils.TargetType.DOMAIN, "example"));
}
@Test
void testFailure_hostDoesNotExist() {
assertThrows(NotFoundException.class, () -> run(TargetType.HOST, "ns1.example.xn--q9jyb4c"));
assertThrows(
NotFoundException.class, () -> run(DnsUtils.TargetType.HOST, "ns1.example.xn--q9jyb4c"));
}
@Test
void testFailure_domainDoesNotExist() {
assertThrows(NotFoundException.class, () -> run(TargetType.DOMAIN, "example.xn--q9jyb4c"));
assertThrows(
NotFoundException.class, () -> run(DnsUtils.TargetType.DOMAIN, "example.xn--q9jyb4c"));
}
}
@@ -15,6 +15,8 @@
package google.registry.dns;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.testing.DatabaseHelper.assertDomainDnsRequests;
import static google.registry.testing.DatabaseHelper.assertNoDnsRequests;
import static google.registry.testing.DatabaseHelper.createTld;
import static google.registry.testing.DatabaseHelper.newDomain;
import static google.registry.testing.DatabaseHelper.persistActiveHost;
@@ -23,14 +25,12 @@ import static google.registry.testing.DatabaseHelper.persistDomainAsDeleted;
import static google.registry.testing.DatabaseHelper.persistResource;
import static javax.servlet.http.HttpServletResponse.SC_NO_CONTENT;
import static javax.servlet.http.HttpServletResponse.SC_OK;
import static org.mockito.Mockito.mock;
import com.google.common.collect.ImmutableSet;
import google.registry.model.eppcommon.StatusValue;
import google.registry.model.host.Host;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
import google.registry.testing.DnsUtilsHelper;
import google.registry.testing.FakeClock;
import google.registry.testing.FakeResponse;
import org.joda.time.DateTime;
@@ -42,8 +42,6 @@ import org.junit.jupiter.api.extension.RegisterExtension;
public class RefreshDnsOnHostRenameActionTest {
private final FakeClock clock = new FakeClock(DateTime.parse("2015-01-15T11:22:33Z"));
private final DnsUtils dnsUtils = mock(DnsUtils.class);
private final DnsUtilsHelper dnsUtilsHelper = new DnsUtilsHelper(dnsUtils);
private final FakeResponse response = new FakeResponse();
@RegisterExtension
@@ -53,7 +51,7 @@ public class RefreshDnsOnHostRenameActionTest {
private RefreshDnsOnHostRenameAction action;
private void createAction(String hostKey) {
action = new RefreshDnsOnHostRenameAction(hostKey, response, dnsUtils);
action = new RefreshDnsOnHostRenameAction(hostKey, response);
}
@BeforeEach
@@ -75,7 +73,7 @@ public class RefreshDnsOnHostRenameActionTest {
persistDomainAsDeleted(newDomain("deleted.tld", host), clock.nowUtc().minusDays(1));
createAction(host.createVKey().stringify());
action.run();
dnsUtilsHelper.assertDomainDnsRequests("example.tld", "otherexample.tld");
assertDomainDnsRequests("example.tld", "otherexample.tld");
assertThat(response.getStatus()).isEqualTo(SC_OK);
}
@@ -83,7 +81,7 @@ public class RefreshDnsOnHostRenameActionTest {
void testFailure_nonexistentHost() {
createAction("kind:Host@sql:rO0ABXQABGJsYWg");
action.run();
dnsUtilsHelper.assertNoMoreDnsRequests();
assertNoDnsRequests();
assertThat(response.getStatus()).isEqualTo(SC_NO_CONTENT);
assertThat(response.getPayload())
.isEqualTo("Host to refresh does not exist: VKey<Host>(sql:blah)");
@@ -95,7 +93,7 @@ public class RefreshDnsOnHostRenameActionTest {
persistResource(newDomain("example.tld", host));
createAction(host.createVKey().stringify());
action.run();
dnsUtilsHelper.assertNoMoreDnsRequests();
assertNoDnsRequests();
assertThat(response.getStatus()).isEqualTo(SC_NO_CONTENT);
assertThat(response.getPayload())
.isEqualTo("Host to refresh is already deleted: ns1.example.tld");
@@ -38,9 +38,8 @@ import com.google.common.collect.Ordering;
import com.google.common.truth.Truth;
import com.google.re2j.Matcher;
import com.google.re2j.Pattern;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.reporting.HistoryEntry.Type;
@@ -48,7 +47,6 @@ import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldState;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
import google.registry.testing.TaskQueueExtension;
import google.registry.tmch.TmchData;
import google.registry.tmch.TmchTestData;
import org.joda.money.Money;
@@ -73,8 +71,6 @@ class EppLifecycleDomainTest extends EppTestCase {
final JpaIntegrationTestExtension jpa =
new JpaTestExtensions.Builder().withClock(clock).buildIntegrationTestExtension();
@RegisterExtension final TaskQueueExtension taskQueue = new TaskQueueExtension();
@BeforeEach
void beforeEach() {
createTlds("example", "tld");
@@ -361,24 +357,22 @@ class EppLifecycleDomainTest extends EppTestCase {
"CODE", "2303", "MSG", "The domain with given ID (example.tld) doesn't exist."));
// The expected one-time billing event, that should have an associated Cancellation.
OneTime oneTimeCreateBillingEvent = makeOneTimeCreateBillingEvent(domain, createTime);
OneTime oneTimeRenewBillingEvent = makeOneTimeRenewBillingEvent(domain, renewTime);
BillingEvent createBillingEvent = makeOneTimeCreateBillingEvent(domain, createTime);
BillingEvent renewBillingEvent = makeOneTimeRenewBillingEvent(domain, renewTime);
// Verify that the OneTime billing event associated with the domain creation is canceled.
assertBillingEventsForResource(
domain,
// There should be one-time billing events for the create and the renew.
oneTimeCreateBillingEvent,
oneTimeRenewBillingEvent,
createBillingEvent,
renewBillingEvent,
// There should be two ended recurring billing events, one each from the create and renew.
// (The former was ended by the renew and the latter was ended by the delete.)
makeRecurringCreateBillingEvent(domain, createTime.plusYears(2), renewTime),
makeRecurringRenewBillingEvent(domain, createTime.plusYears(5), deleteTime),
makeCreateRecurrence(domain, createTime.plusYears(2), renewTime),
makeRenewRecurrence(domain, createTime.plusYears(5), deleteTime),
// There should be Cancellations offsetting both of the one-times.
makeCancellationBillingEventForCreate(
domain, oneTimeCreateBillingEvent, createTime, deleteTime),
makeCancellationBillingEventForRenew(
domain, oneTimeRenewBillingEvent, renewTime, deleteTime));
makeCancellationBillingEventForCreate(domain, createBillingEvent, createTime, deleteTime),
makeCancellationBillingEventForRenew(domain, renewBillingEvent, renewTime, deleteTime));
// Verify that the registration expiration time was set back to the creation time, because the
// entire cost of registration was refunded. We have to do this through the DB instead of EPP
@@ -424,16 +418,15 @@ class EppLifecycleDomainTest extends EppTestCase {
"CODE", "2303", "MSG", "The domain with given ID (example.tld) doesn't exist."));
// The expected one-time billing event, that should have an associated Cancellation.
OneTime oneTimeCreateBillingEvent = makeOneTimeCreateBillingEvent(domain, createTime);
BillingEvent createBillingEvent = makeOneTimeCreateBillingEvent(domain, createTime);
// Verify that the OneTime billing event associated with the domain creation is canceled.
assertBillingEventsForResource(
domain,
// Check the existence of the expected create one-time billing event.
oneTimeCreateBillingEvent,
makeRecurringCreateBillingEvent(domain, createTime.plusYears(2), deleteTime),
createBillingEvent,
makeCreateRecurrence(domain, createTime.plusYears(2), deleteTime),
// Check for the existence of a cancellation for the given one-time billing event.
makeCancellationBillingEventForCreate(
domain, oneTimeCreateBillingEvent, createTime, deleteTime));
makeCancellationBillingEventForCreate(domain, createBillingEvent, createTime, deleteTime));
// Verify that the registration expiration time was set back to the creation time, because the
// entire cost of registration was refunded. We have to do this through the DB instead of EPP
@@ -495,7 +488,7 @@ class EppLifecycleDomainTest extends EppTestCase {
assertBillingEventsForResource(
domain,
makeOneTimeCreateBillingEvent(domain, createTime),
makeRecurringCreateBillingEvent(domain, createTime.plusYears(2), deleteTime));
makeCreateRecurrence(domain, createTime.plusYears(2), deleteTime));
assertThatLogoutSucceeds();
@@ -541,8 +534,8 @@ class EppLifecycleDomainTest extends EppTestCase {
// Verify that the OneTime billing event associated with the base fee of domain registration and
// is canceled and the autorenew is ended, but that the EAP fee is not canceled.
OneTime expectedCreateEapBillingEvent =
new BillingEvent.OneTime.Builder()
BillingEvent expectedCreateEapBillingEvent =
new BillingEvent.Builder()
.setReason(Reason.FEE_EARLY_ACCESS)
.setTargetId("example.tld")
.setRegistrarId("NewRegistrar")
@@ -555,17 +548,17 @@ class EppLifecycleDomainTest extends EppTestCase {
.build();
// The expected one-time billing event, that should have an associated Cancellation.
OneTime expectedOneTimeCreateBillingEvent = makeOneTimeCreateBillingEvent(domain, createTime);
BillingEvent expectedCreateBillingEvent = makeOneTimeCreateBillingEvent(domain, createTime);
assertBillingEventsForResource(
domain,
// Check for the expected create one-time billing event ...
expectedOneTimeCreateBillingEvent,
expectedCreateBillingEvent,
// ... and the expected one-time EAP fee billing event ...
expectedCreateEapBillingEvent,
makeRecurringCreateBillingEvent(domain, createTime.plusYears(2), deleteTime),
makeCreateRecurrence(domain, createTime.plusYears(2), deleteTime),
// ... and verify that the create one-time billing event was canceled ...
makeCancellationBillingEventForCreate(
domain, expectedOneTimeCreateBillingEvent, createTime, deleteTime));
domain, expectedCreateBillingEvent, createTime, deleteTime));
// ... but there was NOT a Cancellation for the EAP fee, as this would fail if additional
// billing events were present.

Some files were not shown because too many files have changed in this diff Show More