1
0
mirror of https://github.com/google/nomulus synced 2026-07-12 02:52:38 +00:00

Compare commits

...

25 Commits

Author SHA1 Message Date
sarahcaseybot acdbc65c51 Change Registry object reference to Tld in configuration.md (#2021) 2023-05-12 12:32:02 -04:00
Weimin Yu d510531f65 Remove the deprecatd DefaultCredential (#2032)
Use the ApplicationDefaultCredential annotation instead.

The new annotation has been verified in sandbox and production using the
'executeCannedScript' endpoint. The verification code is removed in this
PR too.
2023-05-11 13:46:36 -04:00
Lai Jiang 0d4dd57fe7 Fix a typo (#2031) 2023-05-11 13:26:07 -04:00
Pavlo Tkach 2667a0e977 Expand nomulus get_domain command to load up deleted domain data too (#2018) 2023-05-10 16:05:03 -04:00
gbrodman 1aef31efff Allow usage of standard HTTP requests in CloudTasksUtils (#2013)
This adds a possible configuration point "defaultServiceAccount" (which
in GAE will be the standard GAE service account). If this is configured,
CloudTasksUtils can create tasks with standard HTTP requests with an
OIDC token corresponding to that service account, as opposed to using
the AppEngine-specific request methods.

This also works with IAP, in that if IAP is on and we specify the IAP
client ID in the config, CloudTasksUtils will use the IAP client ID as
the token audience and the request will successfully be passed through
the IAP layer.

Tetsted in QA.
2023-05-09 16:02:12 -04:00
Lai Jiang 4d19245c29 Change usage grouping key in the invoice CSV (#2024)
This column is used by the billing team to create invoices. Registrars
have asked that a single invoice be created for a given registrar,
instead of one per registrar-tld pair. This should have no other effect
on the billing pipeline as the invoice grouping key has a description
field that also contains the TLD, so the granularity as a whole does not
change.
2023-05-09 11:25:11 -04:00
Lai Jiang 4b34307a6e Delete DatabaseMigrationStateSchedule (#2001)
We have been using it as a poor man's timed flag that triggers a system
behavior change after a certain time. We have no foreseeable future use
for it now that the DNS pull queue related code is deleted. If in the
future a need for such a flag arises, we are better off implementing a
proper flag system than hijacking this class any way.
2023-05-08 14:36:28 -04:00
Pavlo Tkach 55243e7cf6 Adds cloud scheduler and tasks deployer (#1999) 2023-05-04 15:57:32 -04:00
Lai Jiang e14764b4c8 Remove DNS pull queue (#2000)
This is the last dependency on GAE pull queue, therefore we can delete
the pull queue config from queue.xml as well.
2023-05-04 13:21:53 -04:00
dependabot[bot] 68810f7a30 Bump engine.io and socket.io in /console-webapp (#2022)
Bumps [engine.io](https://github.com/socketio/engine.io) and [socket.io](https://github.com/socketio/socket.io). These dependencies needed to be updated together.

Updates `engine.io` from 6.2.1 to 6.4.2
- [Release notes](https://github.com/socketio/engine.io/releases)
- [Changelog](https://github.com/socketio/engine.io/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/engine.io/compare/6.2.1...6.4.2)

Updates `socket.io` from 4.5.2 to 4.6.1
- [Release notes](https://github.com/socketio/socket.io/releases)
- [Changelog](https://github.com/socketio/socket.io/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/socket.io/compare/4.5.2...4.6.1)

---
updated-dependencies:
- dependency-name: engine.io
  dependency-type: indirect
- dependency-name: socket.io
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-04 12:50:19 -04:00
Ben McIlwain 14d245b1e3 Remove duplicate info from create/update reserved list command output (#2020)
It was repeating the domain label twice for every reserved list entry. It used
to look like this:

baddies=baddies,FULLY_BLOCKED
2023-05-03 17:31:23 -04:00
Weimin Yu 61ab29ae9e Prober ssl cert update automation (#2019)
Defined CloudBuild script and docker image that automatically
updates probers' SSL certs
2023-05-03 15:57:50 -04:00
Weimin Yu 6742e5bf23 Remove CloudSql wipeout cron job in crash (#2017)
No more production data in crash. This allows us to repopulate crash
with test data.
2023-05-02 14:44:09 -04:00
Weimin Yu c7f69eba1d Prepare switch of credential annotation (#2014)
* Prepare switch of credential annotation

Prepare the switch from DefaultCredential to ApplicationCredential.

In nomulus tools, start using the new annotation. This is tested by
successfully using the nomulus curl command, which actually needs a
valid credential to work.

For remaining use cases of the old annotation in Nomulus server, add
some code that relies on the new credential to work. Once these code
are tested in sandbox and production, we will switch the annotations.
2023-05-01 11:23:19 -04:00
gbrodman 578988d5ea Don't allow a list of the empty string in List<String> fields (#2011)
If the user does, e.g. `--allowed_nameservers=` (or contact ids) that
shouldn't mean a list consisting solely of the empty string.

Using this parameter / converter allows us to ensure that lists of
strings look reasonable.
2023-04-28 17:59:17 -04:00
sarahcaseybot c17b8285f9 Don't apply non-premium default tokens to premium names (#2007)
* Don't apply non-premium default tokens to premium names

* Add test for renew

* Remove premium check from try/catch block

* Add check in validateToken

* Update docs

* Add validateForPremiums

* Better method name

* Shorten error message to fit as reason

* Add missing extension catch

* Remove extra javadoc

* Fix merge conflicts and change error message

* Update flow docs
2023-04-28 17:56:15 -04:00
gbrodman ff8a08f40e Fix typo in pipeline name (#2016) 2023-04-28 17:05:24 -04:00
gbrodman a341058282 Refactor / rename Billing object classes (#1993)
This includes renaming the billing classes to match the SQL table names,
as well as splitting them out into their own separate top-level classes.
The rest of the changes are mostly renaming variables and comments etc.

We now use `BillingBase` as the name of the common billing superclass,
because one-time events are called BillingEvents
2023-04-28 14:27:37 -04:00
Weimin Yu 16758879f0 Allow rotation when updating registrar cert (#2012)
* Allow rotation when updating registrar cert

When updating a registrar's primary cert, add a flag to activate
rotation of previous primary cert to failover.

This functionality is part of the prober ssl cert renewal automation.
2023-04-27 14:42:11 -04:00
Lai Jiang 2021247ab4 Update README on how to manually push schema (#2009) 2023-04-26 16:32:15 -04:00
Lai Jiang 4fc7038690 Make a few minor changes to make the linter happy (#2010)
<!-- Reviewable:start -->
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/2010)
<!-- Reviewable:end -->
2023-04-26 15:49:32 -04:00
Weimin Yu 9272e7fd14 Add a test of failover certificate (#2008)
Verifies that client can log in with correct failover certificate.
2023-04-26 15:47:47 -04:00
sarahcaseybot e1afe00758 Require token transition schedules for default tokens (#2005) 2023-04-21 17:38:10 -04:00
sarahcaseybot 203c20c040 Use a TLD's configured TTLs if they are present (#1992)
* Use tld's configured TTLs if they are present

* Change to optional

* Use optionals better
2023-04-21 13:47:10 -04:00
Lai Jiang bd0cea0d87 Remove AppEngineServiceUtils (#2003)
The only method that is called from this class is setNumInstances. However we
don't current use `nomulus set_num_instances` anywhere. If we need to change
the number of instances, it is either done by updating appengine-web.xml, which
is deployed by Spinnaker, or doing it manually as a break-glass fix via gcloud
or on Pantheon.
2023-04-21 10:11:12 -04:00
217 changed files with 4560 additions and 6742 deletions
+1 -1
View File
@@ -103,7 +103,7 @@ explodeWar.doLast {
file("${it.explodedAppDirectory}/WEB-INF/lib/tools.jar").setWritable(true)
}
appengineDeployAll.finalizedBy ':cloudSchedulerDeployer'
appengineDeployAll.finalizedBy ':deployCloudSchedulerAndQueue'
rootProject.deploy.dependsOn appengineDeployAll
rootProject.stage.dependsOn appengineStage
tasks['war'].dependsOn ':console-webapp:buildConsoleWebappProd'
+9 -3
View File
@@ -558,17 +558,23 @@ task coreDev {
javadocDependentTasks.each { tasks.javadoc.dependsOn(it) }
// Runs the script, which deploys cloud scheduler tasks based on the config
task cloudSchedulerDeployer {
// Runs the script, which deploys cloud scheduler and tasks based on the config
task deployCloudSchedulerAndQueue {
doLast {
def env = environment
if (!prodOrSandboxEnv) {
exec {
commandLine 'go', 'run',
"${rootDir}/release/builder/cloudSchedulerDeployer.go",
"${rootDir}/release/builder/deployCloudSchedulerAndQueue.go",
"${rootDir}/core/src/main/java/google/registry/env/${env}/default/WEB-INF/cloud-scheduler-tasks.xml",
"domain-registry-${env}"
}
exec {
commandLine 'go', 'run',
"${rootDir}/release/builder/deployCloudSchedulerAndQueue.go",
"${rootDir}/core/src/main/java/google/registry/env/common/default/WEB-INF/cloud-tasks-queue.xml",
"domain-registry-${env}"
}
}
}
}
+60 -48
View File
@@ -3898,10 +3898,13 @@
"dev": true
},
"node_modules/@types/cors": {
"version": "2.8.12",
"resolved": "https://registry.npmjs.org/@types/cors/-/cors-2.8.12.tgz",
"integrity": "sha512-vt+kDhq/M2ayberEtJcIN/hxXy1Pk+59g2FV/ZQceeaTyCtCucjL2Q7FXlFjtWn4n15KCr1NE2lNNFhp0lEThw==",
"dev": true
"version": "2.8.13",
"resolved": "https://registry.npmjs.org/@types/cors/-/cors-2.8.13.tgz",
"integrity": "sha512-RG8AStHlUiV5ysZQKq97copd2UmVYw3/pRMLefISZ3S1hK104Cwm7iLQ3fTKx+lsUH2CE8FlLaYeEA2LSeqYUA==",
"dev": true,
"dependencies": {
"@types/node": "*"
}
},
"node_modules/@types/eslint": {
"version": "8.4.6",
@@ -5907,9 +5910,9 @@
}
},
"node_modules/engine.io": {
"version": "6.2.1",
"resolved": "https://registry.npmjs.org/engine.io/-/engine.io-6.2.1.tgz",
"integrity": "sha512-ECceEFcAaNRybd3lsGQKas3ZlMVjN3cyWwMP25D2i0zWfyiytVbTpRPa34qrr+FHddtpBVOmq4H/DCv1O0lZRA==",
"version": "6.4.2",
"resolved": "https://registry.npmjs.org/engine.io/-/engine.io-6.4.2.tgz",
"integrity": "sha512-FKn/3oMiJjrOEOeUub2WCox6JhxBXq/Zn3fZOMCBxKnNYtsdKjxhl7yR3fZhM9PV+rdE75SU5SYMc+2PGzo+Tg==",
"dev": true,
"dependencies": {
"@types/cookie": "^0.4.1",
@@ -5921,16 +5924,16 @@
"cors": "~2.8.5",
"debug": "~4.3.1",
"engine.io-parser": "~5.0.3",
"ws": "~8.2.3"
"ws": "~8.11.0"
},
"engines": {
"node": ">=10.0.0"
}
},
"node_modules/engine.io-parser": {
"version": "5.0.4",
"resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-5.0.4.tgz",
"integrity": "sha512-+nVFp+5z1E3HcToEnO7ZIj3g+3k9389DvWtvJZz0T6/eOCPIyyxehFcedoYrZQrp0LgQbD9pPXhpMBKMd5QURg==",
"version": "5.0.6",
"resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-5.0.6.tgz",
"integrity": "sha512-tjuoZDMAdEhVnSFleYPCtdL2GXwVTGtNjoeJd9IhIG3C1xs9uwxqRNEu5WpnDZCaozwVlK/nuQhpodhXSIMaxw==",
"dev": true,
"engines": {
"node": ">=10.0.0"
@@ -10880,27 +10883,30 @@
}
},
"node_modules/socket.io": {
"version": "4.5.2",
"resolved": "https://registry.npmjs.org/socket.io/-/socket.io-4.5.2.tgz",
"integrity": "sha512-6fCnk4ARMPZN448+SQcnn1u8OHUC72puJcNtSgg2xS34Cu7br1gQ09YKkO1PFfDn/wyUE9ZgMAwosJed003+NQ==",
"version": "4.6.1",
"resolved": "https://registry.npmjs.org/socket.io/-/socket.io-4.6.1.tgz",
"integrity": "sha512-KMcaAi4l/8+xEjkRICl6ak8ySoxsYG+gG6/XfRCPJPQ/haCRIJBTL4wIl8YCsmtaBovcAXGLOShyVWQ/FG8GZA==",
"dev": true,
"dependencies": {
"accepts": "~1.3.4",
"base64id": "~2.0.0",
"debug": "~4.3.2",
"engine.io": "~6.2.0",
"socket.io-adapter": "~2.4.0",
"socket.io-parser": "~4.2.0"
"engine.io": "~6.4.1",
"socket.io-adapter": "~2.5.2",
"socket.io-parser": "~4.2.1"
},
"engines": {
"node": ">=10.0.0"
}
},
"node_modules/socket.io-adapter": {
"version": "2.4.0",
"resolved": "https://registry.npmjs.org/socket.io-adapter/-/socket.io-adapter-2.4.0.tgz",
"integrity": "sha512-W4N+o69rkMEGVuk2D/cvca3uYsvGlMwsySWV447y99gUPghxq42BxqLNMndb+a1mm/5/7NeXVQS7RLa2XyXvYg==",
"dev": true
"version": "2.5.2",
"resolved": "https://registry.npmjs.org/socket.io-adapter/-/socket.io-adapter-2.5.2.tgz",
"integrity": "sha512-87C3LO/NOMc+eMcpcxUBebGjkpMDkNBS9tf7KJqcDsmL936EChtVva71Dw2q4tQcuVC+hAUy4an2NO/sYXmwRA==",
"dev": true,
"dependencies": {
"ws": "~8.11.0"
}
},
"node_modules/socket.io-parser": {
"version": "4.2.1",
@@ -12219,9 +12225,9 @@
"dev": true
},
"node_modules/ws": {
"version": "8.2.3",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.2.3.tgz",
"integrity": "sha512-wBuoj1BDpC6ZQ1B7DWQBYVLphPWkm8i9Y0/3YdHjHKHiohOJ1ws+3OccDWtH+PoC9DZD5WOTrJvNbWvjS6JWaA==",
"version": "8.11.0",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.11.0.tgz",
"integrity": "sha512-HPG3wQd9sNQoT9xHyNCXoDUa+Xw/VevmY9FoHyQ+g+rrMn4j6FB4np7Z0OhdTgjx6MgQLK7jwSy1YecU1+4Asg==",
"dev": true,
"engines": {
"node": ">=10.0.0"
@@ -15175,10 +15181,13 @@
"dev": true
},
"@types/cors": {
"version": "2.8.12",
"resolved": "https://registry.npmjs.org/@types/cors/-/cors-2.8.12.tgz",
"integrity": "sha512-vt+kDhq/M2ayberEtJcIN/hxXy1Pk+59g2FV/ZQceeaTyCtCucjL2Q7FXlFjtWn4n15KCr1NE2lNNFhp0lEThw==",
"dev": true
"version": "2.8.13",
"resolved": "https://registry.npmjs.org/@types/cors/-/cors-2.8.13.tgz",
"integrity": "sha512-RG8AStHlUiV5ysZQKq97copd2UmVYw3/pRMLefISZ3S1hK104Cwm7iLQ3fTKx+lsUH2CE8FlLaYeEA2LSeqYUA==",
"dev": true,
"requires": {
"@types/node": "*"
}
},
"@types/eslint": {
"version": "8.4.6",
@@ -16746,9 +16755,9 @@
}
},
"engine.io": {
"version": "6.2.1",
"resolved": "https://registry.npmjs.org/engine.io/-/engine.io-6.2.1.tgz",
"integrity": "sha512-ECceEFcAaNRybd3lsGQKas3ZlMVjN3cyWwMP25D2i0zWfyiytVbTpRPa34qrr+FHddtpBVOmq4H/DCv1O0lZRA==",
"version": "6.4.2",
"resolved": "https://registry.npmjs.org/engine.io/-/engine.io-6.4.2.tgz",
"integrity": "sha512-FKn/3oMiJjrOEOeUub2WCox6JhxBXq/Zn3fZOMCBxKnNYtsdKjxhl7yR3fZhM9PV+rdE75SU5SYMc+2PGzo+Tg==",
"dev": true,
"requires": {
"@types/cookie": "^0.4.1",
@@ -16760,13 +16769,13 @@
"cors": "~2.8.5",
"debug": "~4.3.1",
"engine.io-parser": "~5.0.3",
"ws": "~8.2.3"
"ws": "~8.11.0"
}
},
"engine.io-parser": {
"version": "5.0.4",
"resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-5.0.4.tgz",
"integrity": "sha512-+nVFp+5z1E3HcToEnO7ZIj3g+3k9389DvWtvJZz0T6/eOCPIyyxehFcedoYrZQrp0LgQbD9pPXhpMBKMd5QURg==",
"version": "5.0.6",
"resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-5.0.6.tgz",
"integrity": "sha512-tjuoZDMAdEhVnSFleYPCtdL2GXwVTGtNjoeJd9IhIG3C1xs9uwxqRNEu5WpnDZCaozwVlK/nuQhpodhXSIMaxw==",
"dev": true
},
"enhanced-resolve": {
@@ -20509,24 +20518,27 @@
"dev": true
},
"socket.io": {
"version": "4.5.2",
"resolved": "https://registry.npmjs.org/socket.io/-/socket.io-4.5.2.tgz",
"integrity": "sha512-6fCnk4ARMPZN448+SQcnn1u8OHUC72puJcNtSgg2xS34Cu7br1gQ09YKkO1PFfDn/wyUE9ZgMAwosJed003+NQ==",
"version": "4.6.1",
"resolved": "https://registry.npmjs.org/socket.io/-/socket.io-4.6.1.tgz",
"integrity": "sha512-KMcaAi4l/8+xEjkRICl6ak8ySoxsYG+gG6/XfRCPJPQ/haCRIJBTL4wIl8YCsmtaBovcAXGLOShyVWQ/FG8GZA==",
"dev": true,
"requires": {
"accepts": "~1.3.4",
"base64id": "~2.0.0",
"debug": "~4.3.2",
"engine.io": "~6.2.0",
"socket.io-adapter": "~2.4.0",
"socket.io-parser": "~4.2.0"
"engine.io": "~6.4.1",
"socket.io-adapter": "~2.5.2",
"socket.io-parser": "~4.2.1"
}
},
"socket.io-adapter": {
"version": "2.4.0",
"resolved": "https://registry.npmjs.org/socket.io-adapter/-/socket.io-adapter-2.4.0.tgz",
"integrity": "sha512-W4N+o69rkMEGVuk2D/cvca3uYsvGlMwsySWV447y99gUPghxq42BxqLNMndb+a1mm/5/7NeXVQS7RLa2XyXvYg==",
"dev": true
"version": "2.5.2",
"resolved": "https://registry.npmjs.org/socket.io-adapter/-/socket.io-adapter-2.5.2.tgz",
"integrity": "sha512-87C3LO/NOMc+eMcpcxUBebGjkpMDkNBS9tf7KJqcDsmL936EChtVva71Dw2q4tQcuVC+hAUy4an2NO/sYXmwRA==",
"dev": true,
"requires": {
"ws": "~8.11.0"
}
},
"socket.io-parser": {
"version": "4.2.1",
@@ -21486,9 +21498,9 @@
"dev": true
},
"ws": {
"version": "8.2.3",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.2.3.tgz",
"integrity": "sha512-wBuoj1BDpC6ZQ1B7DWQBYVLphPWkm8i9Y0/3YdHjHKHiohOJ1ws+3OccDWtH+PoC9DZD5WOTrJvNbWvjS6JWaA==",
"version": "8.11.0",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.11.0.tgz",
"integrity": "sha512-HPG3wQd9sNQoT9xHyNCXoDUa+Xw/VevmY9FoHyQ+g+rrMn4j6FB4np7Z0OhdTgjx6MgQLK7jwSy1YecU1+4Asg==",
"dev": true,
"requires": {}
},
+2 -2
View File
@@ -749,8 +749,8 @@ if (environment == 'alpha') {
],
expandBilling :
[
mainClass: 'google.registry.beam.billing.ExpandRecurringBillingEventsPipeline',
metaData : 'google/registry/beam/expand_recurring_billing_events_pipeline_metadata.json'
mainClass: 'google.registry.beam.billing.ExpandBillingRecurrencesPipeline',
metaData : 'google/registry/beam/expand_billing_recurrences_pipeline_metadata.json'
],
rde :
[
@@ -17,7 +17,6 @@ package google.registry.batch;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_REQUESTED_TIME;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_RESAVE_TIMES;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_RESOURCE_KEY;
import static google.registry.batch.CannedScriptExecutionAction.SCRIPT_PARAM;
import static google.registry.request.RequestParameters.extractBooleanParameter;
import static google.registry.request.RequestParameters.extractIntParameter;
import static google.registry.request.RequestParameters.extractLongParameter;
@@ -105,16 +104,15 @@ public class BatchModule {
}
@Provides
@Parameter(ExpandRecurringBillingEventsAction.PARAM_START_TIME)
@Parameter(ExpandBillingRecurrencesAction.PARAM_START_TIME)
static Optional<DateTime> provideStartTime(HttpServletRequest req) {
return extractOptionalDatetimeParameter(
req, ExpandRecurringBillingEventsAction.PARAM_START_TIME);
return extractOptionalDatetimeParameter(req, ExpandBillingRecurrencesAction.PARAM_START_TIME);
}
@Provides
@Parameter(ExpandRecurringBillingEventsAction.PARAM_END_TIME)
@Parameter(ExpandBillingRecurrencesAction.PARAM_END_TIME)
static Optional<DateTime> provideEndTime(HttpServletRequest req) {
return extractOptionalDatetimeParameter(req, ExpandRecurringBillingEventsAction.PARAM_END_TIME);
return extractOptionalDatetimeParameter(req, ExpandBillingRecurrencesAction.PARAM_END_TIME);
}
@Provides
@@ -124,9 +122,9 @@ public class BatchModule {
}
@Provides
@Parameter(ExpandRecurringBillingEventsAction.PARAM_ADVANCE_CURSOR)
@Parameter(ExpandBillingRecurrencesAction.PARAM_ADVANCE_CURSOR)
static boolean provideAdvanceCursor(HttpServletRequest req) {
return extractBooleanParameter(req, ExpandRecurringBillingEventsAction.PARAM_ADVANCE_CURSOR);
return extractBooleanParameter(req, ExpandBillingRecurrencesAction.PARAM_ADVANCE_CURSOR);
}
@Provides
@@ -140,11 +138,4 @@ public class BatchModule {
static boolean provideIsDryRun(HttpServletRequest req) {
return extractBooleanParameter(req, PARAM_DRY_RUN);
}
// TODO(b/234424397): remove method after credential changes are rolled out.
@Provides
@Parameter(SCRIPT_PARAM)
static String provideScriptName(HttpServletRequest req) {
return extractRequiredParameter(req, SCRIPT_PARAM);
}
}
@@ -16,26 +16,23 @@ package google.registry.batch;
import static google.registry.request.Action.Method.POST;
import com.google.common.collect.ImmutableMap;
import com.google.common.flogger.FluentLogger;
import google.registry.batch.cannedscript.GroupsApiChecker;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
import javax.inject.Inject;
/**
* Action that executes a canned script specified by the caller.
*
* <p>This class is introduced to help the safe rollout of credential changes. The delegated
* credentials in particular, benefit from this: they require manual configuration of the peer
* system in each environment, and may wait hours or even days after deployment until triggered by
* user activities.
* <p>This class provides a hook for invoking hard-coded methods. The main use case is to verify in
* Sandbox and Production environments new features that depend on environment-specific
* configurations. For example, the {@code DelegatedCredential}, which requires correct GWorkspace
* configuration, has been tested this way. Since it is a hassle to add or remove endpoints, we keep
* this class all the time.
*
* <p>This action can be invoked using the Nomulus CLI command: {@code nomulus -e ${env} curl
* --service BACKEND -X POST -u '/_dr/task/executeCannedScript?script=${script_name}'}
* --service BACKEND -X POST -u '/_dr/task/executeCannedScript}'}
*/
// TODO(b/234424397): remove class after credential changes are rolled out.
@Action(
service = Action.Service.BACKEND,
path = "/_dr/task/executeCannedScript",
@@ -45,29 +42,18 @@ import javax.inject.Inject;
public class CannedScriptExecutionAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
static final String SCRIPT_PARAM = "script";
static final ImmutableMap<String, Runnable> SCRIPTS =
ImmutableMap.of("runGroupsApiChecks", GroupsApiChecker::runGroupsApiChecks);
private final String scriptName;
@Inject
CannedScriptExecutionAction(@Parameter(SCRIPT_PARAM) String scriptName) {
logger.atInfo().log("Received request to run script %s", scriptName);
this.scriptName = scriptName;
CannedScriptExecutionAction() {
logger.atInfo().log("Received request to run scripts.");
}
@Override
public void run() {
if (!SCRIPTS.containsKey(scriptName)) {
throw new IllegalArgumentException("Script not found:" + scriptName);
}
try {
SCRIPTS.get(scriptName).run();
logger.atInfo().log("Finished running %s.", scriptName);
// Invoke canned scripts here.
logger.atInfo().log("Finished running scripts.");
} catch (Throwable t) {
logger.atWarning().withCause(t).log("Error executing %s", scriptName);
logger.atWarning().withCause(t).log("Error executing scripts.");
throw new RuntimeException("Execution failed.");
}
}
@@ -16,6 +16,7 @@ package google.registry.batch;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.tools.ServiceConnection.getServer;
import static java.util.concurrent.TimeUnit.SECONDS;
import com.google.api.gax.rpc.ApiException;
@@ -23,6 +24,8 @@ import com.google.cloud.tasks.v2.AppEngineHttpRequest;
import com.google.cloud.tasks.v2.AppEngineRouting;
import com.google.cloud.tasks.v2.CloudTasksClient;
import com.google.cloud.tasks.v2.HttpMethod;
import com.google.cloud.tasks.v2.HttpRequest;
import com.google.cloud.tasks.v2.OidcToken;
import com.google.cloud.tasks.v2.QueueName;
import com.google.cloud.tasks.v2.Task;
import com.google.common.base.Joiner;
@@ -46,7 +49,10 @@ import java.nio.charset.StandardCharsets;
import java.util.Arrays;
import java.util.Optional;
import java.util.Random;
import java.util.function.BiConsumer;
import java.util.function.Consumer;
import java.util.function.Supplier;
import javax.annotation.Nullable;
import javax.inject.Inject;
import org.joda.time.Duration;
@@ -61,6 +67,9 @@ public class CloudTasksUtils implements Serializable {
private final Clock clock;
private final String projectId;
private final String locationId;
// defaultServiceAccount and iapClientId are nullable because Optional isn't serializable
@Nullable private final String defaultServiceAccount;
@Nullable private final String iapClientId;
private final SerializableCloudTasksClient client;
@Inject
@@ -69,11 +78,15 @@ public class CloudTasksUtils implements Serializable {
Clock clock,
@Config("projectId") String projectId,
@Config("locationId") String locationId,
@Config("defaultServiceAccount") Optional<String> defaultServiceAccount,
@Config("iapClientId") Optional<String> iapClientId,
SerializableCloudTasksClient client) {
this.retrier = retrier;
this.clock = clock;
this.projectId = projectId;
this.locationId = locationId;
this.defaultServiceAccount = defaultServiceAccount.orElse(null);
this.iapClientId = iapClientId.orElse(null);
this.client = client;
}
@@ -98,6 +111,74 @@ public class CloudTasksUtils implements Serializable {
return enqueue(queue, Arrays.asList(tasks));
}
/**
* Converts a (possible) set of params into an HTTP request via the appropriate method.
*
* <p>For GET requests we add them on to the URL, and for POST requests we add them in the body of
* the request.
*
* <p>The parameters {@code putHeadersFunction} and {@code setBodyFunction} are used so that this
* method can be called with either an AppEngine HTTP request or a standard non-AppEngine HTTP
* request. The two objects do not have the same methods, but both have ways of setting headers /
* body.
*
* @return the resulting path (unchanged for POST requests, with params added for GET requests)
*/
private String processRequestParameters(
String path,
HttpMethod method,
Multimap<String, String> params,
BiConsumer<String, String> putHeadersFunction,
Consumer<ByteString> setBodyFunction) {
if (CollectionUtils.isNullOrEmpty(params)) {
return path;
}
Escaper escaper = UrlEscapers.urlPathSegmentEscaper();
String encodedParams =
Joiner.on("&")
.join(
params.entries().stream()
.map(
entry ->
String.format(
"%s=%s",
escaper.escape(entry.getKey()), escaper.escape(entry.getValue())))
.collect(toImmutableList()));
if (method.equals(HttpMethod.GET)) {
return String.format("%s?%s", path, encodedParams);
}
putHeadersFunction.accept(HttpHeaders.CONTENT_TYPE, MediaType.FORM_DATA.toString());
setBodyFunction.accept(ByteString.copyFrom(encodedParams, StandardCharsets.UTF_8));
return path;
}
/**
* Creates a {@link Task} that does not use AppEngine for submission.
*
* <p>This uses the standard Cloud Tasks auth format to create and send an OIDC ID token set to
* the default service account. That account must have permission to submit tasks to Cloud Tasks.
*/
private Task createNonAppEngineTask(
String path, HttpMethod method, Service service, Multimap<String, String> params) {
HttpRequest.Builder requestBuilder = HttpRequest.newBuilder().setHttpMethod(method);
path =
processRequestParameters(
path, method, params, requestBuilder::putHeaders, requestBuilder::setBody);
OidcToken.Builder oidcTokenBuilder =
OidcToken.newBuilder().setServiceAccountEmail(defaultServiceAccount);
// If the service is using IAP, add that as the audience for the token so the request can be
// appropriately authed. Otherwise, use the project name.
if (iapClientId != null) {
oidcTokenBuilder.setAudience(iapClientId);
} else {
oidcTokenBuilder.setAudience(projectId);
}
requestBuilder.setOidcToken(oidcTokenBuilder.build());
String totalPath = String.format("%s%s", getServer(service), path);
requestBuilder.setUrl(totalPath);
return Task.newBuilder().setHttpRequest(requestBuilder.build()).build();
}
/**
* Create a {@link Task} to be enqueued.
*
@@ -123,34 +204,21 @@ public class CloudTasksUtils implements Serializable {
method.equals(HttpMethod.GET) || method.equals(HttpMethod.POST),
"HTTP method %s is used. Only GET and POST are allowed.",
method);
AppEngineHttpRequest.Builder requestBuilder =
AppEngineHttpRequest.newBuilder()
.setHttpMethod(method)
.setAppEngineRouting(
AppEngineRouting.newBuilder().setService(service.toString()).build());
if (!CollectionUtils.isNullOrEmpty(params)) {
Escaper escaper = UrlEscapers.urlPathSegmentEscaper();
String encodedParams =
Joiner.on("&")
.join(
params.entries().stream()
.map(
entry ->
String.format(
"%s=%s",
escaper.escape(entry.getKey()), escaper.escape(entry.getValue())))
.collect(toImmutableList()));
if (method == HttpMethod.GET) {
path = String.format("%s?%s", path, encodedParams);
} else {
requestBuilder
.putHeaders(HttpHeaders.CONTENT_TYPE, MediaType.FORM_DATA.toString())
.setBody(ByteString.copyFrom(encodedParams, StandardCharsets.UTF_8));
}
// If the default service account is configured, send a standard non-AppEngine HTTP request
if (defaultServiceAccount != null) {
return createNonAppEngineTask(path, method, service, params);
} else {
AppEngineHttpRequest.Builder requestBuilder =
AppEngineHttpRequest.newBuilder()
.setHttpMethod(method)
.setAppEngineRouting(
AppEngineRouting.newBuilder().setService(service.toString()).build());
path =
processRequestParameters(
path, method, params, requestBuilder::putHeaders, requestBuilder::setBody);
requestBuilder.setRelativeUri(path);
return Task.newBuilder().setAppEngineHttpRequest(requestBuilder.build()).build();
}
requestBuilder.setRelativeUri(path);
return Task.newBuilder().setAppEngineHttpRequest(requestBuilder.build()).build();
}
/**
@@ -19,6 +19,7 @@ import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.batch.BatchModule.PARAM_DRY_RUN;
import static google.registry.config.RegistryEnvironment.PRODUCTION;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_DELETE;
import static google.registry.model.tld.Registries.getTldsOfType;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
@@ -32,7 +33,6 @@ import com.google.common.collect.Sets;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryEnvironment;
import google.registry.dns.DnsUtils;
import google.registry.model.CreateAutoTimestamp;
import google.registry.model.EppResourceUtils;
import google.registry.model.domain.Domain;
@@ -98,8 +98,6 @@ public class DeleteProberDataAction implements Runnable {
/** Number of domains to retrieve and delete per SQL transaction. */
private static final int BATCH_SIZE = 1000;
@Inject DnsUtils dnsUtils;
@Inject
@Parameter(PARAM_DRY_RUN)
boolean isDryRun;
@@ -222,7 +220,7 @@ public class DeleteProberDataAction implements Runnable {
}
}
private void hardDeleteDomainsAndHosts(
private static void hardDeleteDomainsAndHosts(
ImmutableList<String> domainRepoIds, ImmutableList<String> hostNames) {
tm().query("DELETE FROM Host WHERE hostName IN :hostNames")
.setParameter("hostNames", hostNames)
@@ -264,6 +262,6 @@ public class DeleteProberDataAction implements Runnable {
// messages, or auto-renews because those will all be hard-deleted the next time the job runs
// anyway.
tm().putAll(ImmutableList.of(deletedDomain, historyEntry));
dnsUtils.requestDomainDnsRefresh(deletedDomain.getDomainName());
requestDomainDnsRefresh(deletedDomain.getDomainName());
}
}
@@ -29,11 +29,11 @@ import com.google.api.services.dataflow.model.LaunchFlexTemplateRequest;
import com.google.api.services.dataflow.model.LaunchFlexTemplateResponse;
import com.google.common.collect.ImmutableMap;
import com.google.common.flogger.FluentLogger;
import google.registry.beam.billing.ExpandRecurringBillingEventsPipeline;
import google.registry.beam.billing.ExpandBillingRecurrencesPipeline;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryEnvironment;
import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.common.Cursor;
import google.registry.request.Action;
import google.registry.request.Parameter;
@@ -46,20 +46,20 @@ import javax.inject.Inject;
import org.joda.time.DateTime;
/**
* An action that kicks off a {@link ExpandRecurringBillingEventsPipeline} dataflow job to expand
* {@link Recurring} billing events into synthetic {@link OneTime} events.
* An action that kicks off a {@link ExpandBillingRecurrencesPipeline} dataflow job to expand {@link
* BillingRecurrence} billing events into synthetic {@link BillingEvent} events.
*/
@Action(
service = Action.Service.BACKEND,
path = "/_dr/task/expandRecurringBillingEvents",
path = "/_dr/task/expandBillingRecurrences",
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
public class ExpandRecurringBillingEventsAction implements Runnable {
public class ExpandBillingRecurrencesAction implements Runnable {
public static final String PARAM_START_TIME = "startTime";
public static final String PARAM_END_TIME = "endTime";
public static final String PARAM_ADVANCE_CURSOR = "advanceCursor";
private static final String PIPELINE_NAME = "expand_recurring_billing_events_pipeline";
private static final String PIPELINE_NAME = "expand_billing_recurrences_pipeline";
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
@Inject Clock clock;
@@ -97,7 +97,7 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
@Inject Response response;
@Inject
ExpandRecurringBillingEventsAction() {}
ExpandBillingRecurrencesAction() {}
@Override
public void run() {
@@ -133,7 +133,7 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
.put("advanceCursor", Boolean.toString(advanceCursor))
.build());
logger.atInfo().log(
"Launching recurring billing event expansion pipeline for event time range [%s, %s)%s.",
"Launching billing recurrence expansion pipeline for event time range [%s, %s)%s.",
startTime,
endTime,
isDryRun ? " in dry run mode" : advanceCursor ? "" : " without advancing the cursor");
@@ -152,7 +152,7 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
response.setStatus(SC_OK);
response.setPayload(
String.format(
"Launched recurring billing event expansion pipeline: %s",
"Launched billing recurrence expansion pipeline: %s",
launchResponse.getJob().getId()));
} catch (IOException e) {
logger.atWarning().withCause(e).log("Pipeline Launch failed");
@@ -1,122 +0,0 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch.cannedscript;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.util.RegistrarUtils.normalizeRegistrarId;
import com.google.api.services.admin.directory.Directory;
import com.google.api.services.groupssettings.Groupssettings;
import com.google.common.base.Supplier;
import com.google.common.base.Suppliers;
import com.google.common.base.Throwables;
import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import dagger.Component;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule;
import google.registry.config.CredentialModule.AdcDelegatedCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.groups.DirectoryGroupsConnection;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.util.GoogleCredentialsBundle;
import google.registry.util.UtilsModule;
import java.util.List;
import java.util.Set;
import javax.inject.Singleton;
/**
* Verifies that the credential with the {@link AdcDelegatedCredential} annotation can be used to
* access the Google Workspace Groups API.
*/
// TODO(b/234424397): remove class after credential changes are rolled out.
public class GroupsApiChecker {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final Supplier<GroupsConnectionComponent> COMPONENT_SUPPLIER =
Suppliers.memoize(DaggerGroupsApiChecker_GroupsConnectionComponent::create);
public static void runGroupsApiChecks() {
GroupsConnectionComponent component = COMPONENT_SUPPLIER.get();
DirectoryGroupsConnection groupsConnection = component.groupsConnection();
List<Registrar> registrars =
Streams.stream(Registrar.loadAllCached())
.filter(registrar -> registrar.isLive() && registrar.getType() == Registrar.Type.REAL)
.collect(toImmutableList());
for (Registrar registrar : registrars) {
for (final RegistrarPoc.Type type : RegistrarPoc.Type.values()) {
String groupKey =
String.format(
"%s-%s-contacts@%s",
normalizeRegistrarId(registrar.getRegistrarId()),
type.getDisplayName(),
component.gSuiteDomainName());
try {
Set<String> currentMembers = groupsConnection.getMembersOfGroup(groupKey);
logger.atInfo().log("Found %s members for %s.", currentMembers.size(), groupKey);
} catch (Exception e) {
Throwables.throwIfUnchecked(e);
throw new RuntimeException(e);
}
}
}
}
@Singleton
@Component(
modules = {
ConfigModule.class,
CredentialModule.class,
GroupsApiModule.class,
UtilsModule.class
})
interface GroupsConnectionComponent {
DirectoryGroupsConnection groupsConnection();
@Config("gSuiteDomainName")
String gSuiteDomainName();
}
@Module
static class GroupsApiModule {
@Provides
static Directory provideDirectory(
@AdcDelegatedCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Directory.Builder(
credentialsBundle.getHttpTransport(),
credentialsBundle.getJsonFactory(),
credentialsBundle.getHttpRequestInitializer())
.setApplicationName(projectId)
.build();
}
@Provides
static Groupssettings provideGroupsSettings(
@AdcDelegatedCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Groupssettings.Builder(
credentialsBundle.getHttpTransport(),
credentialsBundle.getJsonFactory(),
credentialsBundle.getHttpRequestInitializer())
.setApplicationName(projectId)
.build();
}
}
}
@@ -64,7 +64,7 @@ public abstract class BillingEvent implements Serializable {
"amount",
"flags");
/** Returns the unique ID for the {@code OneTime} associated with this event. */
/** Returns the unique ID for the {@code BillingEvent} associated with this event. */
abstract long id();
/** Returns the UTC DateTime this event becomes billable. */
@@ -189,7 +189,7 @@ public abstract class BillingEvent implements Serializable {
.minusDays(1)
.toString(),
billingId(),
String.format("%s - %s", registrarId(), tld()),
registrarId(),
String.format("%s | TLD: %s | TERM: %d-year", action(), tld(), years()),
amount(),
currency(),
@@ -233,7 +233,7 @@ public abstract class BillingEvent implements Serializable {
/** Returns the billing account id, which is the {@code BillingEvent.billingId}. */
abstract String productAccountKey();
/** Returns the invoice grouping key, which is in the format "registrarId - tld". */
/** Returns the invoice grouping key, which is the registrar ID. */
abstract String usageGroupingKey();
/** Returns a description of the item, formatted as "action | TLD: tld | TERM: n-year." */
@@ -38,10 +38,10 @@ import google.registry.flows.custom.CustomLogicModule;
import google.registry.flows.domain.DomainPricingLogic;
import google.registry.flows.domain.DomainPricingLogic.AllocationTokenInvalidForPremiumNameException;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingEvent.Cancellation;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingBase.Flag;
import google.registry.model.billing.BillingCancellation;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.common.Cursor;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
@@ -77,48 +77,49 @@ import org.apache.beam.sdk.values.PDone;
import org.joda.time.DateTime;
/**
* Definition of a Dataflow Flex pipeline template, which expands {@link Recurring} to {@link
* OneTime} when an autorenew occurs within the given time frame.
* Definition of a Dataflow Flex pipeline template, which expands {@link BillingRecurrence} to
* {@link BillingEvent} when an autorenew occurs within the given time frame.
*
* <p>This pipeline works in three stages:
*
* <ul>
* <li>Gather the {@link Recurring}s that are in scope for expansion. The exact condition of
* {@link Recurring}s to include can be found in {@link #getRecurringsInScope(Pipeline)}.
* <li>Expand the {@link Recurring}s to {@link OneTime} (and corresponding {@link DomainHistory})
* that fall within the [{@link #startTime}, {@link #endTime}) window, excluding those that
* are already present (to make this pipeline idempotent when running with the same parameters
* multiple times, either in parallel or in sequence). The {@link Recurring} is also updated
* with the information on when it was last expanded, so it would not be in scope for
* expansion until at least a year later.
* <li>Gather the {@link BillingRecurrence}s that are in scope for expansion. The exact condition
* of {@link BillingRecurrence}s to include can be found in {@link
* #getRecurrencesInScope(Pipeline)}.
* <li>Expand the {@link BillingRecurrence}s to {@link BillingEvent} (and corresponding {@link
* DomainHistory}) that fall within the [{@link #startTime}, {@link #endTime}) window,
* excluding those that are already present (to make this pipeline idempotent when running
* with the same parameters multiple times, either in parallel or in sequence). The {@link
* BillingRecurrence} is also updated with the information on when it was last expanded, so it
* would not be in scope for expansion until at least a year later.
* <li>If the cursor for billing events should be advanced, advance it to {@link #endTime} after
* all of the expansions in the previous step is done, only when it is currently at {@link
* #startTime}.
* </ul>
*
* <p>Note that the creation of new {@link OneTime} and {@link DomainHistory} is done speculatively
* as soon as its event time is in scope for expansion (i.e. within the window of operation). If a
* domain is subsequently cancelled during the autorenew grace period, a {@link Cancellation} would
* have been created to cancel the {@link OneTime} out. Similarly, a {@link DomainHistory} for the
* delete will be created which negates the effect of the speculatively created {@link
* DomainHistory}, specifically for the transaction records. Both the {@link OneTime} and {@link
* DomainHistory} will only be used (and cancelled out) when the billing time becomes effective,
* which is after the grace period, when the cancellations would have been written, if need be. This
* is no different from what we do with manual renewals or normal creates, where entities are always
* created for the action regardless of whether their effects will be negated later due to
* subsequent actions within respective grace periods.
* <p>Note that the creation of new {@link BillingEvent} and {@link DomainHistory} is done
* speculatively as soon as its event time is in scope for expansion (i.e. within the window of
* operation). If a domain is subsequently cancelled during the autorenew grace period, a {@link
* BillingCancellation} would have been created to cancel the {@link BillingEvent} out. Similarly, a
* {@link DomainHistory} for the delete will be created which negates the effect of the
* speculatively created {@link DomainHistory}, specifically for the transaction records. Both the
* {@link BillingEvent} and {@link DomainHistory} will only be used (and cancelled out) when the
* billing time becomes effective, which is after the grace period, when the cancellations would
* have been written, if need be. This is no different from what we do with manual renewals or
* normal creates, where entities are always created for the action regardless of whether their
* effects will be negated later due to subsequent actions within respective grace periods.
*
* <p>To stage this template locally, run {@code ./nom_build :core:sBP --environment=alpha \
* --pipeline=expandBilling}.
*
* <p>Then, you can run the staged template via the API client library, gCloud or a raw REST call.
*
* @see Cancellation#forGracePeriod
* @see BillingCancellation#forGracePeriod
* @see google.registry.flows.domain.DomainFlowUtils#createCancelingRecords
* @see <a href="https://cloud.google.com/dataflow/docs/guides/templates/using-flex-templates">Using
* Flex Templates</a>
*/
public class ExpandRecurringBillingEventsPipeline implements Serializable {
public class ExpandBillingRecurrencesPipeline implements Serializable {
private static final long serialVersionUID = -5827984301386630194L;
@@ -128,7 +129,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
static {
PipelineComponent pipelineComponent =
DaggerExpandRecurringBillingEventsPipeline_PipelineComponent.create();
DaggerExpandBillingRecurrencesPipeline_PipelineComponent.create();
domainPricingLogic = pipelineComponent.domainPricingLogic();
batchSize = pipelineComponent.batchSize();
}
@@ -139,8 +140,8 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
private final DateTime endTime;
private final boolean isDryRun;
private final boolean advanceCursor;
private final Counter recurringsInScopeCounter =
Metrics.counter("ExpandBilling", "Recurrings in scope for expansion");
private final Counter recurrencesInScopeCounter =
Metrics.counter("ExpandBilling", "Recurrences in scope for expansion");
// Note that this counter is only accurate when running in dry run mode. Because SQL persistence
// is a side effect and not idempotent, a transaction to save OneTimes could be successful but the
// transform that contains it could be still be retried, rolling back the counter increment. The
@@ -150,8 +151,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
private final Counter oneTimesToExpandCounter =
Metrics.counter("ExpandBilling", "OneTimes that would be expanded");
ExpandRecurringBillingEventsPipeline(
ExpandRecurringBillingEventsPipelineOptions options, Clock clock) {
ExpandBillingRecurrencesPipeline(ExpandBillingRecurrencesPipelineOptions options, Clock clock) {
startTime = DateTime.parse(options.getStartTime());
endTime = DateTime.parse(options.getEndTime());
checkArgument(
@@ -170,16 +170,16 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
}
void setupPipeline(Pipeline pipeline) {
PCollection<KV<Integer, Long>> recurringIds = getRecurringsInScope(pipeline);
PCollection<Void> expanded = expandRecurrings(recurringIds);
PCollection<KV<Integer, Long>> recurrenceIds = getRecurrencesInScope(pipeline);
PCollection<Void> expanded = expandRecurrences(recurrenceIds);
if (!isDryRun && advanceCursor) {
advanceCursor(expanded);
}
}
PCollection<KV<Integer, Long>> getRecurringsInScope(Pipeline pipeline) {
PCollection<KV<Integer, Long>> getRecurrencesInScope(Pipeline pipeline) {
return pipeline.apply(
"Read all Recurrings in scope",
"Read all Recurrences in scope",
// Use native query because JPQL does not support timestamp arithmetics.
RegistryJpaIO.read(
"SELECT billing_recurrence_id "
@@ -203,7 +203,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
endTime.minusYears(1)),
true,
(BigInteger id) -> {
recurringsInScopeCounter.inc();
recurrencesInScopeCounter.inc();
// Note that because all elements are mapped to the same dummy key, the next
// batching transform will effectively be serial. This however does not matter for
// our use case because the elements were obtained from a SQL read query, which
@@ -222,13 +222,13 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
.withCoder(KvCoder.of(VarIntCoder.of(), VarLongCoder.of())));
}
private PCollection<Void> expandRecurrings(PCollection<KV<Integer, Long>> recurringIds) {
return recurringIds
private PCollection<Void> expandRecurrences(PCollection<KV<Integer, Long>> recurrenceIds) {
return recurrenceIds
.apply(
"Group into batches",
GroupIntoBatches.<Integer, Long>ofSize(batchSize).withShardedKey())
.apply(
"Expand and save Recurrings into OneTimes and corresponding DomainHistories",
"Expand and save Recurrences into OneTimes and corresponding DomainHistories",
MapElements.into(voids())
.via(
element -> {
@@ -237,7 +237,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
() -> {
ImmutableSet.Builder<ImmutableObject> results =
new ImmutableSet.Builder<>();
ids.forEach(id -> expandOneRecurring(id, results));
ids.forEach(id -> expandOneRecurrence(id, results));
if (!isDryRun) {
tm().putAll(results.build());
}
@@ -246,14 +246,16 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
}));
}
private void expandOneRecurring(Long recurringId, ImmutableSet.Builder<ImmutableObject> results) {
Recurring recurring = tm().loadByKey(Recurring.createVKey(recurringId));
private void expandOneRecurrence(
Long recurrenceId, ImmutableSet.Builder<ImmutableObject> results) {
BillingRecurrence billingRecurrence =
tm().loadByKey(BillingRecurrence.createVKey(recurrenceId));
// Determine the complete set of EventTimes this recurring event should expand to within
// Determine the complete set of EventTimes this recurrence event should expand to within
// [max(recurrenceLastExpansion + 1 yr, startTime), min(recurrenceEndTime, endTime)).
//
// This range should always be legal for recurrings that are returned from the query. However,
// it is possible that the recurring has changed between when the read transformation occurred
// This range should always be legal for recurrences that are returned from the query. However,
// it is possible that the recurrence has changed between when the read transformation occurred
// and now. This could be caused by some out-of-process mutations (such as a domain deletion
// closing out a previously open-ended recurrence), or more subtly, Beam could execute the same
// work multiple times due to transient communication issues between workers and the scheduler.
@@ -264,23 +266,25 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
// to work on the same batch. The second worker would see a new recurrence_last_expansion that
// causes the range to be illegal.
//
// The best way to handle any unexpected behavior is to simply drop the recurring from
// The best way to handle any unexpected behavior is to simply drop the recurrence from
// expansion, if its new state still calls for an expansion, it would be picked up the next time
// the pipeline runs.
ImmutableSet<DateTime> eventTimes;
try {
eventTimes =
ImmutableSet.copyOf(
recurring
billingRecurrence
.getRecurrenceTimeOfYear()
.getInstancesInRange(
Range.closedOpen(
latestOf(recurring.getRecurrenceLastExpansion().plusYears(1), startTime),
earliestOf(recurring.getRecurrenceEndTime(), endTime))));
latestOf(
billingRecurrence.getRecurrenceLastExpansion().plusYears(1),
startTime),
earliestOf(billingRecurrence.getRecurrenceEndTime(), endTime))));
} catch (IllegalArgumentException e) {
return;
}
Domain domain = tm().loadByKey(Domain.createVKey(recurring.getDomainRepoId()));
Domain domain = tm().loadByKey(Domain.createVKey(billingRecurrence.getDomainRepoId()));
Tld tld = Tld.get(domain.getTld());
// Find the times for which the OneTime billing event are already created, making this expansion
@@ -292,7 +296,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
"SELECT eventTime FROM BillingEvent WHERE cancellationMatchingBillingEvent ="
+ " :key",
DateTime.class)
.setParameter("key", recurring.createVKey())
.setParameter("key", billingRecurrence.createVKey())
.getResultList());
Set<DateTime> eventTimesToExpand = difference(eventTimes, existingEventTimes);
@@ -301,7 +305,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
return;
}
DateTime recurrenceLastExpansionTime = recurring.getRecurrenceLastExpansion();
DateTime recurrenceLastExpansionTime = billingRecurrence.getRecurrenceLastExpansion();
// Create new OneTime and DomainHistory for EventTimes that needs to be expanded.
for (DateTime eventTime : eventTimesToExpand) {
@@ -333,7 +337,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
DomainHistory historyEntry =
new DomainHistory.Builder()
.setBySuperuser(false)
.setRegistrarId(recurring.getRegistrarId())
.setRegistrarId(billingRecurrence.getRegistrarId())
.setModificationTime(tm().getTransactionTime())
.setDomain(domain)
.setPeriod(Period.create(1, YEARS))
@@ -385,35 +389,43 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
// It is OK to always create a OneTime, even though the domain might be deleted or transferred
// later during autorenew grace period, as a cancellation will always be written out in those
// instances.
OneTime oneTime = null;
BillingEvent billingEvent = null;
try {
oneTime =
new OneTime.Builder()
billingEvent =
new BillingEvent.Builder()
.setBillingTime(billingTime)
.setRegistrarId(recurring.getRegistrarId())
.setRegistrarId(billingRecurrence.getRegistrarId())
// Determine the cost for a one-year renewal.
.setCost(
domainPricingLogic
.getRenewPrice(
tld, recurring.getTargetId(), eventTime, 1, recurring, Optional.empty())
tld,
billingRecurrence.getTargetId(),
eventTime,
1,
billingRecurrence,
Optional.empty())
.getRenewCost())
.setEventTime(eventTime)
.setFlags(union(recurring.getFlags(), Flag.SYNTHETIC))
.setFlags(union(billingRecurrence.getFlags(), Flag.SYNTHETIC))
.setDomainHistory(historyEntry)
.setPeriodYears(1)
.setReason(recurring.getReason())
.setReason(billingRecurrence.getReason())
.setSyntheticCreationTime(endTime)
.setCancellationMatchingBillingEvent(recurring)
.setTargetId(recurring.getTargetId())
.setCancellationMatchingBillingEvent(billingRecurrence)
.setTargetId(billingRecurrence.getTargetId())
.build();
} catch (AllocationTokenInvalidForPremiumNameException e) {
// This should not be reached since we are not using an allocation token
return;
}
results.add(oneTime);
results.add(billingEvent);
}
results.add(
recurring.asBuilder().setRecurrenceLastExpansion(recurrenceLastExpansionTime).build());
billingRecurrence
.asBuilder()
.setRecurrenceLastExpansion(recurrenceLastExpansionTime)
.build());
}
private PDone advanceCursor(PCollection<Void> persisted) {
@@ -456,11 +468,11 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
}
public static void main(String[] args) {
PipelineOptionsFactory.register(ExpandRecurringBillingEventsPipelineOptions.class);
ExpandRecurringBillingEventsPipelineOptions options =
PipelineOptionsFactory.register(ExpandBillingRecurrencesPipelineOptions.class);
ExpandBillingRecurrencesPipelineOptions options =
PipelineOptionsFactory.fromArgs(args)
.withValidation()
.as(ExpandRecurringBillingEventsPipelineOptions.class);
.as(ExpandBillingRecurrencesPipelineOptions.class);
// Hardcode the transaction level to be at serializable we do not want concurrent runs of the
// pipeline for the same window to create duplicate OneTimes. This ensures that the set of
// existing OneTimes do not change by the time new OneTimes are inserted within a transaction.
@@ -468,7 +480,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
// Per PostgreSQL, serializable isolation level does not introduce any blocking beyond that
// present in repeatable read other than some overhead related to monitoring possible
// serializable anomalies. Therefore, in most cases, since each worker of the same job works on
// a different set of recurrings, it is not possible for their execution order to affect
// a different set of recurrences, it is not possible for their execution order to affect
// serialization outcome, and the performance penalty should be minimum when using serializable
// compared to using repeatable read.
//
@@ -480,7 +492,7 @@ public class ExpandRecurringBillingEventsPipeline implements Serializable {
// See: https://www.postgresql.org/docs/current/transaction-iso.html
options.setIsolationOverride(TransactionIsolationLevel.TRANSACTION_SERIALIZABLE);
Pipeline pipeline = Pipeline.create(options);
new ExpandRecurringBillingEventsPipeline(options, new SystemClock()).run(pipeline);
new ExpandBillingRecurrencesPipeline(options, new SystemClock()).run(pipeline);
}
@Singleton
@@ -18,7 +18,7 @@ import google.registry.beam.common.RegistryPipelineOptions;
import org.apache.beam.sdk.options.Default;
import org.apache.beam.sdk.options.Description;
public interface ExpandRecurringBillingEventsPipelineOptions extends RegistryPipelineOptions {
public interface ExpandBillingRecurrencesPipelineOptions extends RegistryPipelineOptions {
@Description(
"The inclusive lower bound of on the range of event times that will be expanded, in ISO 8601"
+ " format")
@@ -22,8 +22,8 @@ import google.registry.beam.billing.BillingEvent.InvoiceGroupingKey;
import google.registry.beam.billing.BillingEvent.InvoiceGroupingKey.InvoiceGroupingKeyCoder;
import google.registry.beam.common.RegistryJpaIO;
import google.registry.beam.common.RegistryJpaIO.Read;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingBase.Flag;
import google.registry.model.billing.BillingEvent;
import google.registry.model.registrar.Registrar;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.reporting.billing.BillingModule;
@@ -86,29 +86,30 @@ public class InvoicingPipeline implements Serializable {
void setupPipeline(Pipeline pipeline) {
options.setIsolationOverride(TransactionIsolationLevel.TRANSACTION_READ_COMMITTED);
PCollection<BillingEvent> billingEvents = readFromCloudSql(options, pipeline);
PCollection<google.registry.beam.billing.BillingEvent> billingEvents =
readFromCloudSql(options, pipeline);
saveInvoiceCsv(billingEvents, options);
saveDetailedCsv(billingEvents, options);
}
static PCollection<BillingEvent> readFromCloudSql(
static PCollection<google.registry.beam.billing.BillingEvent> readFromCloudSql(
InvoicingPipelineOptions options, Pipeline pipeline) {
Read<Object[], BillingEvent> read =
RegistryJpaIO.<Object[], BillingEvent>read(
Read<Object[], google.registry.beam.billing.BillingEvent> read =
RegistryJpaIO.<Object[], google.registry.beam.billing.BillingEvent>read(
makeCloudSqlQuery(options.getYearMonth()), false, row -> parseRow(row).orElse(null))
.withCoder(SerializableCoder.of(BillingEvent.class));
.withCoder(SerializableCoder.of(google.registry.beam.billing.BillingEvent.class));
PCollection<BillingEvent> billingEventsWithNulls =
PCollection<google.registry.beam.billing.BillingEvent> billingEventsWithNulls =
pipeline.apply("Read BillingEvents from Cloud SQL", read);
// Remove null billing events
return billingEventsWithNulls.apply(Filter.by(Objects::nonNull));
}
private static Optional<BillingEvent> parseRow(Object[] row) {
OneTime oneTime = (OneTime) row[0];
private static Optional<google.registry.beam.billing.BillingEvent> parseRow(Object[] row) {
BillingEvent billingEvent = (BillingEvent) row[0];
Registrar registrar = (Registrar) row[1];
CurrencyUnit currency = oneTime.getCost().getCurrencyUnit();
CurrencyUnit currency = billingEvent.getCost().getCurrencyUnit();
if (!registrar.getBillingAccountMap().containsKey(currency)) {
logger.atSevere().log(
"Registrar %s does not have a product account key for the currency unit: %s",
@@ -117,37 +118,40 @@ public class InvoicingPipeline implements Serializable {
}
return Optional.of(
BillingEvent.create(
oneTime.getId(),
oneTime.getBillingTime(),
oneTime.getEventTime(),
google.registry.beam.billing.BillingEvent.create(
billingEvent.getId(),
billingEvent.getBillingTime(),
billingEvent.getEventTime(),
registrar.getRegistrarId(),
registrar.getBillingAccountMap().get(currency),
registrar.getPoNumber().orElse(""),
DomainNameUtils.getTldFromDomainName(oneTime.getTargetId()),
oneTime.getReason().toString(),
oneTime.getTargetId(),
oneTime.getDomainRepoId(),
Optional.ofNullable(oneTime.getPeriodYears()).orElse(0),
oneTime.getCost().getCurrencyUnit().toString(),
oneTime.getCost().getAmount().doubleValue(),
DomainNameUtils.getTldFromDomainName(billingEvent.getTargetId()),
billingEvent.getReason().toString(),
billingEvent.getTargetId(),
billingEvent.getDomainRepoId(),
Optional.ofNullable(billingEvent.getPeriodYears()).orElse(0),
billingEvent.getCost().getCurrencyUnit().toString(),
billingEvent.getCost().getAmount().doubleValue(),
String.join(
" ", oneTime.getFlags().stream().map(Flag::toString).collect(toImmutableSet()))));
" ",
billingEvent.getFlags().stream().map(Flag::toString).collect(toImmutableSet()))));
}
/** Transform that converts a {@code BillingEvent} into an invoice CSV row. */
private static class GenerateInvoiceRows
extends PTransform<PCollection<BillingEvent>, PCollection<String>> {
extends PTransform<
PCollection<google.registry.beam.billing.BillingEvent>, PCollection<String>> {
private static final long serialVersionUID = -8090619008258393728L;
@Override
public PCollection<String> expand(PCollection<BillingEvent> input) {
public PCollection<String> expand(
PCollection<google.registry.beam.billing.BillingEvent> input) {
return input
.apply(
"Map to invoicing key",
MapElements.into(TypeDescriptor.of(InvoiceGroupingKey.class))
.via(BillingEvent::getInvoiceGroupingKey))
.via(google.registry.beam.billing.BillingEvent::getInvoiceGroupingKey))
.apply(
"Filter out free events", Filter.by((InvoiceGroupingKey key) -> key.unitPrice() != 0))
.setCoder(new InvoiceGroupingKeyCoder())
@@ -161,7 +165,8 @@ public class InvoicingPipeline implements Serializable {
/** Saves the billing events to a single overall invoice CSV file. */
static void saveInvoiceCsv(
PCollection<BillingEvent> billingEvents, InvoicingPipelineOptions options) {
PCollection<google.registry.beam.billing.BillingEvent> billingEvents,
InvoicingPipelineOptions options) {
billingEvents
.apply("Generate overall invoice rows", new GenerateInvoiceRows())
.apply(
@@ -182,16 +187,17 @@ public class InvoicingPipeline implements Serializable {
/** Saves the billing events to detailed report CSV files keyed by registrar-tld pairs. */
static void saveDetailedCsv(
PCollection<BillingEvent> billingEvents, InvoicingPipelineOptions options) {
PCollection<google.registry.beam.billing.BillingEvent> billingEvents,
InvoicingPipelineOptions options) {
String yearMonth = options.getYearMonth();
billingEvents.apply(
"Write detailed report for each registrar-tld pair",
FileIO.<String, BillingEvent>writeDynamic()
FileIO.<String, google.registry.beam.billing.BillingEvent>writeDynamic()
.to(
String.format(
"%s/%s/%s",
options.getBillingBucketUrl(), BillingModule.INVOICES_DIRECTORY, yearMonth))
.by(BillingEvent::getDetailedReportGroupingKey)
.by(google.registry.beam.billing.BillingEvent::getDetailedReportGroupingKey)
.withNumShards(1)
.withDestinationCoder(StringUtf8Coder.of())
.withNaming(
@@ -200,8 +206,8 @@ public class InvoicingPipeline implements Serializable {
String.format(
"%s_%s_%s.csv", BillingModule.DETAIL_REPORT_PREFIX, yearMonth, key))
.via(
Contextful.fn(BillingEvent::toCsv),
TextIO.sink().withHeader(BillingEvent.getHeader())));
Contextful.fn(google.registry.beam.billing.BillingEvent::toCsv),
TextIO.sink().withHeader(google.registry.beam.billing.BillingEvent.getHeader())));
}
/** Create the Cloud SQL query for a given yearMonth at runtime. */
@@ -20,7 +20,7 @@ import com.google.common.collect.ImmutableList;
import dagger.Module;
import dagger.Provides;
import dagger.multibindings.Multibinds;
import google.registry.config.CredentialModule.DefaultCredential;
import google.registry.config.CredentialModule.ApplicationDefaultCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.util.GoogleCredentialsBundle;
import java.util.Map;
@@ -34,7 +34,7 @@ public abstract class BigqueryModule {
@Provides
static Bigquery provideBigquery(
@DefaultCredential GoogleCredentialsBundle credentialsBundle,
@ApplicationDefaultCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Bigquery.Builder(
credentialsBundle.getHttpTransport(),
@@ -22,7 +22,7 @@ import dagger.Provides;
import google.registry.batch.CloudTasksUtils;
import google.registry.batch.CloudTasksUtils.GcpCloudTasksClient;
import google.registry.batch.CloudTasksUtils.SerializableCloudTasksClient;
import google.registry.config.CredentialModule.DefaultCredential;
import google.registry.config.CredentialModule.ApplicationDefaultCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.util.GoogleCredentialsBundle;
import java.io.IOException;
@@ -41,7 +41,7 @@ public abstract class CloudTasksUtilsModule {
// Provides a supplier instead of using a Dagger @Provider because the latter is not serializable.
@Provides
public static Supplier<CloudTasksClient> provideCloudTasksClientSupplier(
@DefaultCredential GoogleCredentialsBundle credentials) {
@ApplicationDefaultCredential GoogleCredentialsBundle credentials) {
return (Supplier<CloudTasksClient> & Serializable)
() -> {
CloudTasksClient client;
@@ -66,38 +66,6 @@ public abstract class CredentialModule {
return GoogleCredentialsBundle.create(credential);
}
/**
* Provides the default {@link GoogleCredentialsBundle} from the Google Cloud runtime.
*
* <p>The credential returned depends on the runtime environment:
*
* <ul>
* <li>On AppEngine, returns the service account credential for
* PROJECT_ID@appspot.gserviceaccount.com
* <li>On Compute Engine, returns the service account credential for
* PROJECT_NUMBER-compute@developer.gserviceaccount.com
* <li>On end user host, this returns the credential downloaded by gcloud. Please refer to <a
* href="https://cloud.google.com/sdk/gcloud/reference/auth/application-default/login">Cloud
* SDK documentation</a> for details.
* </ul>
*/
@DefaultCredential
@Provides
@Singleton
public static GoogleCredentialsBundle provideDefaultCredential(
@Config("defaultCredentialOauthScopes") ImmutableList<String> requiredScopes) {
GoogleCredentials credential;
try {
credential = GoogleCredentials.getApplicationDefault();
} catch (IOException e) {
throw new RuntimeException(e);
}
if (credential.createScopedRequired()) {
credential = credential.createScoped(requiredScopes);
}
return GoogleCredentialsBundle.create(credential);
}
/**
* Provides a {@link GoogleCredentialsBundle} for accessing Google Workspace APIs, such as Drive
* and Sheets.
@@ -162,13 +130,6 @@ public abstract class CredentialModule {
@Retention(RetentionPolicy.RUNTIME)
public @interface ApplicationDefaultCredential {}
/** Dagger qualifier for the Application Default Credential. */
@Qualifier
@Documented
@Retention(RetentionPolicy.RUNTIME)
@Deprecated // Switching to @ApplicationDefaultCredential
public @interface DefaultCredential {}
/** Dagger qualifier for the credential for Google Workspace APIs. */
@Qualifier
@Documented
@@ -108,12 +108,6 @@ public final class RegistryConfig {
return config.gcpProject.projectId;
}
@Provides
@Config("serviceAccountEmails")
public static ImmutableList<String> provideServiceAccountEmails(RegistryConfigSettings config) {
return ImmutableList.copyOf(config.gcpProject.serviceAccountEmails);
}
@Provides
@Config("projectIdNumber")
public static long provideProjectIdNumber(RegistryConfigSettings config) {
@@ -126,6 +120,18 @@ public final class RegistryConfig {
return config.gcpProject.locationId;
}
@Provides
@Config("serviceAccountEmails")
public static ImmutableList<String> provideServiceAccountEmails(RegistryConfigSettings config) {
return ImmutableList.copyOf(config.gcpProject.serviceAccountEmails);
}
@Provides
@Config("defaultServiceAccount")
public static Optional<String> provideDefaultServiceAccount(RegistryConfigSettings config) {
return Optional.ofNullable(config.gcpProject.defaultServiceAccount);
}
/**
* The filename of the logo to be displayed in the header of the registrar console.
*
@@ -55,6 +55,7 @@ public class RegistryConfigSettings {
public String toolsServiceUrl;
public String pubapiServiceUrl;
public List<String> serviceAccountEmails;
public String defaultServiceAccount;
}
/** Configuration options for OAuth settings for authenticating users. */
@@ -27,6 +27,9 @@ gcpProject:
serviceAccountEmails:
- default-service-account-email@email.com
- cloud-scheduler-email@email.com
# The default service account with which the service is running. For example,
# on GAE this would be {project-id}@appspot.gserviceaccount.com
defaultServiceAccount: null
gSuite:
# Publicly accessible domain name of the running G Suite instance.
@@ -140,13 +140,25 @@ public final class TldFanoutAction implements Runnable {
for (String tld : tlds) {
Task task = createTask(tld, flowThruParams);
Task createdTask = cloudTasksUtils.enqueue(queue, task);
outputPayload.append(
String.format(
"- Task: '%s', tld: '%s', endpoint: '%s'\n",
createdTask.getName(), tld, createdTask.getAppEngineHttpRequest().getRelativeUri()));
logger.atInfo().log(
"Task: '%s', tld: '%s', endpoint: '%s'.",
createdTask.getName(), tld, createdTask.getAppEngineHttpRequest().getRelativeUri());
if (createdTask.hasAppEngineHttpRequest()) {
outputPayload.append(
String.format(
"- Task: '%s', tld: '%s', endpoint: '%s'\n",
createdTask.getName(),
tld,
createdTask.getAppEngineHttpRequest().getRelativeUri()));
logger.atInfo().log(
"Task: '%s', tld: '%s', endpoint: '%s'.",
createdTask.getName(), tld, createdTask.getAppEngineHttpRequest().getRelativeUri());
} else {
outputPayload.append(
String.format(
"- Task: '%s', tld: '%s', endpoint: '%s'\n",
createdTask.getName(), tld, createdTask.getHttpRequest().getUrl()));
logger.atInfo().log(
"Task: '%s', tld: '%s', endpoint: '%s'.",
createdTask.getName(), tld, createdTask.getHttpRequest().getUrl());
}
}
response.setContentType(PLAIN_TEXT_UTF_8);
response.setPayload(outputPayload.toString());
@@ -1,41 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.dns;
/** Static class for DNS-related constants. */
public class DnsConstants {
private DnsConstants() {}
/** The name of the DNS pull queue. */
public static final String DNS_PULL_QUEUE_NAME = "dns-pull"; // See queue.xml.
/** The name of the DNS publish push queue. */
public static final String DNS_PUBLISH_PUSH_QUEUE_NAME = "dns-publish"; // See queue.xml.
/** The parameter to use for storing the target type ("domain" or "host" or "zone"). */
public static final String DNS_TARGET_TYPE_PARAM = "Target-Type";
/** The parameter to use for storing the target name (domain or host name) with the task. */
public static final String DNS_TARGET_NAME_PARAM = "Target-Name";
/** The parameter to use for storing the creation time with the task. */
public static final String DNS_TARGET_CREATE_TIME_PARAM = "Create-Time";
/** The possible values of the {@code DNS_TARGET_TYPE_PARAM} parameter. */
public enum TargetType {
DOMAIN,
HOST
}
}
@@ -14,8 +14,6 @@
package google.registry.dns;
import static google.registry.dns.DnsConstants.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.DnsConstants.DNS_PULL_QUEUE_NAME;
import static google.registry.dns.RefreshDnsOnHostRenameAction.PARAM_HOST_KEY;
import static google.registry.request.RequestParameters.extractEnumParameter;
import static google.registry.request.RequestParameters.extractIntParameter;
@@ -24,20 +22,17 @@ import static google.registry.request.RequestParameters.extractOptionalParameter
import static google.registry.request.RequestParameters.extractRequiredParameter;
import static google.registry.request.RequestParameters.extractSetOfParameters;
import com.google.appengine.api.taskqueue.Queue;
import com.google.appengine.api.taskqueue.QueueFactory;
import com.google.common.hash.HashFunction;
import com.google.common.hash.Hashing;
import dagger.Binds;
import dagger.Module;
import dagger.Provides;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.dns.DnsUtils.TargetType;
import google.registry.dns.writer.DnsWriterZone;
import google.registry.request.Parameter;
import google.registry.request.RequestParameters;
import java.util.Optional;
import java.util.Set;
import javax.inject.Named;
import javax.servlet.http.HttpServletRequest;
import org.joda.time.DateTime;
@@ -71,18 +66,6 @@ public abstract class DnsModule {
return Hashing.murmur3_32_fixed();
}
@Provides
@Named(DNS_PULL_QUEUE_NAME)
static Queue provideDnsPullQueue() {
return QueueFactory.getQueue(DNS_PULL_QUEUE_NAME);
}
@Provides
@Named(DNS_PUBLISH_PUSH_QUEUE_NAME)
static Queue provideDnsUpdatePushQueue() {
return QueueFactory.getQueue(DNS_PUBLISH_PUSH_QUEUE_NAME);
}
@Provides
@Parameter(PARAM_PUBLISH_TASK_ENQUEUED)
static DateTime provideCreateTime(HttpServletRequest req) {
@@ -1,170 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.dns;
import static com.google.appengine.api.taskqueue.QueueFactory.getQueue;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.dns.DnsConstants.DNS_PULL_QUEUE_NAME;
import static google.registry.dns.DnsConstants.DNS_TARGET_CREATE_TIME_PARAM;
import static google.registry.dns.DnsConstants.DNS_TARGET_NAME_PARAM;
import static google.registry.dns.DnsConstants.DNS_TARGET_TYPE_PARAM;
import static google.registry.model.tld.Registries.assertTldExists;
import static google.registry.request.RequestParameters.PARAM_TLD;
import static google.registry.util.DomainNameUtils.getTldFromDomainName;
import static java.util.concurrent.TimeUnit.MILLISECONDS;
import com.google.appengine.api.taskqueue.Queue;
import com.google.appengine.api.taskqueue.QueueConstants;
import com.google.appengine.api.taskqueue.TaskHandle;
import com.google.appengine.api.taskqueue.TaskOptions;
import com.google.appengine.api.taskqueue.TaskOptions.Method;
import com.google.appengine.api.taskqueue.TransientFailureException;
import com.google.apphosting.api.DeadlineExceededException;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableList;
import com.google.common.flogger.FluentLogger;
import com.google.common.net.InternetDomainName;
import com.google.common.util.concurrent.RateLimiter;
import google.registry.config.RegistryConfig;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.model.tld.Registries;
import google.registry.util.Clock;
import google.registry.util.NonFinalForTesting;
import java.util.List;
import java.util.Optional;
import java.util.logging.Level;
import javax.inject.Inject;
import javax.inject.Named;
import org.joda.time.Duration;
/**
* Methods for manipulating the queue used for DNS write tasks.
*
* <p>This includes a {@link RateLimiter} to limit the {@link Queue#leaseTasks} call rate to 9 QPS,
* to stay under the 10 QPS limit for this function.
*
* <p>Note that overlapping calls to {@link ReadDnsQueueAction} (the only place where {@link
* DnsQueue#leaseTasks} is used) will have different rate limiters, so they could exceed the allowed
* rate. This should be rare though - because {@link DnsQueue#leaseTasks} is only used in {@link
* ReadDnsQueueAction}, which is run as a cron job with running time shorter than the cron repeat
* time - meaning there should never be two instances running at once.
*
* @see RegistryConfig.ConfigModule#provideReadDnsRefreshRequestsRuntime()
*/
public class DnsQueue {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final Queue queue;
final Clock clock;
// Queue.leaseTasks is limited to 10 requests per second as per
// https://cloud.google.com/appengine/docs/standard/java/javadoc/com/google/appengine/api/taskqueue/Queue.html
// "If you generate more than 10 LeaseTasks requests per second, only the first 10 requests will
// return results. The others will return no results."
private static final RateLimiter rateLimiter = RateLimiter.create(9);
@Inject
DnsQueue(@Named(DNS_PULL_QUEUE_NAME) Queue queue, Clock clock) {
this.queue = queue;
this.clock = clock;
}
Clock getClock() {
return clock;
}
@VisibleForTesting
public static DnsQueue createForTesting(Clock clock) {
return new DnsQueue(getQueue(DNS_PULL_QUEUE_NAME), clock);
}
@NonFinalForTesting
@VisibleForTesting
long leaseTasksBatchSize = QueueConstants.maxLeaseCount();
/** Enqueues the given task type with the given target name to the DNS queue. */
private TaskHandle addToQueue(
TargetType targetType, String targetName, String tld, Duration countdown) {
logger.atInfo().log(
"Adding task type=%s, target=%s, tld=%s to pull queue %s (%d tasks currently on queue).",
targetType, targetName, tld, DNS_PULL_QUEUE_NAME, queue.fetchStatistics().getNumTasks());
return queue.add(
TaskOptions.Builder.withDefaults()
.method(Method.PULL)
.countdownMillis(countdown.getMillis())
.param(DNS_TARGET_TYPE_PARAM, targetType.toString())
.param(DNS_TARGET_NAME_PARAM, targetName)
.param(DNS_TARGET_CREATE_TIME_PARAM, clock.nowUtc().toString())
.param(PARAM_TLD, tld));
}
/** Adds a task to the queue to refresh the DNS information for the specified subordinate host. */
TaskHandle addHostRefreshTask(String hostName) {
Optional<InternetDomainName> tld = Registries.findTldForName(InternetDomainName.from(hostName));
checkArgument(
tld.isPresent(), String.format("%s is not a subordinate host to a known tld", hostName));
return addToQueue(TargetType.HOST, hostName, tld.get().toString(), Duration.ZERO);
}
/** Enqueues a task to refresh DNS for the specified domain now. */
TaskHandle addDomainRefreshTask(String domainName) {
return addDomainRefreshTask(domainName, Duration.ZERO);
}
/** Enqueues a task to refresh DNS for the specified domain at some point in the future. */
TaskHandle addDomainRefreshTask(String domainName, Duration countdown) {
return addToQueue(
TargetType.DOMAIN,
domainName,
assertTldExists(getTldFromDomainName(domainName)),
countdown);
}
/**
* Returns the maximum number of tasks that can be leased with {@link #leaseTasks}.
*
* <p>If this many tasks are returned, then there might be more tasks still waiting in the queue.
*
* <p>If less than this number of tasks are returned, then there are no more items in the queue.
*/
public long getLeaseTasksBatchSize() {
return leaseTasksBatchSize;
}
/** Returns handles for a batch of tasks, leased for the specified duration. */
public List<TaskHandle> leaseTasks(Duration leaseDuration) {
try {
rateLimiter.acquire();
int numTasks = queue.fetchStatistics().getNumTasks();
logger.at((numTasks >= leaseTasksBatchSize) ? Level.WARNING : Level.INFO).log(
"There are %d tasks in the DNS queue '%s'.", numTasks, DNS_PULL_QUEUE_NAME);
return queue.leaseTasks(leaseDuration.getMillis(), MILLISECONDS, leaseTasksBatchSize);
} catch (TransientFailureException | DeadlineExceededException e) {
logger.atSevere().withCause(e).log("Failed leasing tasks too fast.");
return ImmutableList.of();
}
}
/** Delete a list of tasks, removing them from the queue permanently. */
public void deleteTasks(List<TaskHandle> tasks) {
try {
queue.deleteTask(tasks);
} catch (TransientFailureException | DeadlineExceededException e) {
logger.atSevere().withCause(e).log("Failed deleting tasks too fast.");
}
}
}
@@ -19,55 +19,42 @@ import static google.registry.persistence.transaction.TransactionManagerFactory.
import com.google.common.collect.ImmutableList;
import com.google.common.net.InternetDomainName;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.model.common.DatabaseMigrationStateSchedule;
import google.registry.model.common.DatabaseMigrationStateSchedule.MigrationState;
import google.registry.model.common.DnsRefreshRequest;
import google.registry.model.tld.Registries;
import google.registry.model.tld.Tld;
import java.util.Collection;
import javax.inject.Inject;
import java.util.Optional;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/** Utility class to handle DNS refresh requests. */
// TODO: Make this a static util function once we are done with the DNS pull queue migration.
public class DnsUtils {
public final class DnsUtils {
private final DnsQueue dnsQueue;
/** The name of the DNS publish push queue. */
public static final String DNS_PUBLISH_PUSH_QUEUE_NAME = "dns-publish"; // See queue.xml.
@Inject
DnsUtils(DnsQueue dnsQueue) {
this.dnsQueue = dnsQueue;
}
private DnsUtils() {}
private void requestDnsRefresh(String name, TargetType type, Duration delay) {
private static void requestDnsRefresh(String name, TargetType type, Duration delay) {
// Throws an IllegalArgumentException if the name is not under a managed TLD -- we only update
// DNS for names that are under our management.
String tld = Registries.findTldForNameOrThrow(InternetDomainName.from(name)).toString();
if (usePullQueue()) {
if (TargetType.HOST.equals(type)) {
dnsQueue.addHostRefreshTask(name);
} else {
dnsQueue.addDomainRefreshTask(name, delay);
}
} else {
tm().transact(
() ->
tm().insert(
new DnsRefreshRequest(
type, name, tld, tm().getTransactionTime().plus(delay))));
}
tm().transact(
() ->
tm().insert(
new DnsRefreshRequest(
type, name, tld, tm().getTransactionTime().plus(delay))));
}
public void requestDomainDnsRefresh(String domainName, Duration delay) {
public static void requestDomainDnsRefresh(String domainName, Duration delay) {
requestDnsRefresh(domainName, TargetType.DOMAIN, delay);
}
public void requestDomainDnsRefresh(String domainName) {
public static void requestDomainDnsRefresh(String domainName) {
requestDomainDnsRefresh(domainName, Duration.ZERO);
}
public void requestHostDnsRefresh(String hostName) {
public static void requestHostDnsRefresh(String hostName) {
requestDnsRefresh(hostName, TargetType.HOST, Duration.ZERO);
}
@@ -83,7 +70,7 @@ public class DnsUtils {
* <li>The last time they were processed is before the cooldown period.
* </ul>
*/
public ImmutableList<DnsRefreshRequest> readAndUpdateRequestsWithLatestProcessTime(
public static ImmutableList<DnsRefreshRequest> readAndUpdateRequestsWithLatestProcessTime(
String tld, Duration cooldown, int batchSize) {
return tm().transact(
() -> {
@@ -103,7 +90,7 @@ public class DnsUtils {
// queued up for publishing, not when it is actually published by the DNS
// writer. This timestamp acts as a cooldown so the same request will not be
// retried too frequently. See DnsRefreshRequest.getLastProcessTime for a
// detailed explaination.
// detailed explanation.
.map(e -> e.updateProcessTime(transactionTime))
.collect(toImmutableList());
tm().updateAll(requests);
@@ -117,7 +104,7 @@ public class DnsUtils {
* <p>Note that if a request entity has already been deleted, the method still succeeds without
* error because all we care about is that it no longer exists after the method runs.
*/
public void deleteRequests(Collection<DnsRefreshRequest> requests) {
public static void deleteRequests(Collection<DnsRefreshRequest> requests) {
tm().transact(
() ->
tm().delete(
@@ -126,8 +113,21 @@ public class DnsUtils {
.collect(toImmutableList())));
}
private boolean usePullQueue() {
return !DatabaseMigrationStateSchedule.getValueAtTime(dnsQueue.getClock().nowUtc())
.equals(MigrationState.DNS_SQL);
public static long getDnsAPlusAAAATtlForHost(String host, Duration dnsDefaultATtl) {
Optional<InternetDomainName> tldName = Registries.findTldForName(InternetDomainName.from(host));
Duration dnsAPlusAaaaTtl = dnsDefaultATtl;
if (tldName.isPresent()) {
Tld tld = Tld.get(tldName.get().toString());
if (tld.getDnsAPlusAaaaTtl().isPresent()) {
dnsAPlusAaaaTtl = tld.getDnsAPlusAaaaTtl().get();
}
}
return dnsAPlusAaaaTtl.getStandardSeconds();
}
/** The possible values of the {@code DNS_TARGET_TYPE_PARAM} parameter. */
public enum TargetType {
DOMAIN,
HOST
}
}
@@ -15,7 +15,6 @@
package google.registry.dns;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.dns.DnsConstants.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.DnsModule.PARAM_DNS_WRITER;
import static google.registry.dns.DnsModule.PARAM_DOMAINS;
import static google.registry.dns.DnsModule.PARAM_HOSTS;
@@ -23,6 +22,9 @@ import static google.registry.dns.DnsModule.PARAM_LOCK_INDEX;
import static google.registry.dns.DnsModule.PARAM_NUM_PUBLISH_LOCKS;
import static google.registry.dns.DnsModule.PARAM_PUBLISH_TASK_ENQUEUED;
import static google.registry.dns.DnsModule.PARAM_REFRESH_REQUEST_TIME;
import static google.registry.dns.DnsUtils.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.dns.DnsUtils.requestHostDnsRefresh;
import static google.registry.model.EppResourceUtils.loadByForeignKey;
import static google.registry.request.Action.Method.POST;
import static google.registry.request.RequestParameters.PARAM_TLD;
@@ -85,7 +87,6 @@ public final class PublishDnsUpdatesAction implements Runnable, Callable<Void> {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final DnsUtils dnsUtils;
private final DnsWriterProxy dnsWriterProxy;
private final DnsMetrics dnsMetrics;
private final Duration timeout;
@@ -94,10 +95,10 @@ public final class PublishDnsUpdatesAction implements Runnable, Callable<Void> {
/**
* The DNS writer to use for this batch.
*
* <p>This comes from the fanout in {@link ReadDnsQueueAction} which dispatches each batch to be
* published by each DNS writer on the TLD. So this field contains the value of one of the DNS
* writers configured in {@link Tld#getDnsWriters()}, as of the time the batch was written out
* (and not necessarily currently).
* <p>This comes from the fanout in {@link ReadDnsRefreshRequestsAction} which dispatches each
* batch to be published by each DNS writer on the TLD. So this field contains the value of one of
* the DNS writers configured in {@link Tld#getDnsWriters()}, as of the time the batch was written
* out (and not necessarily currently).
*/
private final String dnsWriter;
@@ -139,7 +140,6 @@ public final class PublishDnsUpdatesAction implements Runnable, Callable<Void> {
@Config("gSuiteOutgoingEmailAddress") InternetAddress gSuiteOutgoingEmailAddress,
@Header(APP_ENGINE_RETRY_HEADER) Optional<Integer> appEngineRetryCount,
@Header(CLOUD_TASKS_RETRY_HEADER) Optional<Integer> cloudTasksRetryCount,
DnsUtils dnsUtils,
DnsWriterProxy dnsWriterProxy,
DnsMetrics dnsMetrics,
LockHandler lockHandler,
@@ -147,12 +147,11 @@ public final class PublishDnsUpdatesAction implements Runnable, Callable<Void> {
CloudTasksUtils cloudTasksUtils,
SendEmailService sendEmailService,
Response response) {
this.dnsUtils = dnsUtils;
this.dnsWriterProxy = dnsWriterProxy;
this.dnsMetrics = dnsMetrics;
this.timeout = timeout;
this.sendEmailService = sendEmailService;
this.retryCount =
retryCount =
cloudTasksRetryCount.orElse(
appEngineRetryCount.orElseThrow(
() -> new IllegalStateException("Missing a valid retry count header")));
@@ -276,7 +275,7 @@ public final class PublishDnsUpdatesAction implements Runnable, Callable<Void> {
return null;
}
private InternetAddress emailToInternetAddress(String email) {
private static InternetAddress emailToInternetAddress(String email) {
try {
return new InternetAddress(email, true);
} catch (Exception e) {
@@ -306,7 +305,7 @@ public final class PublishDnsUpdatesAction implements Runnable, Callable<Void> {
registrar.get().getContacts().stream()
.filter(c -> c.getTypes().contains(RegistrarPoc.Type.ADMIN))
.map(RegistrarPoc::getEmailAddress)
.map(this::emailToInternetAddress)
.map(PublishDnsUpdatesAction::emailToInternetAddress)
.collect(toImmutableList());
sendEmailService.sendEmail(
@@ -356,10 +355,10 @@ public final class PublishDnsUpdatesAction implements Runnable, Callable<Void> {
private void requeueBatch() {
logger.atInfo().log("Requeueing batch for retry.");
for (String domain : nullToEmpty(domains)) {
dnsUtils.requestDomainDnsRefresh(domain);
requestDomainDnsRefresh(domain);
}
for (String host : nullToEmpty(hosts)) {
dnsUtils.requestHostDnsRefresh(host);
requestHostDnsRefresh(host);
}
}
@@ -1,401 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.dns;
import static com.google.common.collect.ImmutableSetMultimap.toImmutableSetMultimap;
import static com.google.common.collect.Sets.difference;
import static google.registry.dns.DnsConstants.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.DnsConstants.DNS_TARGET_CREATE_TIME_PARAM;
import static google.registry.dns.DnsConstants.DNS_TARGET_NAME_PARAM;
import static google.registry.dns.DnsConstants.DNS_TARGET_TYPE_PARAM;
import static google.registry.dns.DnsModule.PARAM_DNS_WRITER;
import static google.registry.dns.DnsModule.PARAM_DOMAINS;
import static google.registry.dns.DnsModule.PARAM_HOSTS;
import static google.registry.dns.DnsModule.PARAM_LOCK_INDEX;
import static google.registry.dns.DnsModule.PARAM_NUM_PUBLISH_LOCKS;
import static google.registry.dns.DnsModule.PARAM_PUBLISH_TASK_ENQUEUED;
import static google.registry.dns.DnsModule.PARAM_REFRESH_REQUEST_TIME;
import static google.registry.request.RequestParameters.PARAM_TLD;
import static google.registry.util.DomainNameUtils.getSecondLevelDomain;
import static java.nio.charset.StandardCharsets.UTF_8;
import com.google.appengine.api.taskqueue.TaskHandle;
import com.google.auto.value.AutoValue;
import com.google.cloud.tasks.v2.Task;
import com.google.common.collect.ComparisonChain;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableMultimap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSetMultimap;
import com.google.common.collect.Iterables;
import com.google.common.collect.Ordering;
import com.google.common.flogger.FluentLogger;
import com.google.common.hash.HashFunction;
import com.google.common.hash.Hashing;
import google.registry.batch.CloudTasksUtils;
import google.registry.config.RegistryConfig.Config;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.model.tld.Registries;
import google.registry.model.tld.Tld;
import google.registry.request.Action;
import google.registry.request.Action.Service;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import java.io.UnsupportedEncodingException;
import java.util.Collection;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.stream.Collectors;
import javax.inject.Inject;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/**
* Action for fanning out DNS refresh tasks by TLD, using data taken from the DNS pull queue.
*
* <h3>Parameters Reference</h3>
*
* <ul>
* <li>{@code jitterSeconds} Randomly delay each task by up to this many seconds.
* </ul>
*/
@Action(
service = Action.Service.BACKEND,
path = "/_dr/cron/readDnsQueue",
automaticallyPrintOk = true,
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
public final class ReadDnsQueueAction implements Runnable {
private static final String PARAM_JITTER_SECONDS = "jitterSeconds";
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
/**
* Buffer time since the end of this action until retriable tasks are available again.
*
* <p>We read batches of tasks from the queue in a loop. Any task that will need to be retried has
* to be kept out of the queue for the duration of this action - otherwise we will lease it again
* in a subsequent loop.
*
* <p>The 'requestedMaximumDuration' value is the maximum delay between the first and last calls
* to lease tasks, hence we want the lease duration to be (slightly) longer than that.
* LEASE_PADDING is the value we add to {@link #requestedMaximumDuration} to make sure the lease
* duration is indeed longer.
*/
private static final Duration LEASE_PADDING = Duration.standardMinutes(1);
private final int tldUpdateBatchSize;
private final Duration requestedMaximumDuration;
private final Optional<Integer> jitterSeconds;
private final Clock clock;
private final DnsQueue dnsQueue;
private final HashFunction hashFunction;
private final CloudTasksUtils cloudTasksUtils;
@Inject
ReadDnsQueueAction(
@Config("dnsTldUpdateBatchSize") int tldUpdateBatchSize,
@Config("readDnsRefreshRequestsActionRuntime") Duration requestedMaximumDuration,
@Parameter(PARAM_JITTER_SECONDS) Optional<Integer> jitterSeconds,
Clock clock,
DnsQueue dnsQueue,
HashFunction hashFunction,
CloudTasksUtils cloudTasksUtils) {
this.tldUpdateBatchSize = tldUpdateBatchSize;
this.requestedMaximumDuration = requestedMaximumDuration;
this.jitterSeconds = jitterSeconds;
this.clock = clock;
this.dnsQueue = dnsQueue;
this.hashFunction = hashFunction;
this.cloudTasksUtils = cloudTasksUtils;
}
/** Container for items we pull out of the DNS pull queue and process for fanout. */
@AutoValue
abstract static class RefreshItem implements Comparable<RefreshItem> {
static RefreshItem create(TargetType type, String name, DateTime creationTime) {
return new AutoValue_ReadDnsQueueAction_RefreshItem(type, name, creationTime);
}
abstract TargetType type();
abstract String name();
abstract DateTime creationTime();
@Override
public int compareTo(RefreshItem other) {
return ComparisonChain.start()
.compare(this.type(), other.type())
.compare(this.name(), other.name())
.compare(this.creationTime(), other.creationTime())
.result();
}
}
/** Leases all tasks from the pull queue and creates per-tld update actions for them. */
@Override
public void run() {
DateTime requestedEndTime = clock.nowUtc().plus(requestedMaximumDuration);
ImmutableSet<String> tlds = Registries.getTlds();
while (requestedEndTime.isAfterNow()) {
List<TaskHandle> tasks = dnsQueue.leaseTasks(requestedMaximumDuration.plus(LEASE_PADDING));
logger.atInfo().log("Leased %d DNS update tasks.", tasks.size());
if (!tasks.isEmpty()) {
dispatchTasks(ImmutableSet.copyOf(tasks), tlds);
}
if (tasks.size() < dnsQueue.getLeaseTasksBatchSize()) {
return;
}
}
}
/** A set of tasks grouped based on the action to take on them. */
@AutoValue
abstract static class ClassifiedTasks {
/**
* List of tasks we want to keep in the queue (want to retry in the future).
*
* <p>Normally, any task we lease from the queue will be deleted - either because we are going
* to process it now (these tasks are part of refreshItemsByTld), or because these tasks are
* "corrupt" in some way (don't parse, don't have the required parameters etc.).
*
* <p>Some tasks however are valid, but can't be processed at the moment. These tasks will be
* kept in the queue for future processing.
*
* <p>This includes tasks belonging to paused TLDs (which we want to process once the TLD is
* unpaused) and tasks belonging to (currently) unknown TLDs.
*/
abstract ImmutableSet<TaskHandle> tasksToKeep();
/** The paused TLDs for which we found at least one refresh request. */
abstract ImmutableSet<String> pausedTlds();
/**
* The unknown TLDs for which we found at least one refresh request.
*
* <p>Unknown TLDs might have valid requests because the list of TLDs is heavily cached. Hence,
* when we add a new TLD - it is possible that some instances will not have that TLD in their
* list yet. We don't want to discard these tasks, so we wait a bit and retry them again.
*
* <p>This is less likely for production TLDs but is quite likely for test TLDs where we might
* create a TLD and then use it within seconds.
*/
abstract ImmutableSet<String> unknownTlds();
/**
* All the refresh items we need to actually fulfill, grouped by TLD.
*
* <p>By default, the multimap is ordered - this can be changed with the builder.
*
* <p>The items for each TLD will be grouped together, and domains and hosts will be grouped
* within a TLD.
*
* <p>The grouping and ordering of domains and hosts is not technically necessary, but a
* predictable ordering makes it possible to write detailed tests.
*/
abstract ImmutableSetMultimap<String, RefreshItem> refreshItemsByTld();
static Builder builder() {
Builder builder = new AutoValue_ReadDnsQueueAction_ClassifiedTasks.Builder();
builder
.refreshItemsByTldBuilder()
.orderKeysBy(Ordering.natural())
.orderValuesBy(Ordering.natural());
return builder;
}
@AutoValue.Builder
abstract static class Builder {
abstract ImmutableSet.Builder<TaskHandle> tasksToKeepBuilder();
abstract ImmutableSet.Builder<String> pausedTldsBuilder();
abstract ImmutableSet.Builder<String> unknownTldsBuilder();
abstract ImmutableSetMultimap.Builder<String, RefreshItem> refreshItemsByTldBuilder();
abstract ClassifiedTasks build();
}
}
/**
* Creates per-tld update actions for tasks leased from the pull queue.
*
* <p>Will return "irrelevant" tasks to the queue for future processing. "Irrelevant" tasks are
* tasks for paused TLDs or tasks for TLDs not part of {@link Registries#getTlds()}.
*/
private void dispatchTasks(ImmutableSet<TaskHandle> tasks, ImmutableSet<String> tlds) {
ClassifiedTasks classifiedTasks = classifyTasks(tasks, tlds);
if (!classifiedTasks.pausedTlds().isEmpty()) {
logger.atInfo().log(
"The dns-pull queue is paused for TLDs: %s.", classifiedTasks.pausedTlds());
}
if (!classifiedTasks.unknownTlds().isEmpty()) {
logger.atWarning().log(
"The dns-pull queue has unknown TLDs: %s.", classifiedTasks.unknownTlds());
}
bucketRefreshItems(classifiedTasks.refreshItemsByTld());
if (!classifiedTasks.tasksToKeep().isEmpty()) {
logger.atWarning().log(
"Keeping %d DNS update tasks in the queue.", classifiedTasks.tasksToKeep().size());
}
// Delete the tasks we don't want to see again from the queue.
//
// tasksToDelete includes both the tasks that we already fulfilled in this call (were part of
// refreshItemsByTld) and "corrupt" tasks we weren't able to parse correctly (this shouldn't
// happen, and we logged a "severe" error)
//
// We let the lease on the rest of the tasks (the tasks we want to keep) expire on its own. The
// reason we don't release these tasks back to the queue immediately is that we are going to
// immediately read another batch of tasks from the queue - and we don't want to get the same
// tasks again.
ImmutableSet<TaskHandle> tasksToDelete =
difference(tasks, classifiedTasks.tasksToKeep()).immutableCopy();
logger.atInfo().log("Removing %d DNS update tasks from the queue.", tasksToDelete.size());
dnsQueue.deleteTasks(tasksToDelete.asList());
logger.atInfo().log("Done processing DNS tasks.");
}
/**
* Classifies the given tasks based on what action we need to take on them.
*
* <p>Note that some tasks might appear in multiple categories (if multiple actions are to be
* taken on them) or in no category (if no action is to be taken on them)
*/
private static ClassifiedTasks classifyTasks(
ImmutableSet<TaskHandle> tasks, ImmutableSet<String> tlds) {
ClassifiedTasks.Builder classifiedTasksBuilder = ClassifiedTasks.builder();
// Read all tasks on the DNS pull queue and load them into the refresh item multimap.
for (TaskHandle task : tasks) {
try {
Map<String, String> params = ImmutableMap.copyOf(task.extractParams());
DateTime creationTime = DateTime.parse(params.get(DNS_TARGET_CREATE_TIME_PARAM));
String tld = params.get(PARAM_TLD);
if (tld == null) {
logger.atSevere().log(
"Discarding invalid DNS refresh request %s; no TLD specified.", task);
} else if (!tlds.contains(tld)) {
classifiedTasksBuilder.tasksToKeepBuilder().add(task);
classifiedTasksBuilder.unknownTldsBuilder().add(tld);
} else if (Tld.get(tld).getDnsPaused()) {
classifiedTasksBuilder.tasksToKeepBuilder().add(task);
classifiedTasksBuilder.pausedTldsBuilder().add(tld);
} else {
String typeString = params.get(DNS_TARGET_TYPE_PARAM);
String name = params.get(DNS_TARGET_NAME_PARAM);
TargetType type = TargetType.valueOf(typeString);
switch (type) {
case DOMAIN:
case HOST:
classifiedTasksBuilder
.refreshItemsByTldBuilder()
.put(tld, RefreshItem.create(type, name, creationTime));
break;
default:
logger.atSevere().log(
"Discarding DNS refresh request %s of type %s.", task, typeString);
break;
}
}
} catch (RuntimeException | UnsupportedEncodingException e) {
logger.atSevere().withCause(e).log("Discarding invalid DNS refresh request %s.", task);
}
}
return classifiedTasksBuilder.build();
}
/**
* Subdivides the tld to {@link RefreshItem} multimap into buckets by lock index, if applicable.
*
* <p>If the tld has numDnsPublishLocks <= 1, we enqueue all updates on the default lock 1 of 1.
*/
private void bucketRefreshItems(ImmutableSetMultimap<String, RefreshItem> refreshItemsByTld) {
// Loop through the multimap by TLD and generate refresh tasks for the hosts and domains for
// each configured DNS writer.
for (Map.Entry<String, Collection<RefreshItem>> tldRefreshItemsEntry
: refreshItemsByTld.asMap().entrySet()) {
String tld = tldRefreshItemsEntry.getKey();
int numPublishLocks = Tld.get(tld).getNumDnsPublishLocks();
// 1 lock or less implies no TLD-wide locks, simply enqueue everything under lock 1 of 1
if (numPublishLocks <= 1) {
enqueueUpdates(tld, 1, 1, tldRefreshItemsEntry.getValue());
} else {
tldRefreshItemsEntry.getValue().stream()
.collect(
toImmutableSetMultimap(
refreshItem -> getLockIndex(tld, numPublishLocks, refreshItem),
refreshItem -> refreshItem))
.asMap()
.forEach((key, value) -> enqueueUpdates(tld, key, numPublishLocks, value));
}
}
}
/**
* Returns the lock index for a given refreshItem.
*
* <p>We hash the second level domain for all records, to group in-bailiwick hosts (the only ones
* we refresh DNS for) with their superordinate domains. We use consistent hashing to determine
* the lock index because it gives us [0,N) bucketing properties out of the box, then add 1 to
* make indexes within [1,N].
*/
private int getLockIndex(String tld, int numPublishLocks, RefreshItem refreshItem) {
String domain = getSecondLevelDomain(refreshItem.name(), tld);
return Hashing.consistentHash(hashFunction.hashString(domain, UTF_8), numPublishLocks) + 1;
}
/**
* Creates DNS refresh tasks for all writers for the tld within a lock index and batches large
* updates into smaller chunks.
*/
private void enqueueUpdates(
String tld, int lockIndex, int numPublishLocks, Collection<RefreshItem> items) {
for (List<RefreshItem> chunk : Iterables.partition(items, tldUpdateBatchSize)) {
DateTime earliestCreateTime =
chunk.stream().map(RefreshItem::creationTime).min(Comparator.naturalOrder()).get();
for (String dnsWriter : Tld.get(tld).getDnsWriters()) {
Task task =
cloudTasksUtils.createPostTaskWithJitter(
PublishDnsUpdatesAction.PATH,
Service.BACKEND,
ImmutableMultimap.<String, String>builder()
.put(PARAM_TLD, tld)
.put(PARAM_DNS_WRITER, dnsWriter)
.put(PARAM_LOCK_INDEX, Integer.toString(lockIndex))
.put(PARAM_NUM_PUBLISH_LOCKS, Integer.toString(numPublishLocks))
.put(PARAM_PUBLISH_TASK_ENQUEUED, clock.nowUtc().toString())
.put(PARAM_REFRESH_REQUEST_TIME, earliestCreateTime.toString())
.put(
PARAM_DOMAINS,
chunk.stream()
.filter(item -> item.type() == TargetType.DOMAIN)
.map(RefreshItem::name)
.collect(Collectors.joining(",")))
.put(
PARAM_HOSTS,
chunk.stream()
.filter(item -> item.type() == TargetType.HOST)
.map(RefreshItem::name)
.collect(Collectors.joining(",")))
.build(),
jitterSeconds);
cloudTasksUtils.enqueue(DNS_PUBLISH_PUSH_QUEUE_NAME, task);
}
}
}
}
@@ -15,7 +15,6 @@
package google.registry.dns;
import static com.google.common.collect.ImmutableSetMultimap.toImmutableSetMultimap;
import static google.registry.dns.DnsConstants.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.DnsModule.PARAM_DNS_JITTER_SECONDS;
import static google.registry.dns.DnsModule.PARAM_DNS_WRITER;
import static google.registry.dns.DnsModule.PARAM_DOMAINS;
@@ -24,6 +23,9 @@ import static google.registry.dns.DnsModule.PARAM_LOCK_INDEX;
import static google.registry.dns.DnsModule.PARAM_NUM_PUBLISH_LOCKS;
import static google.registry.dns.DnsModule.PARAM_PUBLISH_TASK_ENQUEUED;
import static google.registry.dns.DnsModule.PARAM_REFRESH_REQUEST_TIME;
import static google.registry.dns.DnsUtils.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.DnsUtils.deleteRequests;
import static google.registry.dns.DnsUtils.readAndUpdateRequestsWithLatestProcessTime;
import static google.registry.request.Action.Method.POST;
import static google.registry.request.RequestParameters.PARAM_TLD;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
@@ -39,7 +41,7 @@ import com.google.common.hash.HashFunction;
import com.google.common.hash.Hashing;
import google.registry.batch.CloudTasksUtils;
import google.registry.config.RegistryConfig.Config;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.dns.DnsUtils.TargetType;
import google.registry.model.common.DnsRefreshRequest;
import google.registry.model.tld.Tld;
import google.registry.request.Action;
@@ -72,7 +74,6 @@ public final class ReadDnsRefreshRequestsAction implements Runnable {
private final Optional<Integer> jitterSeconds;
private final String tld;
private final Clock clock;
private final DnsUtils dnsUtils;
private final HashFunction hashFunction;
private final CloudTasksUtils cloudTasksUtils;
@@ -83,7 +84,6 @@ public final class ReadDnsRefreshRequestsAction implements Runnable {
@Parameter(PARAM_DNS_JITTER_SECONDS) Optional<Integer> jitterSeconds,
@Parameter(PARAM_TLD) String tld,
Clock clock,
DnsUtils dnsUtils,
HashFunction hashFunction,
CloudTasksUtils cloudTasksUtils) {
this.tldUpdateBatchSize = tldUpdateBatchSize;
@@ -91,7 +91,6 @@ public final class ReadDnsRefreshRequestsAction implements Runnable {
this.jitterSeconds = jitterSeconds;
this.tld = tld;
this.clock = clock;
this.dnsUtils = dnsUtils;
this.hashFunction = hashFunction;
this.cloudTasksUtils = cloudTasksUtils;
}
@@ -112,7 +111,7 @@ public final class ReadDnsRefreshRequestsAction implements Runnable {
int processBatchSize = tldUpdateBatchSize * Tld.get(tld).getNumDnsPublishLocks();
while (requestedEndTime.isAfter(clock.nowUtc())) {
ImmutableList<DnsRefreshRequest> requests =
dnsUtils.readAndUpdateRequestsWithLatestProcessTime(
readAndUpdateRequestsWithLatestProcessTime(
tld, requestedMaximumDuration, processBatchSize);
logger.atInfo().log("Read %d DNS update requests for TLD %s.", requests.size(), tld);
if (!requests.isEmpty()) {
@@ -138,7 +137,7 @@ public final class ReadDnsRefreshRequestsAction implements Runnable {
(lockIndex, bucketedRequests) -> {
try {
enqueueUpdates(lockIndex, numPublishLocks, bucketedRequests);
dnsUtils.deleteRequests(bucketedRequests);
deleteRequests(bucketedRequests);
logger.atInfo().log(
"Processed %d DNS update requests for TLD %s.", bucketedRequests.size(), tld);
} catch (Exception e) {
@@ -14,9 +14,11 @@
package google.registry.dns;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.dns.DnsUtils.requestHostDnsRefresh;
import static google.registry.model.EppResourceUtils.loadByForeignKey;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.dns.DnsUtils.TargetType;
import google.registry.model.EppResource;
import google.registry.model.EppResource.ForeignKeyedEppResource;
import google.registry.model.annotations.ExternalMessagingName;
@@ -39,7 +41,6 @@ import javax.inject.Inject;
public final class RefreshDnsAction implements Runnable {
private final Clock clock;
private final DnsUtils dnsUtils;
private final String domainOrHostName;
private final TargetType type;
@@ -47,12 +48,10 @@ public final class RefreshDnsAction implements Runnable {
RefreshDnsAction(
@Parameter("domainOrHostName") String domainOrHostName,
@Parameter("type") TargetType type,
Clock clock,
DnsUtils dnsUtils) {
Clock clock) {
this.domainOrHostName = domainOrHostName;
this.type = type;
this.clock = clock;
this.dnsUtils = dnsUtils;
}
@Override
@@ -63,11 +62,11 @@ public final class RefreshDnsAction implements Runnable {
switch (type) {
case DOMAIN:
loadAndVerifyExistence(Domain.class, domainOrHostName);
dnsUtils.requestDomainDnsRefresh(domainOrHostName);
requestDomainDnsRefresh(domainOrHostName);
break;
case HOST:
verifyHostIsSubordinate(loadAndVerifyExistence(Host.class, domainOrHostName));
dnsUtils.requestHostDnsRefresh(domainOrHostName);
requestHostDnsRefresh(domainOrHostName);
break;
default:
throw new BadRequestException("Unsupported type: " + type);
@@ -14,6 +14,7 @@
package google.registry.dns;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.dns.RefreshDnsOnHostRenameAction.PATH;
import static google.registry.model.EppResourceUtils.getLinkedDomainKeys;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
@@ -45,14 +46,11 @@ public class RefreshDnsOnHostRenameAction implements Runnable {
private final VKey<Host> hostKey;
private final Response response;
private final DnsUtils dnsUtils;
@Inject
RefreshDnsOnHostRenameAction(
@Parameter(PARAM_HOST_KEY) String hostKey, Response response, DnsUtils dnsUtils) {
RefreshDnsOnHostRenameAction(@Parameter(PARAM_HOST_KEY) String hostKey, Response response) {
this.hostKey = VKey.createEppVKeyFromString(hostKey);
this.response = response;
this.dnsUtils = dnsUtils;
}
@Override
@@ -76,7 +74,7 @@ public class RefreshDnsOnHostRenameAction implements Runnable {
.stream()
.map(domainKey -> tm().loadByKey(domainKey))
.filter(Domain::shouldPublishToDns)
.forEach(domain -> dnsUtils.requestDomainDnsRefresh(domain.getDomainName()));
.forEach(domain -> requestDomainDnsRefresh(domain.getDomainName()));
}
if (!hostValid) {
@@ -16,6 +16,7 @@ package google.registry.dns.writer.clouddns;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.dns.DnsUtils.getDnsAPlusAAAATtlForHost;
import static google.registry.model.EppResourceUtils.loadByForeignKey;
import static google.registry.util.DomainNameUtils.getSecondLevelDomain;
@@ -40,6 +41,7 @@ import google.registry.model.domain.Domain;
import google.registry.model.domain.secdns.DomainDsData;
import google.registry.model.host.Host;
import google.registry.model.tld.Registries;
import google.registry.model.tld.Tld;
import google.registry.util.Clock;
import google.registry.util.Concurrent;
import google.registry.util.Retrier;
@@ -131,6 +133,7 @@ public class CloudDnsWriter extends BaseDnsWriter {
return;
}
Tld tld = Tld.get(domain.get().getTld());
ImmutableSet.Builder<ResourceRecordSet> domainRecords = new ImmutableSet.Builder<>();
// Construct DS records (if any).
@@ -145,7 +148,7 @@ public class CloudDnsWriter extends BaseDnsWriter {
domainRecords.add(
new ResourceRecordSet()
.setName(absoluteDomainName)
.setTtl((int) defaultDsTtl.getStandardSeconds())
.setTtl((int) tld.getDnsDsTtl().orElse(defaultDsTtl).getStandardSeconds())
.setType("DS")
.setKind("dns#resourceRecordSet")
.setRrdatas(ImmutableList.copyOf(dsRrData)));
@@ -170,7 +173,7 @@ public class CloudDnsWriter extends BaseDnsWriter {
domainRecords.add(
new ResourceRecordSet()
.setName(absoluteDomainName)
.setTtl((int) defaultNsTtl.getStandardSeconds())
.setTtl((int) tld.getDnsNsTtl().orElse(defaultNsTtl).getStandardSeconds())
.setType("NS")
.setKind("dns#resourceRecordSet")
.setRrdatas(ImmutableList.copyOf(nsRrData)));
@@ -216,7 +219,7 @@ public class CloudDnsWriter extends BaseDnsWriter {
domainRecords.add(
new ResourceRecordSet()
.setName(absoluteHostName)
.setTtl((int) defaultATtl.getStandardSeconds())
.setTtl((int) getDnsAPlusAAAATtlForHost(hostName, defaultATtl))
.setType("A")
.setKind("dns#resourceRecordSet")
.setRrdatas(ImmutableList.copyOf(aRrData)));
@@ -226,7 +229,7 @@ public class CloudDnsWriter extends BaseDnsWriter {
domainRecords.add(
new ResourceRecordSet()
.setName(absoluteHostName)
.setTtl((int) defaultATtl.getStandardSeconds())
.setTtl((int) getDnsAPlusAAAATtlForHost(hostName, defaultATtl))
.setType("AAAA")
.setKind("dns#resourceRecordSet")
.setRrdatas(ImmutableList.copyOf(aaaaRrData)));
@@ -276,11 +279,12 @@ public class CloudDnsWriter extends BaseDnsWriter {
}
/** Returns the glue records for in-bailiwick nameservers for the given domain+records. */
private Stream<String> filterGlueRecords(String domainName, Stream<ResourceRecordSet> records) {
private static Stream<String> filterGlueRecords(
String domainName, Stream<ResourceRecordSet> records) {
return records
.filter(record -> record.getType().equals("NS"))
.filter(record -> "NS".equals(record.getType()))
.flatMap(record -> record.getRrdatas().stream())
.filter(hostName -> hostName.endsWith("." + domainName) && !hostName.equals(domainName));
.filter(hostName -> hostName.endsWith('.' + domainName) && !hostName.equals(domainName));
}
/** Mutate the zone with the provided map of hostnames to desired DNS records. */
@@ -363,8 +367,8 @@ public class CloudDnsWriter extends BaseDnsWriter {
* <p>This call should be used in conjunction with {@link #getResourceRecordsForDomains} in a
* get-and-set retry loop.
*
* <p>See {@link "https://cloud.google.com/dns/troubleshooting"} for a list of errors produced by
* the Google Cloud DNS API.
* <p>See {@link "<a href="https://cloud.google.com/dns/troubleshooting">Troubleshoot Cloud
* DNS</a>"} for a list of errors produced by the Google Cloud DNS API.
*
* @throws ZoneStateException if the operation could not be completely successfully because the
* records to delete do not exist, already exist or have been modified with different
@@ -417,12 +421,12 @@ public class CloudDnsWriter extends BaseDnsWriter {
* @param hostName the fully qualified hostname
*/
private static String getAbsoluteHostName(String hostName) {
return hostName.endsWith(".") ? hostName : hostName + ".";
return hostName.endsWith(".") ? hostName : hostName + '.';
}
/** Zone state on Cloud DNS does not match the expected state. */
static class ZoneStateException extends RuntimeException {
public ZoneStateException(String reason) {
ZoneStateException(String reason) {
super("Zone state on Cloud DNS does not match the expected state: " + reason);
}
}
@@ -22,7 +22,7 @@ import dagger.Provides;
import dagger.multibindings.IntoMap;
import dagger.multibindings.IntoSet;
import dagger.multibindings.StringKey;
import google.registry.config.CredentialModule.DefaultCredential;
import google.registry.config.CredentialModule.ApplicationDefaultCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.dns.writer.DnsWriter;
import google.registry.util.GoogleCredentialsBundle;
@@ -35,7 +35,7 @@ public abstract class CloudDnsWriterModule {
@Provides
static Dns provideDns(
@DefaultCredential GoogleCredentialsBundle credentialsBundle,
@ApplicationDefaultCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId,
@Config("cloudDnsRootUrl") Optional<String> rootUrl,
@Config("cloudDnsServicePath") Optional<String> servicePath) {
@@ -18,6 +18,7 @@ import static com.google.common.base.Preconditions.checkState;
import static com.google.common.base.Verify.verify;
import static com.google.common.collect.Sets.intersection;
import static com.google.common.collect.Sets.union;
import static google.registry.dns.DnsUtils.getDnsAPlusAAAATtlForHost;
import static google.registry.model.EppResourceUtils.loadByForeignKey;
import com.google.common.base.Joiner;
@@ -31,6 +32,7 @@ import google.registry.model.domain.Domain;
import google.registry.model.domain.secdns.DomainDsData;
import google.registry.model.host.Host;
import google.registry.model.tld.Registries;
import google.registry.model.tld.Tld;
import google.registry.util.Clock;
import java.io.IOException;
import java.net.Inet4Address;
@@ -185,12 +187,13 @@ public class DnsUpdateWriter extends BaseDnsWriter {
private RRset makeDelegationSignerSet(Domain domain) {
RRset signerSet = new RRset();
Tld tld = Tld.get(domain.getTld());
for (DomainDsData signerData : domain.getDsData()) {
DSRecord dsRecord =
new DSRecord(
toAbsoluteName(domain.getDomainName()),
DClass.IN,
dnsDefaultDsTtl.getStandardSeconds(),
tld.getDnsDsTtl().orElse(dnsDefaultDsTtl).getStandardSeconds(),
signerData.getKeyTag(),
signerData.getAlgorithm(),
signerData.getDigestType(),
@@ -224,12 +227,13 @@ public class DnsUpdateWriter extends BaseDnsWriter {
private RRset makeNameServerSet(Domain domain) {
RRset nameServerSet = new RRset();
Tld tld = Tld.get(domain.getTld());
for (String hostName : domain.loadNameserverHostNames()) {
NSRecord record =
new NSRecord(
toAbsoluteName(domain.getDomainName()),
DClass.IN,
dnsDefaultNsTtl.getStandardSeconds(),
tld.getDnsNsTtl().orElse(dnsDefaultNsTtl).getStandardSeconds(),
toAbsoluteName(hostName));
nameServerSet.addRR(record);
}
@@ -244,7 +248,7 @@ public class DnsUpdateWriter extends BaseDnsWriter {
new ARecord(
toAbsoluteName(host.getHostName()),
DClass.IN,
dnsDefaultATtl.getStandardSeconds(),
getDnsAPlusAAAATtlForHost(host.getHostName(), dnsDefaultATtl),
address);
addressSet.addRR(record);
}
@@ -260,7 +264,7 @@ public class DnsUpdateWriter extends BaseDnsWriter {
new AAAARecord(
toAbsoluteName(host.getHostName()),
DClass.IN,
dnsDefaultATtl.getStandardSeconds(),
getDnsAPlusAAAATtlForHost(host.getHostName(), dnsDefaultATtl),
address);
addressSet.addRR(record);
}
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<taskentries>
<entries>
<task>
<url>/_dr/task/rdeStaging</url>
<name>rdeStaging</name>
@@ -80,12 +80,13 @@
</task>
<task>
<url><![CDATA[/_dr/task/expandRecurringBillingEvents?advanceCursor]]></url>
<name>expandRecurringBillingEvents</name>
<url><![CDATA[/_dr/task/expandBillingRecurrences?advanceCursor]]></url>
<name>expandBillingRecurrences</name>
<description>
This job runs an action that creates synthetic OneTime billing events from Recurring billing
events. Events are created for all instances of Recurring billing events that should exist
between the RECURRING_BILLING cursor's time and the execution time of the action.
This job runs an action that creates synthetic one-time billing events
from billing recurrences. Events are created for all recurrences that
should exist between the RECURRING_BILLING cursor's time and the execution
time of the action.
</description>
<schedule>0 3 * * *</schedule>
</task>
@@ -128,16 +129,6 @@
<schedule>0 5 * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/cron/readDnsQueue?jitterSeconds=45]]></url>
<name>readDnsQueue</name>
<description>
Lease all tasks from the dns-pull queue, group by TLD, and invoke PublishDnsUpdates for each
group.
</description>
<schedule>*/1 * * * *</schedule>
</task>
<task>
<url>
<![CDATA[/_dr/cron/fanout?queue=dns-refresh&forEachRealTld&forEachTestTld&endpoint=/_dr/task/readDnsRefreshRequests&dnsJitterSeconds=45]]></url>
@@ -147,4 +138,4 @@
</description>
<schedule>*/1 * * * *</schedule>
</task>
</taskentries>
</entries>
@@ -151,12 +151,6 @@
<url-pattern>/_dr/task/nordnVerify</url-pattern>
</servlet-mapping>
<!-- Reads the DNS push and pull queues and kick off the appropriate tasks to update zone. -->
<servlet-mapping>
<servlet-name>backend-servlet</servlet-name>
<url-pattern>/_dr/cron/readDnsQueue</url-pattern>
</servlet-mapping>
<!-- Reads the DNS refresh requests and kick off the appropriate tasks to update zone. -->
<servlet-mapping>
<servlet-name>backend-servlet</servlet-name>
@@ -246,10 +240,10 @@
<url-pattern>/_dr/task/refreshDnsOnHostRename</url-pattern>
</servlet-mapping>
<!-- Action to expand recurring billing events into OneTimes. -->
<!-- Action to expand BillingRecurrences into BillingEvents. -->
<servlet-mapping>
<servlet-name>backend-servlet</servlet-name>
<url-pattern>/_dr/task/expandRecurringBillingEvents</url-pattern>
<url-pattern>/_dr/task/expandBillingRecurrences</url-pattern>
</servlet-mapping>
<!-- Background action to delete domains past end of autorenewal. -->
@@ -0,0 +1,113 @@
<?xml version="1.0" encoding="UTF-8"?>
<entries>
<!-- Queue template with all supported params -->
<!-- More information - https://cloud.google.com/sdk/gcloud/reference/tasks/queues/create -->
<!--
<queue>
<name></name>
<max-attempts></max-attempts>
<max-backoff></max-backoff>
<max-concurrent-dispatches></max-concurrent-dispatches>
<max-dispatches-per-second></max-dispatches-per-second>
<max-doublings></max-doublings>
<max-retry-duration></max-retry-duration>
<min-backoff></min-backoff>
</queue>
-->
<!-- Queue for reading DNS update requests and batching them off to the dns-publish queue. -->
<queue>
<name>dns-refresh</name>
<max-dispatches-per-second>100</max-dispatches-per-second>
</queue>
<!-- Queue for publishing DNS updates in batches. -->
<queue>
<name>dns-publish</name>
<max-dispatches-per-second>100</max-dispatches-per-second>
<!-- 30 sec backoff increasing linearly up to 30 minutes. -->
<min-backoff>30s</min-backoff>
<max-backoff>1800s</max-backoff>
<max-doublings>0</max-doublings>
</queue>
<!-- Queue for uploading RDE deposits to the escrow provider. -->
<queue>
<name>rde-upload</name>
<max-dispatches-per-second>0.166666667</max-dispatches-per-second>
<max-concurrent-dispatches>5</max-concurrent-dispatches>
<max-retry-duration>14400s</max-retry-duration>
</queue>
<!-- Queue for uploading RDE reports to ICANN. -->
<queue>
<name>rde-report</name>
<max-dispatches-per-second>1</max-dispatches-per-second>
<max-concurrent-dispatches>1</max-concurrent-dispatches>
<max-retry-duration>14400s</max-retry-duration>
</queue>
<!-- Queue for copying BRDA deposits to GCS. -->
<queue>
<name>brda</name>
<max-dispatches-per-second>0.016666667</max-dispatches-per-second>
<max-concurrent-dispatches>10</max-concurrent-dispatches>
<max-retry-duration>82800s</max-retry-duration>
</queue>
<!-- Queue for tasks that trigger domain DNS update upon host rename. -->
<queue>
<name>async-host-rename</name>
<max-dispatches-per-second>1</max-dispatches-per-second>
</queue>
<!-- Queue for tasks that wait for a Beam pipeline to complete (i.e. Spec11 and invoicing). -->
<queue>
<name>beam-reporting</name>
<max-dispatches-per-second>0.016666667</max-dispatches-per-second>
<max-concurrent-dispatches>1</max-concurrent-dispatches>
<max-attempts>5</max-attempts>
<min-backoff>180s</min-backoff>
<max-backoff>180s</max-backoff>
</queue>
<!-- Queue for tasks that communicate with TMCH MarksDB webserver. -->
<queue>
<name>marksdb</name>
<max-dispatches-per-second>0.016666667</max-dispatches-per-second>
<max-concurrent-dispatches>1</max-concurrent-dispatches>
<max-retry-duration>39600s</max-retry-duration> <!-- cron interval minus hour -->
</queue>
<!-- Queue for tasks to produce LORDN CSV reports, populated by a Cloud Scheduler fanout job. -->
<queue>
<name>nordn</name>
<max-dispatches-per-second>1</max-dispatches-per-second>
<max-concurrent-dispatches>10</max-concurrent-dispatches>
<max-retry-duration>39600s</max-retry-duration> <!-- cron interval minus hour -->
</queue>
<!-- Queue for tasks that sync data to Google Spreadsheets. -->
<queue>
<name>sheet</name>
<max-dispatches-per-second>1</max-dispatches-per-second>
<!-- max-concurrent-dispatches is intentionally omitted. -->
<max-retry-duration>3600s</max-retry-duration>
</queue>
<!-- Queue for infrequent cron tasks (i.e. hourly or less often) that should retry three times on failure. -->
<queue>
<name>retryable-cron-tasks</name>
<max-dispatches-per-second>1</max-dispatches-per-second>
<max-attempts>3</max-attempts>
</queue>
<!-- &lt;!&ndash; Queue for async actions that should be run at some point in the future. &ndash;&gt;-->
<queue>
<name>async-actions</name>
<max-dispatches-per-second>1</max-dispatches-per-second>
<max-concurrent-dispatches>5</max-concurrent-dispatches>
</queue>
</entries>
@@ -1,11 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<!-- TODO: @ptkach - Delete once Cloud Api deployer is up and running -->
<queue-entries>
<queue>
<name>dns-pull</name>
<mode>pull</mode>
</queue>
<!-- Queue for reading DNS update requests and batching them off to the dns-publish queue. -->
<queue>
<name>dns-refresh</name>
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<taskentries>
<entries>
<!--
/cron/fanout params:
@@ -129,16 +129,6 @@
<schedule>0 5 * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/cron/readDnsQueue?jitterSeconds=45]]></url>
<name>readDnsQueue</name>
<description>
Lease all tasks from the dns-pull queue, group by TLD, and invoke PublishDnsUpdates for each
group.
</description>
<schedule>*/1 * * * *</schedule>
</task>
<task>
<url>
<![CDATA[/_dr/cron/fanout?queue=dns-refresh&forEachRealTld&forEachTestTld&endpoint=/_dr/task/readDnsRefreshRequests&dnsJitterSeconds=45]]></url>
@@ -158,16 +148,4 @@
</description>
<schedule>7 3 * * *</schedule>
</task>
<!--
The next two wipeout jobs are required when crash has production data.
-->
<task>
<url><![CDATA[/_dr/task/wipeOutCloudSql]]></url>
<name>wipeOutCloudSql</name>
<description>
This job runs an action that deletes all data in Cloud SQL.
</description>
<schedule>7 3 * * 6</schedule>
</task>
</taskentries>
</entries>
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<taskentries>
<entries>
<task>
<url>/_dr/task/rdeStaging</url>
<name>rdeStaging</name>
@@ -122,12 +122,13 @@
</task>
<task>
<url><![CDATA[/_dr/task/expandRecurringBillingEvents?advanceCursor]]></url>
<name>expandRecurringBillingEvents</name>
<url><![CDATA[/_dr/task/expandBillingRecurrences?advanceCursor]]></url>
<name>expandBillingRecurrences</name>
<description>
This job runs an action that creates synthetic OneTime billing events from Recurring billing
events. Events are created for all instances of Recurring billing events that should exist
between the RECURRING_BILLING cursor's time and the execution time of the action.
This job runs an action that creates synthetic one-time billing events
from billing recurrences. Events are created for all recurrences that
should exist between the RECURRING_BILLING cursor's time and the execution
time of the action.
</description>
<schedule>0 3 * * *</schedule>
</task>
@@ -200,16 +201,6 @@
<schedule>0 5 * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/cron/readDnsQueue?jitterSeconds=45]]></url>
<name>readDnsQueue</name>
<description>
Lease all tasks from the dns-pull queue, group by TLD, and invoke PublishDnsUpdates for each
group.
</description>
<schedule>*/1 * * * *</schedule>
</task>
<task>
<url>
<![CDATA[/_dr/cron/fanout?queue=dns-refresh&forEachRealTld&forEachTestTld&endpoint=/_dr/task/readDnsRefreshRequests&dnsJitterSeconds=45]]></url>
@@ -253,7 +244,7 @@
reports to the associated registrars' drive folders.
See GenerateInvoicesAction for more details.
</description>
<!--WARNING: This must occur AFTER expandRecurringBillingEvents and AFTER exportSnapshot, as
<!--WARNING: This must occur AFTER expandBillingRecurrences and AFTER exportSnapshot, as
it uses Bigquery as the source of truth for billable events. ExportSnapshot usually takes
about 2 hours to complete, so we give 11 hours to be safe. Normally, we give 24+ hours (see
icannReportingStaging), but the invoicing team prefers receiving the e-mail on the first of
@@ -282,4 +273,4 @@
</description>
<schedule>0 15 * * 1</schedule>
</task>
</taskentries>
</entries>
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<taskentries>
<entries>
<task>
<url>/_dr/task/rdeStaging</url>
<name>rdeStaging</name>
@@ -11,16 +11,6 @@
<schedule>7 0 * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/cron/readDnsQueue?jitterSeconds=45]]></url>
<name>readDnsQueue</name>
<description>
Lease all tasks from the dns-pull queue, group by TLD, and invoke PublishDnsUpdates for each
group.
</description>
<schedule>*/1 * * * *</schedule>
</task>
<task>
<url>
<![CDATA[/_dr/cron/fanout?queue=dns-refresh&forEachRealTld&forEachTestTld&endpoint=/_dr/task/readDnsRefreshRequests&dnsJitterSeconds=45]]></url>
@@ -78,4 +68,4 @@
</description>
<schedule>7 3 * * 6</schedule>
</task>
</taskentries>
</entries>
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<taskentries>
<entries>
<task>
<url>/_dr/task/rdeStaging</url>
<name>rdeStaging</name>
@@ -82,12 +82,13 @@
</task>
<task>
<url><![CDATA[/_dr/task/expandRecurringBillingEvents?advanceCursor]]></url>
<name>expandRecurringBillingEvents</name>
<url><![CDATA[/_dr/task/expandBillingRecurrences?advanceCursor]]></url>
<name>expandBillingRecurrences</name>
<description>
This job runs an action that creates synthetic OneTime billing events from Recurring billing
events. Events are created for all instances of Recurring billing events that should exist
between the RECURRING_BILLING cursor's time and the execution time of the action.
This job runs an action that creates synthetic one-time billing events
from billing recurrences. Events are created for all recurrences that
should exist between the RECURRING_BILLING cursor's time and the execution
time of the action.
</description>
<schedule>0 3 * * *</schedule>
</task>
@@ -142,16 +143,6 @@
<schedule>0 5 * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/cron/readDnsQueue?jitterSeconds=45]]></url>
<name>readDnsQueue</name>
<description>
Lease all tasks from the dns-pull queue, group by TLD, and invoke PublishDnsUpdates for each
group.
</description>
<schedule>*/1 * * * *</schedule>
</task>
<task>
<url>
<![CDATA[/_dr/cron/fanout?queue=dns-refresh&forEachRealTld&forEachTestTld&endpoint=/_dr/task/readDnsRefreshRequests&dnsJitterSeconds=45]]></url>
@@ -171,4 +162,4 @@
</description>
<schedule>0 15 * * 1</schedule>
</task>
</taskentries>
</entries>
@@ -50,6 +50,7 @@ import google.registry.flows.annotations.ReportingSpec;
import google.registry.flows.custom.DomainCheckFlowCustomLogic;
import google.registry.flows.custom.DomainCheckFlowCustomLogic.BeforeResponseParameters;
import google.registry.flows.custom.DomainCheckFlowCustomLogic.BeforeResponseReturnData;
import google.registry.flows.domain.DomainPricingLogic.AllocationTokenInvalidForPremiumNameException;
import google.registry.flows.domain.token.AllocationTokenDomainCheckResults;
import google.registry.flows.domain.token.AllocationTokenFlowUtils;
import google.registry.flows.domain.token.AllocationTokenFlowUtils.AllocationTokenNotInPromotionException;
@@ -59,7 +60,7 @@ import google.registry.flows.domain.token.AllocationTokenFlowUtils.AllocationTok
import google.registry.flows.domain.token.AllocationTokenFlowUtils.AllocationTokenNotValidForTldException;
import google.registry.model.EppResource;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Check;
import google.registry.model.domain.fee.FeeCheckCommandExtension;
@@ -81,6 +82,7 @@ import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldState;
import google.registry.model.tld.label.ReservationType;
import google.registry.persistence.VKey;
import google.registry.pricing.PricingEngineProxy;
import google.registry.util.Clock;
import java.util.HashSet;
import java.util.Optional;
@@ -271,8 +273,7 @@ public final class DomainCheckFlow implements Flow {
new ImmutableList.Builder<>();
ImmutableMap<String, Domain> domainObjs =
loadDomainsForRestoreChecks(feeCheck, domainNames, existingDomains);
ImmutableMap<String, BillingEvent.Recurring> recurrences =
loadRecurrencesForDomains(domainObjs);
ImmutableMap<String, BillingRecurrence> recurrences = loadRecurrencesForDomains(domainObjs);
for (FeeCheckCommandExtensionItem feeCheckItem : feeCheck.getItems()) {
for (String domainName : getDomainNamesToCheckForFee(feeCheckItem, domainNames.keySet())) {
@@ -292,6 +293,7 @@ public final class DomainCheckFlow implements Flow {
allocationToken.get(),
feeCheckItem.getCommandName(),
registrarId,
PricingEngineProxy.isDomainPremium(domainName, now),
now);
}
handleFeeRequest(
@@ -306,7 +308,8 @@ public final class DomainCheckFlow implements Flow {
availableDomains.contains(domainName),
recurrences.getOrDefault(domainName, null));
responseItems.add(builder.setDomainNameIfSupported(domainName).build());
} catch (AllocationTokenNotValidForCommandException
} catch (AllocationTokenInvalidForPremiumNameException
| AllocationTokenNotValidForCommandException
| AllocationTokenNotValidForDomainException
| AllocationTokenNotValidForRegistrarException
| AllocationTokenNotValidForTldException
@@ -377,16 +380,15 @@ public final class DomainCheckFlow implements Flow {
Maps.transformEntries(existingDomainsToLoad, (k, v) -> (Domain) loadedDomains.get(v)));
}
private ImmutableMap<String, BillingEvent.Recurring> loadRecurrencesForDomains(
private ImmutableMap<String, BillingRecurrence> loadRecurrencesForDomains(
ImmutableMap<String, Domain> domainObjs) {
return tm().transact(
() -> {
ImmutableMap<VKey<? extends BillingEvent.Recurring>, BillingEvent.Recurring>
recurrences =
tm().loadByKeys(
domainObjs.values().stream()
.map(Domain::getAutorenewBillingEvent)
.collect(toImmutableSet()));
ImmutableMap<VKey<? extends BillingRecurrence>, BillingRecurrence> recurrences =
tm().loadByKeys(
domainObjs.values().stream()
.map(Domain::getAutorenewBillingEvent)
.collect(toImmutableSet()));
return ImmutableMap.copyOf(
Maps.transformValues(
domainObjs, d -> recurrences.get(d.getAutorenewBillingEvent())));
@@ -16,6 +16,7 @@ package google.registry.flows.domain;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.flows.FlowUtils.persistEntityChanges;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.verifyResourceDoesNotExist;
@@ -59,7 +60,6 @@ import com.google.auto.value.AutoValue;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.net.InternetDomainName;
import google.registry.dns.DnsUtils;
import google.registry.flows.EppException;
import google.registry.flows.EppException.CommandUseErrorException;
import google.registry.flows.EppException.ParameterValuePolicyErrorException;
@@ -77,11 +77,11 @@ import google.registry.flows.domain.token.AllocationTokenFlowUtils;
import google.registry.flows.exceptions.ResourceAlreadyExistsForThisClientException;
import google.registry.flows.exceptions.ResourceCreateContentionException;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingBase.Flag;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand;
import google.registry.model.domain.DomainCommand.Create;
@@ -227,7 +227,6 @@ public final class DomainCreateFlow implements TransactionalFlow {
@Inject DomainCreateFlowCustomLogic flowCustomLogic;
@Inject DomainFlowTmchUtils tmchUtils;
@Inject DomainPricingLogic pricingLogic;
@Inject DnsUtils dnsUtils;
@Inject DomainCreateFlow() {}
@@ -348,8 +347,8 @@ public final class DomainCreateFlow implements TransactionalFlow {
HistoryEntryId domainHistoryId = new HistoryEntryId(repoId, historyRevisionId);
historyBuilder.setRevisionId(historyRevisionId);
// Bill for the create.
BillingEvent.OneTime createBillingEvent =
createOneTimeBillingEvent(
BillingEvent createBillingEvent =
createBillingEvent(
tld,
isAnchorTenant,
isSunriseCreate,
@@ -360,7 +359,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
allocationToken,
now);
// Create a new autorenew billing event and poll message starting at the expiration time.
BillingEvent.Recurring autorenewBillingEvent =
BillingRecurrence autorenewBillingEvent =
createAutorenewBillingEvent(
domainHistoryId,
registrationExpirationTime,
@@ -426,7 +425,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
allocationToken.get(), domainHistory.getHistoryEntryId()));
}
if (domain.shouldPublishToDns()) {
dnsUtils.requestDomainDnsRefresh(domain.getDomainName());
requestDomainDnsRefresh(domain.getDomainName());
}
EntityChanges entityChanges =
flowCustomLogic.beforeSave(
@@ -572,7 +571,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
return historyBuilder.setType(DOMAIN_CREATE).setPeriod(period).setDomain(domain).build();
}
private BillingEvent.OneTime createOneTimeBillingEvent(
private BillingEvent createBillingEvent(
Tld tld,
boolean isAnchorTenant,
boolean isSunriseCreate,
@@ -594,7 +593,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
// it if it's reserved for other reasons.
flagsBuilder.add(Flag.RESERVED);
}
return new BillingEvent.OneTime.Builder()
return new BillingEvent.Builder()
.setReason(Reason.CREATE)
.setTargetId(targetId)
.setRegistrarId(registrarId)
@@ -612,11 +611,11 @@ public final class DomainCreateFlow implements TransactionalFlow {
.build();
}
private Recurring createAutorenewBillingEvent(
private BillingRecurrence createAutorenewBillingEvent(
HistoryEntryId domainHistoryId,
DateTime registrationExpirationTime,
RenewalPriceInfo renewalpriceInfo) {
return new BillingEvent.Recurring.Builder()
return new BillingRecurrence.Builder()
.setReason(Reason.RENEW)
.setFlags(ImmutableSet.of(Flag.AUTO_RENEW))
.setTargetId(targetId)
@@ -640,9 +639,9 @@ public final class DomainCreateFlow implements TransactionalFlow {
.build();
}
private static BillingEvent.OneTime createEapBillingEvent(
FeesAndCredits feesAndCredits, BillingEvent.OneTime createBillingEvent) {
return new BillingEvent.OneTime.Builder()
private static BillingEvent createEapBillingEvent(
FeesAndCredits feesAndCredits, BillingEvent createBillingEvent) {
return new BillingEvent.Builder()
.setReason(Reason.FEE_EARLY_ACCESS)
.setTargetId(createBillingEvent.getTargetId())
.setRegistrarId(createBillingEvent.getRegistrarId())
@@ -671,7 +670,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
/**
* Determines the {@link RenewalPriceBehavior} and the renewal price that needs be stored in the
* {@link Recurring} billing events.
* {@link BillingRecurrence} billing events.
*
* <p>By default, the renewal price is calculated during the process of renewal. Renewal price
* should be the createCost if and only if the renewal price behavior in the {@link
@@ -697,7 +696,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
}
}
/** A class to store renewal info used in {@link Recurring} billing events. */
/** A class to store renewal info used in {@link BillingRecurrence} billing events. */
@AutoValue
public abstract static class RenewalPriceInfo {
static DomainCreateFlow.RenewalPriceInfo create(
@@ -16,6 +16,7 @@ package google.registry.flows.domain;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Strings.isNullOrEmpty;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.persistEntityChanges;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
@@ -44,7 +45,6 @@ import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Sets;
import google.registry.batch.AsyncTaskEnqueuer;
import google.registry.dns.DnsUtils;
import google.registry.flows.EppException;
import google.registry.flows.EppException.AssociationProhibitsOperationException;
import google.registry.flows.ExtensionManager;
@@ -61,8 +61,9 @@ import google.registry.flows.custom.DomainDeleteFlowCustomLogic.BeforeResponseRe
import google.registry.flows.custom.DomainDeleteFlowCustomLogic.BeforeSaveParameters;
import google.registry.flows.custom.EntityChanges;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingCancellation;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.GracePeriod;
@@ -129,7 +130,6 @@ public final class DomainDeleteFlow implements TransactionalFlow {
@Inject @TargetId String targetId;
@Inject @Superuser boolean isSuperuser;
@Inject DomainHistory.Builder historyBuilder;
@Inject DnsUtils dnsUtils;
@Inject Trid trid;
@Inject AsyncTaskEnqueuer asyncTaskEnqueuer;
@Inject EppResponse.Builder responseBuilder;
@@ -232,15 +232,15 @@ public final class DomainDeleteFlow implements TransactionalFlow {
// No cancellation is written if the grace period was not for a billable event.
if (gracePeriod.hasBillingEvent()) {
entitiesToSave.add(
BillingEvent.Cancellation.forGracePeriod(gracePeriod, now, domainHistoryId, targetId));
if (gracePeriod.getOneTimeBillingEvent() != null) {
BillingCancellation.forGracePeriod(gracePeriod, now, domainHistoryId, targetId));
if (gracePeriod.getBillingEvent() != null) {
// Take the amount of registration time being refunded off the expiration time.
// This can be either add grace periods or renew grace periods.
BillingEvent.OneTime oneTime = tm().loadByKey(gracePeriod.getOneTimeBillingEvent());
newExpirationTime = newExpirationTime.minusYears(oneTime.getPeriodYears());
} else if (gracePeriod.getRecurringBillingEvent() != null) {
BillingEvent billingEvent = tm().loadByKey(gracePeriod.getBillingEvent());
newExpirationTime = newExpirationTime.minusYears(billingEvent.getPeriodYears());
} else if (gracePeriod.getBillingRecurrence() != null) {
// Take 1 year off the registration if in the autorenew grace period (no need to load the
// recurring billing event; all autorenews are for 1 year).
// recurrence billing event; all autorenews are for 1 year).
newExpirationTime = newExpirationTime.minusYears(1);
}
}
@@ -252,15 +252,16 @@ public final class DomainDeleteFlow implements TransactionalFlow {
buildDomainHistory(newDomain, tld, now, durationUntilDelete, inAddGracePeriod);
handlePendingTransferOnDelete(existingDomain, newDomain, now, domainHistory);
// Close the autorenew billing event and poll message. This may delete the poll message. Store
// the updated recurring billing event, we'll need it later and can't reload it.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
BillingEvent.Recurring recurringBillingEvent =
// the updated recurrence billing event, we'll need it later and can't reload it.
BillingRecurrence existingBillingRecurrence =
tm().loadByKey(existingDomain.getAutorenewBillingEvent());
BillingRecurrence billingRecurrence =
updateAutorenewRecurrenceEndTime(
existingDomain, existingRecurring, now, domainHistory.getHistoryEntryId());
existingDomain, existingBillingRecurrence, now, domainHistory.getHistoryEntryId());
// If there's a pending transfer, the gaining client's autorenew billing
// event and poll message will already have been deleted in
// ResourceDeleteFlow since it's listed in serverApproveEntities.
dnsUtils.requestDomainDnsRefresh(existingDomain.getDomainName());
requestDomainDnsRefresh(existingDomain.getDomainName());
entitiesToSave.add(newDomain, domainHistory);
EntityChanges entityChanges =
@@ -280,7 +281,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
? SUCCESS_WITH_ACTION_PENDING
: SUCCESS)
.setResponseExtensions(
getResponseExtensions(recurringBillingEvent, existingDomain, now))
getResponseExtensions(billingRecurrence, existingDomain, now))
.build());
persistEntityChanges(entityChanges);
return responseBuilder
@@ -380,7 +381,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
@Nullable
private ImmutableList<FeeTransformResponseExtension> getResponseExtensions(
BillingEvent.Recurring recurringBillingEvent, Domain existingDomain, DateTime now) {
BillingRecurrence billingRecurrence, Domain existingDomain, DateTime now) {
FeeTransformResponseExtension.Builder feeResponseBuilder = getDeleteResponseBuilder();
if (feeResponseBuilder == null) {
return ImmutableList.of();
@@ -388,7 +389,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
ImmutableList.Builder<Credit> creditsBuilder = new ImmutableList.Builder<>();
for (GracePeriod gracePeriod : existingDomain.getGracePeriods()) {
if (gracePeriod.hasBillingEvent()) {
Money cost = getGracePeriodCost(recurringBillingEvent, gracePeriod, now);
Money cost = getGracePeriodCost(billingRecurrence, gracePeriod, now);
creditsBuilder.add(Credit.create(
cost.negated().getAmount(), FeeType.CREDIT, gracePeriod.getType().getXmlName()));
feeResponseBuilder.setCurrency(checkNotNull(cost.getCurrencyUnit()));
@@ -402,14 +403,14 @@ public final class DomainDeleteFlow implements TransactionalFlow {
}
private Money getGracePeriodCost(
BillingEvent.Recurring recurringBillingEvent, GracePeriod gracePeriod, DateTime now) {
BillingRecurrence billingRecurrence, GracePeriod gracePeriod, DateTime now) {
if (gracePeriod.getType() == GracePeriodStatus.AUTO_RENEW) {
// If we updated the autorenew billing event, reuse it.
DateTime autoRenewTime =
recurringBillingEvent.getRecurrenceTimeOfYear().getLastInstanceBeforeOrAt(now);
billingRecurrence.getRecurrenceTimeOfYear().getLastInstanceBeforeOrAt(now);
return getDomainRenewCost(targetId, autoRenewTime, 1);
}
return tm().loadByKey(checkNotNull(gracePeriod.getOneTimeBillingEvent())).getCost();
return tm().loadByKey(checkNotNull(gracePeriod.getBillingEvent())).getCost();
}
@Nullable
@@ -429,7 +430,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
/** Domain to be deleted has subordinate hosts. */
static class DomainToDeleteHasHostsException extends AssociationProhibitsOperationException {
public DomainToDeleteHasHostsException() {
DomainToDeleteHasHostsException() {
super("Domain to be deleted has subordinate hosts");
}
}
@@ -74,13 +74,13 @@ import google.registry.flows.EppException.ParameterValueSyntaxErrorException;
import google.registry.flows.EppException.RequiredParameterMissingException;
import google.registry.flows.EppException.StatusProhibitsOperationException;
import google.registry.flows.EppException.UnimplementedOptionException;
import google.registry.flows.domain.DomainPricingLogic.AllocationTokenInvalidForPremiumNameException;
import google.registry.flows.domain.token.AllocationTokenFlowUtils;
import google.registry.flows.exceptions.ResourceHasClientUpdateProhibitedException;
import google.registry.model.EppResource;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingBase.Flag;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.contact.Contact;
import google.registry.model.domain.DesignatedContact;
import google.registry.model.domain.DesignatedContact.Type;
@@ -556,8 +556,8 @@ public class DomainFlowUtils {
* Fills in a builder with the data needed for an autorenew billing event for this domain. This
* does not copy over the id of the current autorenew billing event.
*/
public static BillingEvent.Recurring.Builder newAutorenewBillingEvent(Domain domain) {
return new BillingEvent.Recurring.Builder()
public static BillingRecurrence.Builder newAutorenewBillingEvent(Domain domain) {
return new BillingRecurrence.Builder()
.setReason(Reason.RENEW)
.setFlags(ImmutableSet.of(Flag.AUTO_RENEW))
.setTargetId(domain.getDomainName())
@@ -584,11 +584,11 @@ public class DomainFlowUtils {
* (if opening the message interval). This may cause an autorenew billing event to have an end
* time earlier than its event time (i.e. if it's being ended before it was ever triggered).
*
* <p>Returns the new autorenew recurring billing event.
* <p>Returns the new autorenew recurrence billing event.
*/
public static Recurring updateAutorenewRecurrenceEndTime(
public static BillingRecurrence updateAutorenewRecurrenceEndTime(
Domain domain,
Recurring existingRecurring,
BillingRecurrence existingBillingRecurrence,
DateTime newEndTime,
@Nullable HistoryEntryId historyId) {
Optional<PollMessage.Autorenew> autorenewPollMessage =
@@ -623,9 +623,10 @@ public class DomainFlowUtils {
tm().put(updatedAutorenewPollMessage);
}
Recurring newRecurring = existingRecurring.asBuilder().setRecurrenceEndTime(newEndTime).build();
tm().put(newRecurring);
return newRecurring;
BillingRecurrence newBillingRecurrence =
existingBillingRecurrence.asBuilder().setRecurrenceEndTime(newEndTime).build();
tm().put(newBillingRecurrence);
return newBillingRecurrence;
}
/**
@@ -642,7 +643,7 @@ public class DomainFlowUtils {
DomainPricingLogic pricingLogic,
Optional<AllocationToken> allocationToken,
boolean isAvailable,
@Nullable Recurring recurringBillingEvent)
@Nullable BillingRecurrence billingRecurrence)
throws EppException {
DateTime now = currentDate;
// Use the custom effective date specified in the fee check request, if there is one.
@@ -698,7 +699,7 @@ public class DomainFlowUtils {
fees =
pricingLogic
.getRenewPrice(
tld, domainNameString, now, years, recurringBillingEvent, allocationToken)
tld, domainNameString, now, years, billingRecurrence, allocationToken)
.getFees();
break;
case RESTORE:
@@ -724,9 +725,7 @@ public class DomainFlowUtils {
}
builder.setAvailIfSupported(true);
fees =
pricingLogic
.getTransferPrice(tld, domainNameString, now, recurringBillingEvent)
.getFees();
pricingLogic.getTransferPrice(tld, domainNameString, now, billingRecurrence).getFees();
break;
case UPDATE:
builder.setAvailIfSupported(true);
@@ -1219,8 +1218,15 @@ public class DomainFlowUtils {
for (Optional<AllocationToken> token : tokenList) {
try {
AllocationTokenFlowUtils.validateToken(
InternetDomainName.from(domainName), token.get(), commandName, registrarId, now);
} catch (AssociationProhibitsOperationException | StatusProhibitsOperationException e) {
InternetDomainName.from(domainName),
token.get(),
commandName,
registrarId,
isDomainPremium(domainName, now),
now);
} catch (AssociationProhibitsOperationException
| StatusProhibitsOperationException
| AllocationTokenInvalidForPremiumNameException e) {
// Allocation token was not valid for this registration, continue to check the next token in
// the list
continue;
@@ -16,6 +16,7 @@ package google.registry.flows.domain;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.flows.domain.DomainFlowUtils.zeroInCurrency;
import static google.registry.flows.domain.token.AllocationTokenFlowUtils.validateTokenForPossiblePremiumName;
import static google.registry.pricing.PricingEngineProxy.getPricesForDomainName;
import static google.registry.util.DomainNameUtils.getTldFromDomainName;
import static google.registry.util.PreconditionsUtils.checkArgumentPresent;
@@ -29,7 +30,7 @@ import google.registry.flows.custom.DomainPricingCustomLogic.RenewPriceParameter
import google.registry.flows.custom.DomainPricingCustomLogic.RestorePriceParameters;
import google.registry.flows.custom.DomainPricingCustomLogic.TransferPriceParameters;
import google.registry.flows.custom.DomainPricingCustomLogic.UpdatePriceParameters;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.fee.BaseFee;
import google.registry.model.domain.fee.BaseFee.FeeType;
import google.registry.model.domain.fee.Fee;
@@ -113,20 +114,20 @@ public final class DomainPricingLogic {
String domainName,
DateTime dateTime,
int years,
@Nullable Recurring recurringBillingEvent,
@Nullable BillingRecurrence billingRecurrence,
Optional<AllocationToken> allocationToken)
throws AllocationTokenInvalidForPremiumNameException {
checkArgument(years > 0, "Number of years must be positive");
Money renewCost;
DomainPrices domainPrices = getPricesForDomainName(domainName, dateTime);
boolean isRenewCostPremiumPrice;
// recurring billing event is null if the domain is still available. Billing events are created
// recurrence is null if the domain is still available. Billing events are created
// in the process of domain creation.
if (recurringBillingEvent == null) {
if (billingRecurrence == null) {
renewCost = getDomainRenewCostWithDiscount(domainPrices, years, allocationToken);
isRenewCostPremiumPrice = domainPrices.isPremium();
} else {
switch (recurringBillingEvent.getRenewalPriceBehavior()) {
switch (billingRecurrence.getRenewalPriceBehavior()) {
case DEFAULT:
renewCost = getDomainRenewCostWithDiscount(domainPrices, years, allocationToken);
isRenewCostPremiumPrice = domainPrices.isPremium();
@@ -135,11 +136,11 @@ public final class DomainPricingLogic {
// as the creation price, which is stored in the billing event as the renewal price
case SPECIFIED:
checkArgumentPresent(
recurringBillingEvent.getRenewalPrice(),
billingRecurrence.getRenewalPrice(),
"Unexpected behavior: renewal price cannot be null when renewal behavior is"
+ " SPECIFIED");
// Don't apply allocation token to renewal price when SPECIFIED
renewCost = recurringBillingEvent.getRenewalPrice().get().multipliedBy(years);
renewCost = billingRecurrence.getRenewalPrice().get().multipliedBy(years);
isRenewCostPremiumPrice = false;
break;
// if the renewal price behavior is nonpremium, it means that the domain should be renewed
@@ -157,7 +158,7 @@ public final class DomainPricingLogic {
throw new IllegalArgumentException(
String.format(
"Unknown RenewalPriceBehavior enum value: %s",
recurringBillingEvent.getRenewalPriceBehavior()));
billingRecurrence.getRenewalPriceBehavior()));
}
}
return customLogic.customizeRenewPrice(
@@ -200,10 +201,10 @@ public final class DomainPricingLogic {
/** Returns a new transfer price for the pricer. */
FeesAndCredits getTransferPrice(
Tld tld, String domainName, DateTime dateTime, @Nullable Recurring recurringBillingEvent)
Tld tld, String domainName, DateTime dateTime, @Nullable BillingRecurrence billingRecurrence)
throws EppException {
FeesAndCredits renewPrice =
getRenewPrice(tld, domainName, dateTime, 1, recurringBillingEvent, Optional.empty());
getRenewPrice(tld, domainName, dateTime, 1, billingRecurrence, Optional.empty());
return customLogic.customizeTransferPrice(
TransferPriceParameters.newBuilder()
.setFeesAndCredits(
@@ -257,12 +258,7 @@ public final class DomainPricingLogic {
private Money getDomainCostWithDiscount(
boolean isPremium, int years, Optional<AllocationToken> allocationToken, Money oneYearCost)
throws AllocationTokenInvalidForPremiumNameException {
if (allocationToken.isPresent()
&& allocationToken.get().getDiscountFraction() != 0.0
&& isPremium
&& !allocationToken.get().shouldDiscountPremiums()) {
throw new AllocationTokenInvalidForPremiumNameException();
}
validateTokenForPossiblePremiumName(allocationToken, isPremium);
Money totalDomainFlowCost = oneYearCost.multipliedBy(years);
// Apply the allocation token discount, if applicable.
@@ -281,8 +277,8 @@ public final class DomainPricingLogic {
/** An allocation token was provided that is invalid for premium domains. */
public static class AllocationTokenInvalidForPremiumNameException
extends CommandUseErrorException {
AllocationTokenInvalidForPremiumNameException() {
super("A nonzero discount code cannot be applied to premium domains");
public AllocationTokenInvalidForPremiumNameException() {
super("Token not valid for premium name");
}
}
}
@@ -54,10 +54,9 @@ import google.registry.flows.custom.DomainRenewFlowCustomLogic.BeforeSaveParamet
import google.registry.flows.custom.EntityChanges;
import google.registry.flows.domain.token.AllocationTokenFlowUtils;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Renew;
import google.registry.model.domain.DomainHistory;
@@ -203,7 +202,7 @@ public final class DomainRenewFlow implements TransactionalFlow {
validateRegistrationPeriod(now, newExpirationTime);
Optional<FeeRenewCommandExtension> feeRenew =
eppInput.getSingleExtension(FeeRenewCommandExtension.class);
Recurring existingRecurringBillingEvent =
BillingRecurrence existingBillingRecurrence =
tm().loadByKey(existingDomain.getAutorenewBillingEvent());
FeesAndCredits feesAndCredits =
pricingLogic.getRenewPrice(
@@ -211,7 +210,7 @@ public final class DomainRenewFlow implements TransactionalFlow {
targetId,
now,
years,
existingRecurringBillingEvent,
existingBillingRecurrence,
allocationToken);
validateFeeChallenge(feeRenew, feesAndCredits, defaultTokenUsed);
flowCustomLogic.afterValidation(
@@ -223,15 +222,15 @@ public final class DomainRenewFlow implements TransactionalFlow {
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder.setRevisionId(domainHistoryId.getRevisionId());
// Bill for this explicit renew itself.
BillingEvent.OneTime explicitRenewEvent =
BillingEvent explicitRenewEvent =
createRenewBillingEvent(
tldStr, feesAndCredits.getTotalCost(), years, domainHistoryId, allocationToken, now);
// Create a new autorenew billing event and poll message starting at the new expiration time.
BillingEvent.Recurring newAutorenewEvent =
BillingRecurrence newAutorenewEvent =
newAutorenewBillingEvent(existingDomain)
.setEventTime(newExpirationTime)
.setRenewalPrice(existingRecurringBillingEvent.getRenewalPrice().orElse(null))
.setRenewalPriceBehavior(existingRecurringBillingEvent.getRenewalPriceBehavior())
.setRenewalPrice(existingBillingRecurrence.getRenewalPrice().orElse(null))
.setRenewalPriceBehavior(existingBillingRecurrence.getRenewalPriceBehavior())
.setDomainHistoryId(domainHistoryId)
.build();
PollMessage.Autorenew newAutorenewPollMessage =
@@ -240,8 +239,8 @@ public final class DomainRenewFlow implements TransactionalFlow {
.setDomainHistoryId(domainHistoryId)
.build();
// End the old autorenew billing event and poll message now. This may delete the poll message.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
updateAutorenewRecurrenceEndTime(existingDomain, existingRecurring, now, domainHistoryId);
updateAutorenewRecurrenceEndTime(
existingDomain, existingBillingRecurrence, now, domainHistoryId);
Domain newDomain =
existingDomain
.asBuilder()
@@ -338,14 +337,14 @@ public final class DomainRenewFlow implements TransactionalFlow {
}
}
private OneTime createRenewBillingEvent(
private BillingEvent createRenewBillingEvent(
String tld,
Money renewCost,
int years,
HistoryEntryId domainHistoryId,
Optional<AllocationToken> allocationToken,
DateTime now) {
return new BillingEvent.OneTime.Builder()
return new BillingEvent.Builder()
.setReason(Reason.RENEW)
.setTargetId(targetId)
.setRegistrarId(registrarId)
@@ -14,6 +14,7 @@
package google.registry.flows.domain;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -34,7 +35,6 @@ import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.net.InternetDomainName;
import google.registry.dns.DnsUtils;
import google.registry.flows.EppException;
import google.registry.flows.EppException.CommandUseErrorException;
import google.registry.flows.EppException.StatusProhibitsOperationException;
@@ -45,9 +45,9 @@ import google.registry.flows.FlowModule.TargetId;
import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Update;
import google.registry.model.domain.DomainHistory;
@@ -122,7 +122,6 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
@Inject @TargetId String targetId;
@Inject @Superuser boolean isSuperuser;
@Inject DomainHistory.Builder historyBuilder;
@Inject DnsUtils dnsUtils;
@Inject EppResponse.Builder responseBuilder;
@Inject DomainPricingLogic pricingLogic;
@Inject DomainRestoreRequestFlow() {}
@@ -161,7 +160,7 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
entitiesToSave.add(
createRestoreBillingEvent(domainHistoryId, feesAndCredits.getRestoreCost(), now));
BillingEvent.Recurring autorenewEvent =
BillingRecurrence autorenewEvent =
newAutorenewBillingEvent(existingDomain)
.setEventTime(newExpirationTime)
.setRecurrenceEndTime(END_OF_TIME)
@@ -185,7 +184,7 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
entitiesToSave.add(newDomain, domainHistory, autorenewEvent, autorenewPollMessage);
tm().putAll(entitiesToSave.build());
tm().delete(existingDomain.getDeletePollMessage());
dnsUtils.requestDomainDnsRefresh(existingDomain.getDomainName());
requestDomainDnsRefresh(existingDomain.getDomainName());
return responseBuilder
.setExtensions(createResponseExtensions(feesAndCredits, feeUpdate, isExpired))
.build();
@@ -231,7 +230,7 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
private static Domain performRestore(
Domain existingDomain,
DateTime newExpirationTime,
BillingEvent.Recurring autorenewEvent,
BillingRecurrence autorenewEvent,
PollMessage.Autorenew autorenewPollMessage,
DateTime now,
String registrarId) {
@@ -252,19 +251,19 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
.build();
}
private OneTime createRenewBillingEvent(
private BillingEvent createRenewBillingEvent(
HistoryEntryId domainHistoryId, Money renewCost, DateTime now) {
return prepareBillingEvent(domainHistoryId, renewCost, now).setReason(Reason.RENEW).build();
}
private BillingEvent.OneTime createRestoreBillingEvent(
private BillingEvent createRestoreBillingEvent(
HistoryEntryId domainHistoryId, Money restoreCost, DateTime now) {
return prepareBillingEvent(domainHistoryId, restoreCost, now).setReason(Reason.RESTORE).build();
}
private OneTime.Builder prepareBillingEvent(
private BillingEvent.Builder prepareBillingEvent(
HistoryEntryId domainHistoryId, Money cost, DateTime now) {
return new BillingEvent.OneTime.Builder()
return new BillingEvent.Builder()
.setTargetId(targetId)
.setRegistrarId(registrarId)
.setEventTime(now)
@@ -305,14 +304,14 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
/** Restore command cannot have other changes specified. */
static class RestoreCommandIncludesChangesException extends CommandUseErrorException {
public RestoreCommandIncludesChangesException() {
RestoreCommandIncludesChangesException() {
super("Restore command cannot have other changes specified");
}
}
/** Domain is not eligible for restore. */
static class DomainNotEligibleForRestoreException extends StatusProhibitsOperationException {
public DomainNotEligibleForRestoreException() {
DomainNotEligibleForRestoreException() {
super("Domain is not eligible for restore");
}
}
@@ -45,11 +45,12 @@ import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.flows.domain.token.AllocationTokenFlowUtils;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingBase.Flag;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.billing.BillingCancellation;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.GracePeriod;
@@ -149,16 +150,18 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
String gainingRegistrarId = transferData.getGainingRegistrarId();
// Create a transfer billing event for 1 year, unless the superuser extension was used to set
// the transfer period to zero. There is not a transfer cost if the transfer period is zero.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
BillingRecurrence existingBillingRecurrence =
tm().loadByKey(existingDomain.getAutorenewBillingEvent());
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder.setRevisionId(domainHistoryId.getRevisionId());
boolean hasPackageToken = existingDomain.getCurrentPackageToken().isPresent();
Money renewalPrice = hasPackageToken ? null : existingRecurring.getRenewalPrice().orElse(null);
Optional<BillingEvent.OneTime> billingEvent =
Money renewalPrice =
hasPackageToken ? null : existingBillingRecurrence.getRenewalPrice().orElse(null);
Optional<BillingEvent> billingEvent =
transferData.getTransferPeriod().getValue() == 0
? Optional.empty()
: Optional.of(
new BillingEvent.OneTime.Builder()
new BillingEvent.Builder()
.setReason(Reason.TRANSFER)
.setTargetId(targetId)
.setRegistrarId(gainingRegistrarId)
@@ -170,9 +173,9 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
targetId,
transferData.getTransferRequestTime(),
// When removing a domain from a package it should return to the
// default recurring billing behavior so the existing recurring
// default recurrence billing behavior so the existing recurrence
// billing event should not be passed in.
hasPackageToken ? null : existingRecurring)
hasPackageToken ? null : existingBillingRecurrence)
.getRenewCost())
.setEventTime(now)
.setBillingTime(now.plus(Tld.get(tldStr).getTransferGracePeriodLength()))
@@ -192,18 +195,18 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
// still needs to be charged for the auto-renew.
if (billingEvent.isPresent()) {
entitiesToSave.add(
BillingEvent.Cancellation.forGracePeriod(
autorenewGrace, now, domainHistoryId, targetId));
BillingCancellation.forGracePeriod(autorenewGrace, now, domainHistoryId, targetId));
}
}
// Close the old autorenew event and poll message at the transfer time (aka now). This may end
// up deleting the poll message.
updateAutorenewRecurrenceEndTime(existingDomain, existingRecurring, now, domainHistoryId);
updateAutorenewRecurrenceEndTime(
existingDomain, existingBillingRecurrence, now, domainHistoryId);
DateTime newExpirationTime =
computeExDateForApprovalTime(existingDomain, now, transferData.getTransferPeriod());
// Create a new autorenew event starting at the expiration time.
BillingEvent.Recurring autorenewEvent =
new BillingEvent.Recurring.Builder()
BillingRecurrence autorenewEvent =
new BillingRecurrence.Builder()
.setReason(Reason.RENEW)
.setFlags(ImmutableSet.of(Flag.AUTO_RENEW))
.setTargetId(targetId)
@@ -212,7 +215,7 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
.setRenewalPriceBehavior(
hasPackageToken
? RenewalPriceBehavior.DEFAULT
: existingRecurring.getRenewalPriceBehavior())
: existingBillingRecurrence.getRenewalPriceBehavior())
.setRenewalPrice(renewalPrice)
.setRecurrenceEndTime(END_OF_TIME)
.setDomainHistoryId(domainHistoryId)
@@ -248,12 +251,10 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
.setGracePeriods(
billingEvent
.map(
oneTime ->
event ->
ImmutableSet.of(
GracePeriod.forBillingEvent(
GracePeriodStatus.TRANSFER,
existingDomain.getRepoId(),
oneTime)))
GracePeriodStatus.TRANSFER, existingDomain.getRepoId(), event)))
.orElseGet(ImmutableSet::of))
.setLastEppUpdateTime(now)
.setLastEppUpdateRegistrarId(registrarId)
@@ -39,7 +39,7 @@ import google.registry.flows.FlowModule.Superuser;
import google.registry.flows.FlowModule.TargetId;
import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.metadata.MetadataExtension;
@@ -117,9 +117,10 @@ public final class DomainTransferCancelFlow implements TransactionalFlow {
targetId, newDomain.getTransferData(), null, domainHistoryId));
// Reopen the autorenew event and poll message that we closed for the implicit transfer. This
// may recreate the autorenew poll message if it was deleted when the transfer request was made.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
BillingRecurrence existingBillingRecurrence =
tm().loadByKey(existingDomain.getAutorenewBillingEvent());
updateAutorenewRecurrenceEndTime(
existingDomain, existingRecurring, END_OF_TIME, domainHistory.getHistoryEntryId());
existingDomain, existingBillingRecurrence, END_OF_TIME, domainHistory.getHistoryEntryId());
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
tm().delete(existingDomain.getTransferData().getServerApproveEntities());
@@ -41,7 +41,7 @@ import google.registry.flows.FlowModule.Superuser;
import google.registry.flows.FlowModule.TargetId;
import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.metadata.MetadataExtension;
@@ -116,9 +116,10 @@ public final class DomainTransferRejectFlow implements TransactionalFlow {
targetId, newDomain.getTransferData(), null, now, domainHistoryId));
// Reopen the autorenew event and poll message that we closed for the implicit transfer. This
// may end up recreating the poll message if it was deleted upon the transfer request.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
BillingRecurrence existingBillingRecurrence =
tm().loadByKey(existingDomain.getAutorenewBillingEvent());
updateAutorenewRecurrenceEndTime(
existingDomain, existingRecurring, END_OF_TIME, domainHistory.getHistoryEntryId());
existingDomain, existingBillingRecurrence, END_OF_TIME, domainHistory.getHistoryEntryId());
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
tm().delete(existingDomain.getTransferData().getServerApproveEntities());
@@ -52,7 +52,7 @@ import google.registry.flows.exceptions.InvalidTransferPeriodValueException;
import google.registry.flows.exceptions.ObjectAlreadySponsoredException;
import google.registry.flows.exceptions.TransferPeriodMustBeOneYearException;
import google.registry.flows.exceptions.TransferPeriodZeroAndFeeTransferExtensionException;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Transfer;
import google.registry.model.domain.DomainHistory;
@@ -195,13 +195,14 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
throw new TransferPeriodZeroAndFeeTransferExtensionException();
}
// If the period is zero, then there is no fee for the transfer.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
BillingRecurrence existingBillingRecurrence =
tm().loadByKey(existingDomain.getAutorenewBillingEvent());
Optional<FeesAndCredits> feesAndCredits;
if (period.getValue() == 0) {
feesAndCredits = Optional.empty();
} else if (!existingDomain.getCurrentPackageToken().isPresent()) {
feesAndCredits =
Optional.of(pricingLogic.getTransferPrice(tld, targetId, now, existingRecurring));
Optional.of(pricingLogic.getTransferPrice(tld, targetId, now, existingBillingRecurrence));
} else {
// If existing domain is in a package, calculate the transfer price with default renewal price
// behavior
@@ -243,7 +244,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
serverApproveNewExpirationTime,
domainHistoryId,
existingDomain,
existingRecurring,
existingBillingRecurrence,
trid,
gainingClientId,
feesAndCredits.map(FeesAndCredits::getTotalCost),
@@ -274,7 +275,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
// cloneProjectedAtTime() will replace these old autorenew entities with the server approve ones
// that we've created in this flow and stored in pendingTransferData.
updateAutorenewRecurrenceEndTime(
existingDomain, existingRecurring, automaticTransferTime, domainHistoryId);
existingDomain, existingBillingRecurrence, automaticTransferTime, domainHistoryId);
Domain newDomain =
existingDomain
.asBuilder()
@@ -20,11 +20,12 @@ import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import google.registry.model.billing.BillingBase.Flag;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.billing.BillingCancellation;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.Period;
@@ -66,8 +67,8 @@ public final class DomainTransferUtils {
// Unless superuser sets period to 0, add a transfer billing event.
transferDataBuilder.setServerApproveBillingEvent(
serverApproveEntities.stream()
.filter(BillingEvent.OneTime.class::isInstance)
.map(BillingEvent.OneTime.class::cast)
.filter(BillingEvent.class::isInstance)
.map(BillingEvent.class::cast)
.collect(onlyElement())
.createVKey());
}
@@ -75,8 +76,8 @@ public final class DomainTransferUtils {
.setTransferStatus(TransferStatus.PENDING)
.setServerApproveAutorenewEvent(
serverApproveEntities.stream()
.filter(BillingEvent.Recurring.class::isInstance)
.map(BillingEvent.Recurring.class::cast)
.filter(BillingRecurrence.class::isInstance)
.map(BillingRecurrence.class::cast)
.collect(onlyElement())
.createVKey())
.setServerApproveAutorenewPollMessage(
@@ -110,7 +111,7 @@ public final class DomainTransferUtils {
DateTime serverApproveNewExpirationTime,
HistoryEntryId domainHistoryId,
Domain existingDomain,
Recurring existingRecurring,
BillingRecurrence existingBillingRecurrence,
Trid trid,
String gainingRegistrarId,
Optional<Money> transferCost,
@@ -146,12 +147,12 @@ public final class DomainTransferUtils {
.add(
createGainingClientAutorenewEvent(
existingDomain.getCurrentPackageToken().isPresent()
? existingRecurring
? existingBillingRecurrence
.asBuilder()
.setRenewalPriceBehavior(RenewalPriceBehavior.DEFAULT)
.setRenewalPrice(null)
.build()
: existingRecurring,
: existingBillingRecurrence,
serverApproveNewExpirationTime,
domainHistoryId,
targetId,
@@ -246,21 +247,21 @@ public final class DomainTransferUtils {
.build();
}
private static BillingEvent.Recurring createGainingClientAutorenewEvent(
Recurring existingRecurring,
private static BillingRecurrence createGainingClientAutorenewEvent(
BillingRecurrence existingBillingRecurrence,
DateTime serverApproveNewExpirationTime,
HistoryEntryId domainHistoryId,
String targetId,
String gainingRegistrarId) {
return new BillingEvent.Recurring.Builder()
return new BillingRecurrence.Builder()
.setReason(Reason.RENEW)
.setFlags(ImmutableSet.of(Flag.AUTO_RENEW))
.setTargetId(targetId)
.setRegistrarId(gainingRegistrarId)
.setEventTime(serverApproveNewExpirationTime)
.setRecurrenceEndTime(END_OF_TIME)
.setRenewalPriceBehavior(existingRecurring.getRenewalPriceBehavior())
.setRenewalPrice(existingRecurring.getRenewalPrice().orElse(null))
.setRenewalPriceBehavior(existingBillingRecurrence.getRenewalPriceBehavior())
.setRenewalPrice(existingBillingRecurrence.getRenewalPrice().orElse(null))
.setDomainHistoryId(domainHistoryId)
.build();
}
@@ -281,7 +282,7 @@ public final class DomainTransferUtils {
* <p>For details on the policy justification, see b/19430703#comment17 and <a
* href="https://www.icann.org/news/advisory-2002-06-06-en">this ICANN advisory</a>.
*/
private static Optional<BillingEvent.Cancellation> createOptionalAutorenewCancellation(
private static Optional<BillingCancellation> createOptionalAutorenewCancellation(
DateTime automaticTransferTime,
DateTime now,
HistoryEntryId domainHistoryId,
@@ -294,8 +295,7 @@ public final class DomainTransferUtils {
domainAtTransferTime.getGracePeriodsOfType(GracePeriodStatus.AUTO_RENEW), null);
if (autorenewGracePeriod != null && transferCost.isPresent()) {
return Optional.of(
BillingEvent.Cancellation.forGracePeriod(
autorenewGracePeriod, now, domainHistoryId, targetId)
BillingCancellation.forGracePeriod(autorenewGracePeriod, now, domainHistoryId, targetId)
.asBuilder()
.setEventTime(automaticTransferTime)
.build());
@@ -303,14 +303,14 @@ public final class DomainTransferUtils {
return Optional.empty();
}
private static BillingEvent.OneTime createTransferBillingEvent(
private static BillingEvent createTransferBillingEvent(
DateTime automaticTransferTime,
HistoryEntryId domainHistoryId,
String targetId,
String gainingRegistrarId,
Tld registry,
Money transferCost) {
return new BillingEvent.OneTime.Builder()
return new BillingEvent.Builder()
.setReason(Reason.TRANSFER)
.setTargetId(targetId)
.setRegistrarId(gainingRegistrarId)
@@ -19,6 +19,7 @@ import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.collect.ImmutableSortedSet.toImmutableSortedSet;
import static com.google.common.collect.Sets.symmetricDifference;
import static com.google.common.collect.Sets.union;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.flows.FlowUtils.persistEntityChanges;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.checkSameValuesNotAddedAndRemoved;
@@ -49,7 +50,6 @@ import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Ordering;
import com.google.common.collect.Sets;
import com.google.common.net.InternetDomainName;
import google.registry.dns.DnsUtils;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -64,8 +64,8 @@ import google.registry.flows.custom.EntityChanges;
import google.registry.flows.domain.DomainFlowUtils.MissingRegistrantException;
import google.registry.flows.domain.DomainFlowUtils.NameserversNotSpecifiedForTldWithNameserverAllowListException;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.domain.DesignatedContact;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Update;
@@ -156,7 +156,6 @@ public final class DomainUpdateFlow implements TransactionalFlow {
@Inject @Superuser boolean isSuperuser;
@Inject Trid trid;
@Inject DomainHistory.Builder historyBuilder;
@Inject DnsUtils dnsUtils;
@Inject EppResponse.Builder responseBuilder;
@Inject DomainUpdateFlowCustomLogic flowCustomLogic;
@Inject DomainPricingLogic pricingLogic;
@@ -183,11 +182,11 @@ public final class DomainUpdateFlow implements TransactionalFlow {
historyBuilder.setType(DOMAIN_UPDATE).setDomain(newDomain).build();
validateNewState(newDomain);
if (requiresDnsUpdate(existingDomain, newDomain)) {
dnsUtils.requestDomainDnsRefresh(targetId);
requestDomainDnsRefresh(targetId);
}
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
entitiesToSave.add(newDomain, domainHistory);
Optional<BillingEvent.OneTime> statusUpdateBillingEvent =
Optional<BillingEvent> statusUpdateBillingEvent =
createBillingEventForStatusUpdates(existingDomain, newDomain, domainHistory, now);
statusUpdateBillingEvent.ifPresent(entitiesToSave::add);
Optional<PollMessage.OneTime> serverStatusUpdatePollMessage =
@@ -207,7 +206,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
}
/** Determines if any of the changes to new domain should trigger DNS update. */
private boolean requiresDnsUpdate(Domain existingDomain, Domain newDomain) {
private static boolean requiresDnsUpdate(Domain existingDomain, Domain newDomain) {
return existingDomain.shouldPublishToDns() != newDomain.shouldPublishToDns()
|| !Objects.equals(newDomain.getDsData(), existingDomain.getDsData())
|| !Objects.equals(newDomain.getNsHosts(), existingDomain.getNsHosts());
@@ -256,7 +255,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
// We have to verify no duplicate contacts _before_ constructing the domain because it is
// illegal to construct a domain with duplicate contacts.
Sets.SetView<DesignatedContact> newContacts =
Sets.union(Sets.difference(domain.getContacts(), remove.getContacts()), add.getContacts());
union(Sets.difference(domain.getContacts(), remove.getContacts()), add.getContacts());
validateNoDuplicateContacts(newContacts);
Domain.Builder domainBuilder =
@@ -301,7 +300,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
return domainBuilder.build();
}
private void validateRegistrantIsntBeingRemoved(Change change) throws EppException {
private static void validateRegistrantIsntBeingRemoved(Change change) throws EppException {
if (change.getRegistrantContactId() != null && change.getRegistrantContactId().isEmpty()) {
throw new MissingRegistrantException();
}
@@ -314,7 +313,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
* compliant with the additions or amendments, otherwise existing data can become invalid and
* cause Domain update failure.
*/
private void validateNewState(Domain newDomain) throws EppException {
private static void validateNewState(Domain newDomain) throws EppException {
validateRequiredContactsPresent(newDomain.getRegistrant(), newDomain.getContacts());
validateDsData(newDomain.getDsData());
validateNameserversCountForTld(
@@ -324,7 +323,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
}
/** Some status updates cost money. Bill only once no matter how many of them are changed. */
private Optional<BillingEvent.OneTime> createBillingEventForStatusUpdates(
private Optional<BillingEvent> createBillingEventForStatusUpdates(
Domain existingDomain, Domain newDomain, DomainHistory historyEntry, DateTime now) {
Optional<MetadataExtension> metadataExtension =
eppInput.getSingleExtension(MetadataExtension.class);
@@ -334,7 +333,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
if (statusValue.isChargedStatus()) {
// Only charge once.
return Optional.of(
new BillingEvent.OneTime.Builder()
new BillingEvent.Builder()
.setReason(Reason.SERVER_STATUS)
.setTargetId(targetId)
.setRegistrarId(registrarId)
@@ -368,17 +367,17 @@ public final class DomainUpdateFlow implements TransactionalFlow {
.collect(toImmutableSortedSet(Ordering.natural()));
String msg;
if (addedServerStatuses.size() > 0 && removedServerStatuses.size() > 0) {
if (!addedServerStatuses.isEmpty() && !removedServerStatuses.isEmpty()) {
msg =
String.format(
"The registry administrator has added the status(es) %s and removed the status(es)"
+ " %s.",
addedServerStatuses, removedServerStatuses);
} else if (addedServerStatuses.size() > 0) {
} else if (!addedServerStatuses.isEmpty()) {
msg =
String.format(
"The registry administrator has added the status(es) %s.", addedServerStatuses);
} else if (removedServerStatuses.size() > 0) {
} else if (!removedServerStatuses.isEmpty()) {
msg =
String.format(
"The registry administrator has removed the status(es) %s.", removedServerStatuses);
@@ -16,6 +16,7 @@ package google.registry.flows.domain.token;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.pricing.PricingEngineProxy.isDomainPremium;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableList;
@@ -26,8 +27,9 @@ import google.registry.flows.EppException;
import google.registry.flows.EppException.AssociationProhibitsOperationException;
import google.registry.flows.EppException.AuthorizationErrorException;
import google.registry.flows.EppException.StatusProhibitsOperationException;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.flows.domain.DomainPricingLogic.AllocationTokenInvalidForPremiumNameException;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand;
import google.registry.model.domain.fee.FeeQueryCommandExtensionItem.CommandName;
@@ -79,7 +81,13 @@ public class AllocationTokenFlowUtils {
ImmutableMap.Builder<InternetDomainName, String> resultsBuilder = new ImmutableMap.Builder<>();
for (InternetDomainName domainName : domainNames) {
try {
validateToken(domainName, tokenEntity, CommandName.CREATE, registrarId, now);
validateToken(
domainName,
tokenEntity,
CommandName.CREATE,
registrarId,
isDomainPremium(domainName.toString(), now),
now);
validDomainNames.add(domainName);
} catch (EppException e) {
resultsBuilder.put(domainName, e.getMessage());
@@ -105,7 +113,7 @@ public class AllocationTokenFlowUtils {
/**
* Validates a given token. The token could be invalid if it has allowed client IDs or TLDs that
* do not include this client ID / TLD, or if the token has a promotion that is not currently
* running.
* running, or the token is not valid for a premium name when necessary.
*
* @throws EppException if the token is invalid in any way
*/
@@ -114,33 +122,48 @@ public class AllocationTokenFlowUtils {
AllocationToken token,
CommandName commandName,
String registrarId,
boolean isPremium,
DateTime now)
throws EppException {
// Only tokens with default behavior require validation
if (TokenBehavior.DEFAULT.equals(token.getTokenBehavior())) {
if (!token.getAllowedEppActions().isEmpty()
&& !token.getAllowedEppActions().contains(commandName)) {
throw new AllocationTokenNotValidForCommandException();
}
if (!token.getAllowedRegistrarIds().isEmpty()
&& !token.getAllowedRegistrarIds().contains(registrarId)) {
throw new AllocationTokenNotValidForRegistrarException();
}
if (!token.getAllowedTlds().isEmpty()
&& !token.getAllowedTlds().contains(domainName.parent().toString())) {
throw new AllocationTokenNotValidForTldException();
}
if (token.getDomainName().isPresent()
&& !token.getDomainName().get().equals(domainName.toString())) {
throw new AllocationTokenNotValidForDomainException();
}
// Tokens without status transitions will just have a single-entry NOT_STARTED map, so only
// check the status transitions map if it's non-trivial.
if (token.getTokenStatusTransitions().size() > 1
&& !TokenStatus.VALID.equals(token.getTokenStatusTransitions().getValueAtTime(now))) {
throw new AllocationTokenNotInPromotionException();
}
if (!TokenBehavior.DEFAULT.equals(token.getTokenBehavior())) {
return;
}
validateTokenForPossiblePremiumName(Optional.of(token), isPremium);
if (!token.getAllowedEppActions().isEmpty()
&& !token.getAllowedEppActions().contains(commandName)) {
throw new AllocationTokenNotValidForCommandException();
}
if (!token.getAllowedRegistrarIds().isEmpty()
&& !token.getAllowedRegistrarIds().contains(registrarId)) {
throw new AllocationTokenNotValidForRegistrarException();
}
if (!token.getAllowedTlds().isEmpty()
&& !token.getAllowedTlds().contains(domainName.parent().toString())) {
throw new AllocationTokenNotValidForTldException();
}
if (token.getDomainName().isPresent()
&& !token.getDomainName().get().equals(domainName.toString())) {
throw new AllocationTokenNotValidForDomainException();
}
// Tokens without status transitions will just have a single-entry NOT_STARTED map, so only
// check the status transitions map if it's non-trivial.
if (token.getTokenStatusTransitions().size() > 1
&& !TokenStatus.VALID.equals(token.getTokenStatusTransitions().getValueAtTime(now))) {
throw new AllocationTokenNotInPromotionException();
}
}
/** Validates that the given token is valid for a premium name if the name is premium. */
public static void validateTokenForPossiblePremiumName(
Optional<AllocationToken> token, boolean isPremium)
throws AllocationTokenInvalidForPremiumNameException {
if (token.isPresent()
&& token.get().getDiscountFraction() != 0.0
&& isPremium
&& !token.get().shouldDiscountPremiums()) {
throw new AllocationTokenInvalidForPremiumNameException();
}
}
@@ -187,6 +210,7 @@ public class AllocationTokenFlowUtils {
tokenEntity,
CommandName.CREATE,
registrarId,
isDomainPremium(command.getDomainName(), now),
now);
return Optional.of(tokenCustomLogic.validateToken(command, tokenEntity, tld, registrarId, now));
}
@@ -209,6 +233,7 @@ public class AllocationTokenFlowUtils {
tokenEntity,
commandName,
registrarId,
isDomainPremium(existingDomain.getDomainName(), now),
now);
return Optional.of(
tokenCustomLogic.validateToken(existingDomain, tokenEntity, tld, registrarId, now));
@@ -236,16 +261,16 @@ public class AllocationTokenFlowUtils {
return domain;
}
Recurring newRecurringBillingEvent =
BillingRecurrence newBillingRecurrence =
tm().loadByKey(domain.getAutorenewBillingEvent())
.asBuilder()
.setRenewalPriceBehavior(RenewalPriceBehavior.DEFAULT)
.setRenewalPrice(null)
.build();
// the Recurring billing event is reloaded later in the renew flow, so we synchronize changed
// RecurringBillingEvent with storage manually
tm().put(newRecurringBillingEvent);
// the Recurrence is reloaded later in the renew flow, so we synchronize changed
// Recurrences with storage manually
tm().put(newBillingRecurrence);
tm().getEntityManager().flush();
tm().getEntityManager().clear();
@@ -253,7 +278,7 @@ public class AllocationTokenFlowUtils {
return domain
.asBuilder()
.setCurrentPackageToken(null)
.setAutorenewBillingEvent(newRecurringBillingEvent.createVKey())
.setAutorenewBillingEvent(newBillingRecurrence.createVKey())
.build();
}
@@ -14,6 +14,7 @@
package google.registry.flows.host;
import static google.registry.dns.DnsUtils.requestHostDnsRefresh;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.verifyResourceDoesNotExist;
import static google.registry.flows.host.HostFlowUtils.lookupSuperordinateDomain;
@@ -28,7 +29,6 @@ import static google.registry.util.CollectionUtils.isNullOrEmpty;
import com.google.common.collect.ImmutableSet;
import google.registry.config.RegistryConfig.Config;
import google.registry.dns.DnsUtils;
import google.registry.flows.EppException;
import google.registry.flows.EppException.ParameterValueRangeErrorException;
import google.registry.flows.EppException.RequiredParameterMissingException;
@@ -85,7 +85,6 @@ public final class HostCreateFlow implements TransactionalFlow {
@Inject @RegistrarId String registrarId;
@Inject @TargetId String targetId;
@Inject HostHistory.Builder historyBuilder;
@Inject DnsUtils dnsUtils;
@Inject EppResponse.Builder responseBuilder;
@Inject
@@ -138,7 +137,7 @@ public final class HostCreateFlow implements TransactionalFlow {
.build());
// Only update DNS if this is a subordinate host. External hosts have no glue to write, so
// they are only written as NS records from the referencing domain.
dnsUtils.requestHostDnsRefresh(targetId);
requestHostDnsRefresh(targetId);
}
tm().insertAll(entitiesToSave);
return responseBuilder.setResData(HostCreateData.create(targetId, now)).build();
@@ -14,6 +14,7 @@
package google.registry.flows.host;
import static google.registry.dns.DnsUtils.requestHostDnsRefresh;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.checkLinkedDomains;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -24,7 +25,6 @@ import static google.registry.model.eppoutput.Result.Code.SUCCESS;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import google.registry.dns.DnsUtils;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -71,7 +71,6 @@ public final class HostDeleteFlow implements TransactionalFlow {
StatusValue.PENDING_DELETE,
StatusValue.SERVER_DELETE_PROHIBITED);
@Inject DnsUtils dnsUtils;
@Inject ExtensionManager extensionManager;
@Inject @RegistrarId String registrarId;
@Inject @TargetId String targetId;
@@ -104,7 +103,7 @@ public final class HostDeleteFlow implements TransactionalFlow {
}
Host newHost = existingHost.asBuilder().setStatusValues(null).setDeletionTime(now).build();
if (existingHost.isSubordinate()) {
dnsUtils.requestHostDnsRefresh(existingHost.getHostName());
requestHostDnsRefresh(existingHost.getHostName());
tm().update(
tm().loadByKey(existingHost.getSuperordinateDomain())
.asBuilder()
@@ -16,6 +16,7 @@ package google.registry.flows.host;
import static com.google.common.base.MoreObjects.firstNonNull;
import static com.google.common.collect.Sets.union;
import static google.registry.dns.DnsUtils.requestHostDnsRefresh;
import static google.registry.dns.RefreshDnsOnHostRenameAction.PARAM_HOST_KEY;
import static google.registry.dns.RefreshDnsOnHostRenameAction.QUEUE_HOST_RENAME;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
@@ -37,7 +38,6 @@ import com.google.common.collect.ImmutableMultimap;
import com.google.common.collect.ImmutableSet;
import google.registry.batch.AsyncTaskEnqueuer;
import google.registry.batch.CloudTasksUtils;
import google.registry.dns.DnsUtils;
import google.registry.dns.RefreshDnsOnHostRenameAction;
import google.registry.flows.EppException;
import google.registry.flows.EppException.ObjectAlreadyExistsException;
@@ -124,7 +124,6 @@ public final class HostUpdateFlow implements TransactionalFlow {
@Inject @Superuser boolean isSuperuser;
@Inject HostHistory.Builder historyBuilder;
@Inject AsyncTaskEnqueuer asyncTaskEnqueuer;
@Inject DnsUtils dnsUtils;
@Inject EppResponse.Builder responseBuilder;
@Inject CloudTasksUtils cloudTasksUtils;
@@ -241,7 +240,7 @@ public final class HostUpdateFlow implements TransactionalFlow {
verifyNoDisallowedStatuses(existingHost, DISALLOWED_STATUSES);
}
private void verifyHasIpsIffIsExternal(Update command, Host existingHost, Host newHost)
private static void verifyHasIpsIffIsExternal(Update command, Host existingHost, Host newHost)
throws EppException {
boolean wasSubordinate = existingHost.isSubordinate();
boolean willBeSubordinate = newHost.isSubordinate();
@@ -266,14 +265,14 @@ public final class HostUpdateFlow implements TransactionalFlow {
// Only update DNS for subordinate hosts. External hosts have no glue to write, so they
// are only written as NS records from the referencing domain.
if (existingHost.isSubordinate()) {
dnsUtils.requestHostDnsRefresh(existingHost.getHostName());
requestHostDnsRefresh(existingHost.getHostName());
}
// In case of a rename, there are many updates we need to queue up.
if (((Update) resourceCommand).getInnerChange().getHostName() != null) {
// If the renamed host is also subordinate, then we must enqueue an update to write the new
// glue.
if (newHost.isSubordinate()) {
dnsUtils.requestHostDnsRefresh(newHost.getHostName());
requestHostDnsRefresh(newHost.getHostName());
}
// We must also enqueue updates for all domains that use this host as their nameserver so
// that their NS records can be updated to point at the new name.
@@ -317,14 +316,14 @@ public final class HostUpdateFlow implements TransactionalFlow {
/** Host with specified name already exists. */
static class HostAlreadyExistsException extends ObjectAlreadyExistsException {
public HostAlreadyExistsException(String hostName) {
HostAlreadyExistsException(String hostName) {
super(String.format("Object with given ID (%s) already exists", hostName));
}
}
/** Cannot add IP addresses to an external host. */
static class CannotAddIpToExternalHostException extends ParameterValueRangeErrorException {
public CannotAddIpToExternalHostException() {
CannotAddIpToExternalHostException() {
super("Cannot add IP addresses to external hosts");
}
}
@@ -332,14 +331,14 @@ public final class HostUpdateFlow implements TransactionalFlow {
/** Cannot remove all IP addresses from a subordinate host. */
static class CannotRemoveSubordinateHostLastIpException
extends StatusProhibitsOperationException {
public CannotRemoveSubordinateHostLastIpException() {
CannotRemoveSubordinateHostLastIpException() {
super("Cannot remove all IP addresses from a subordinate host");
}
}
/** Cannot rename an external host. */
static class CannotRenameExternalHostException extends StatusProhibitsOperationException {
public CannotRenameExternalHostException() {
CannotRenameExternalHostException() {
super("Cannot rename an external host");
}
}
@@ -31,7 +31,7 @@ import com.google.common.collect.ImmutableMap;
import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import com.google.common.net.MediaType;
import google.registry.config.CredentialModule.DefaultCredential;
import google.registry.config.CredentialModule.ApplicationDefaultCredential;
import google.registry.util.GoogleCredentialsBundle;
import java.io.IOException;
import java.io.InputStream;
@@ -64,7 +64,7 @@ public class GcsUtils implements Serializable {
}
@Inject
public GcsUtils(@DefaultCredential GoogleCredentialsBundle credentialsBundle) {
public GcsUtils(@ApplicationDefaultCredential GoogleCredentialsBundle credentialsBundle) {
this(
StorageOptions.newBuilder()
.setCredentials(credentialsBundle.getGoogleCredentials())
@@ -0,0 +1,264 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.billing;
import static com.google.common.base.Preconditions.checkNotNull;
import static google.registry.util.CollectionUtils.forceEmptyToNull;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableCopy;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.google.common.collect.ImmutableSet;
import google.registry.model.Buildable;
import google.registry.model.ImmutableObject;
import google.registry.model.UnsafeSerializable;
import google.registry.model.annotations.IdAllocation;
import google.registry.model.domain.DomainHistory;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
import google.registry.persistence.VKey;
import java.util.Set;
import javax.annotation.Nullable;
import javax.persistence.Column;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.Id;
import javax.persistence.MappedSuperclass;
import org.joda.time.DateTime;
/** A billable event in a domain's lifecycle. */
@MappedSuperclass
public abstract class BillingBase extends ImmutableObject
implements Buildable, TransferServerApproveEntity, UnsafeSerializable {
/** The reason for the bill, which maps 1:1 to skus in go/registry-billing-skus. */
public enum Reason {
CREATE(true),
@Deprecated // DO NOT USE THIS REASON. IT REMAINS BECAUSE OF HISTORICAL DATA. SEE b/31676071.
ERROR(false),
FEE_EARLY_ACCESS(true),
RENEW(true),
RESTORE(true),
SERVER_STATUS(false),
TRANSFER(true);
private final boolean requiresPeriod;
Reason(boolean requiresPeriod) {
this.requiresPeriod = requiresPeriod;
}
/**
* Returns whether billing events with this reason have a period years associated with them.
*
* <p>Note that this is an "if an only if" condition.
*/
public boolean hasPeriodYears() {
return requiresPeriod;
}
}
/** Set of flags that can be applied to billing events. */
public enum Flag {
ALLOCATION,
ANCHOR_TENANT,
AUTO_RENEW,
/** Landrush billing events are historical only and are no longer created. */
LANDRUSH,
/**
* This flag is used on create {@link BillingEvent} billing events for domains that were
* reserved.
*
* <p>This can happen when allocation tokens are used or superusers override a domain
* reservation. These cases can need special handling in billing/invoicing. Anchor tenants will
* never have this flag applied; they will have ANCHOR_TENANT instead.
*/
RESERVED,
SUNRISE,
/**
* This flag will be added to any {@link BillingEvent} events that are created via, e.g., an
* automated process to expand {@link BillingRecurrence} events.
*/
SYNTHETIC
}
/**
* Sets of renewal price behaviors that can be applied to billing recurrences.
*
* <p>When a client renews a domain, they could be charged differently, depending on factors such
* as the client type and the domain itself.
*/
public enum RenewalPriceBehavior {
/**
* This indicates the renewal price is the default price.
*
* <p>By default, if the domain is premium, then premium price will be used. Otherwise, the
* standard price of the TLD will be used.
*/
DEFAULT,
/**
* This indicates the domain will be renewed at standard price even if it's a premium domain.
*
* <p>We chose to name this "NONPREMIUM" rather than simply "STANDARD" to avoid confusion
* between "STANDARD" and "DEFAULT".
*
* <p>This price behavior is used with anchor tenants.
*/
NONPREMIUM,
/**
* This indicates that the renewalPrice in {@link BillingRecurrence} will be used for domain
* renewal.
*
* <p>The renewalPrice has a non-null value iff the price behavior is set to "SPECIFIED". This
* behavior is used with internal registrations.
*/
SPECIFIED
}
/** Entity id. */
@IdAllocation @Id Long id;
/** The registrar to bill. */
@Column(name = "registrarId", nullable = false)
String clientId;
/** Revision id of the entry in DomainHistory table that ths bill belongs to. */
@Column(nullable = false)
Long domainHistoryRevisionId;
/** ID of the EPP resource that the bill is for. */
@Column(nullable = false)
String domainRepoId;
/** When this event was created. For recurrence events, this is also the recurrence start time. */
@Column(nullable = false)
DateTime eventTime;
/** The reason for the bill. */
@Enumerated(EnumType.STRING)
@Column(nullable = false)
Reason reason;
/** The fully qualified domain name of the domain that the bill is for. */
@Column(name = "domain_name", nullable = false)
String targetId;
@Nullable Set<Flag> flags;
public String getRegistrarId() {
return clientId;
}
public long getDomainHistoryRevisionId() {
return domainHistoryRevisionId;
}
public String getDomainRepoId() {
return domainRepoId;
}
public DateTime getEventTime() {
return eventTime;
}
public long getId() {
return id;
}
public Reason getReason() {
return reason;
}
public String getTargetId() {
return targetId;
}
public HistoryEntryId getHistoryEntryId() {
return new HistoryEntryId(domainRepoId, domainHistoryRevisionId);
}
public ImmutableSet<Flag> getFlags() {
return nullToEmptyImmutableCopy(flags);
}
@Override
public abstract VKey<? extends BillingBase> createVKey();
/** Override Buildable.asBuilder() to give this method stronger typing. */
@Override
public abstract Builder<?, ?> asBuilder();
/** An abstract builder for {@link BillingBase}. */
public abstract static class Builder<T extends BillingBase, B extends Builder<?, ?>>
extends GenericBuilder<T, B> {
protected Builder() {}
protected Builder(T instance) {
super(instance);
}
public B setReason(Reason reason) {
getInstance().reason = reason;
return thisCastToDerived();
}
public B setId(long id) {
getInstance().id = id;
return thisCastToDerived();
}
public B setRegistrarId(String registrarId) {
getInstance().clientId = registrarId;
return thisCastToDerived();
}
public B setEventTime(DateTime eventTime) {
getInstance().eventTime = eventTime;
return thisCastToDerived();
}
public B setTargetId(String targetId) {
getInstance().targetId = targetId;
return thisCastToDerived();
}
public B setFlags(ImmutableSet<Flag> flags) {
getInstance().flags = forceEmptyToNull(checkArgumentNotNull(flags, "flags"));
return thisCastToDerived();
}
public B setDomainHistoryId(HistoryEntryId domainHistoryId) {
getInstance().domainHistoryRevisionId = domainHistoryId.getRevisionId();
getInstance().domainRepoId = domainHistoryId.getRepoId();
return thisCastToDerived();
}
public B setDomainHistory(DomainHistory domainHistory) {
return setDomainHistoryId(domainHistory.getHistoryEntryId());
}
@Override
public T build() {
T instance = getInstance();
checkNotNull(instance.reason, "Reason must be set");
checkNotNull(instance.clientId, "Registrar ID must be set");
checkNotNull(instance.eventTime, "Event time must be set");
checkNotNull(instance.targetId, "Target ID must be set");
checkNotNull(instance.domainHistoryRevisionId, "Domain History Revision ID must be set");
checkNotNull(instance.domainRepoId, "Domain Repo ID must be set");
return super.build();
}
}
}
@@ -0,0 +1,166 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.billing;
import static com.google.common.base.MoreObjects.firstNonNull;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import com.google.common.collect.ImmutableMap;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.persistence.VKey;
import google.registry.persistence.WithVKey;
import javax.persistence.AttributeOverride;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.Index;
import javax.persistence.Table;
import org.joda.time.DateTime;
/**
* An event representing a cancellation of one of the other two billable event types.
*
* <p>This is implemented as a separate event rather than a bit on BillingEvent in order to preserve
* the immutability of billing events.
*/
@Entity
@Table(
indexes = {
@Index(columnList = "registrarId"),
@Index(columnList = "eventTime"),
@Index(columnList = "domainRepoId"),
@Index(columnList = "billingTime"),
@Index(columnList = "billing_event_id"),
@Index(columnList = "billing_recurrence_id")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_cancellation_id"))
@WithVKey(Long.class)
public class BillingCancellation extends BillingBase {
/** The billing time of the charge that is being cancelled. */
DateTime billingTime;
/** The one-time billing event to cancel, or null for autorenew cancellations. */
@Column(name = "billing_event_id")
VKey<BillingEvent> billingEvent;
/** The Recurrence to cancel, or null for non-autorenew cancellations. */
@Column(name = "billing_recurrence_id")
VKey<BillingRecurrence> billingRecurrence;
public DateTime getBillingTime() {
return billingTime;
}
public VKey<? extends BillingBase> getEventKey() {
return firstNonNull(billingEvent, billingRecurrence);
}
/** The mapping from billable grace period types to originating billing event reasons. */
static final ImmutableMap<GracePeriodStatus, Reason> GRACE_PERIOD_TO_REASON =
ImmutableMap.of(
GracePeriodStatus.ADD, Reason.CREATE,
GracePeriodStatus.AUTO_RENEW, Reason.RENEW,
GracePeriodStatus.RENEW, Reason.RENEW,
GracePeriodStatus.TRANSFER, Reason.TRANSFER);
/**
* Creates a cancellation billing event (parented on the provided history key, and with the
* corresponding event time) that will cancel out the provided grace period's billing event, using
* the supplied targetId and deriving other metadata (clientId, billing time, and the cancellation
* reason) from the grace period.
*/
public static google.registry.model.billing.BillingCancellation forGracePeriod(
GracePeriod gracePeriod,
DateTime eventTime,
HistoryEntryId domainHistoryId,
String targetId) {
checkArgument(
gracePeriod.hasBillingEvent(),
"Cannot create cancellation for grace period without billing event");
Builder builder =
new Builder()
.setReason(checkNotNull(GRACE_PERIOD_TO_REASON.get(gracePeriod.getType())))
.setTargetId(targetId)
.setRegistrarId(gracePeriod.getRegistrarId())
.setEventTime(eventTime)
// The charge being cancelled will take place at the grace period's expiration time.
.setBillingTime(gracePeriod.getExpirationTime())
.setDomainHistoryId(domainHistoryId);
// Set the grace period's billing event using the appropriate Cancellation builder method.
if (gracePeriod.getBillingEvent() != null) {
builder.setBillingEvent(gracePeriod.getBillingEvent());
} else if (gracePeriod.getBillingRecurrence() != null) {
builder.setBillingRecurrence(gracePeriod.getBillingRecurrence());
}
return builder.build();
}
@Override
public VKey<google.registry.model.billing.BillingCancellation> createVKey() {
return createVKey(getId());
}
public static VKey<google.registry.model.billing.BillingCancellation> createVKey(long id) {
return VKey.create(google.registry.model.billing.BillingCancellation.class, id);
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
}
/**
* A builder for {@link google.registry.model.billing.BillingCancellation} since it is immutable.
*/
public static class Builder
extends BillingBase.Builder<google.registry.model.billing.BillingCancellation, Builder> {
public Builder() {}
private Builder(google.registry.model.billing.BillingCancellation instance) {
super(instance);
}
public Builder setBillingTime(DateTime billingTime) {
getInstance().billingTime = billingTime;
return this;
}
public Builder setBillingEvent(VKey<BillingEvent> billingEvent) {
getInstance().billingEvent = billingEvent;
return this;
}
public Builder setBillingRecurrence(VKey<BillingRecurrence> billingRecurrence) {
getInstance().billingRecurrence = billingRecurrence;
return this;
}
@Override
public google.registry.model.billing.BillingCancellation build() {
google.registry.model.billing.BillingCancellation instance = getInstance();
checkNotNull(instance.billingTime, "Must set billing time");
checkNotNull(instance.reason, "Must set reason");
checkState(
(instance.billingEvent == null) != (instance.billingRecurrence == null),
"Cancellations must have exactly one billing event key set");
return super.build();
}
}
}
@@ -1,4 +1,4 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -14,740 +14,197 @@
package google.registry.model.billing;
import static com.google.common.base.MoreObjects.firstNonNull;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.util.CollectionUtils.forceEmptyToNull;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableCopy;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import google.registry.model.Buildable;
import google.registry.model.ImmutableObject;
import google.registry.model.UnsafeSerializable;
import google.registry.model.annotations.IdAllocation;
import google.registry.model.common.TimeOfYear;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
import google.registry.persistence.VKey;
import google.registry.persistence.WithVKey;
import google.registry.persistence.converter.JodaMoneyType;
import java.util.Optional;
import java.util.Set;
import javax.annotation.Nullable;
import javax.persistence.AttributeOverride;
import javax.persistence.AttributeOverrides;
import javax.persistence.Column;
import javax.persistence.Embedded;
import javax.persistence.Entity;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.Id;
import javax.persistence.Index;
import javax.persistence.MappedSuperclass;
import javax.persistence.Table;
import org.hibernate.annotations.Columns;
import org.hibernate.annotations.Type;
import org.joda.money.Money;
import org.joda.time.DateTime;
/** A billable event in a domain's lifecycle. */
@MappedSuperclass
public abstract class BillingEvent extends ImmutableObject
implements Buildable, TransferServerApproveEntity, UnsafeSerializable {
/** A one-time billable event. */
@Entity
@Table(
indexes = {
@Index(columnList = "registrarId"),
@Index(columnList = "eventTime"),
@Index(columnList = "billingTime"),
@Index(columnList = "syntheticCreationTime"),
@Index(columnList = "domainRepoId"),
@Index(columnList = "allocationToken"),
@Index(columnList = "cancellation_matching_billing_recurrence_id")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_event_id"))
@WithVKey(Long.class)
public class BillingEvent extends BillingBase {
/** The reason for the bill, which maps 1:1 to skus in go/registry-billing-skus. */
public enum Reason {
CREATE(true),
@Deprecated // DO NOT USE THIS REASON. IT REMAINS BECAUSE OF HISTORICAL DATA. SEE b/31676071.
ERROR(false),
FEE_EARLY_ACCESS(true),
RENEW(true),
RESTORE(true),
SERVER_STATUS(false),
TRANSFER(true);
/** The billable value. */
@Type(type = JodaMoneyType.TYPE_NAME)
@Columns(columns = {@Column(name = "cost_amount"), @Column(name = "cost_currency")})
Money cost;
private final boolean requiresPeriod;
Reason(boolean requiresPeriod) {
this.requiresPeriod = requiresPeriod;
}
/**
* Returns whether billing events with this reason have a period years associated with them.
*
* <p>Note that this is an "if an only if" condition.
*/
public boolean hasPeriodYears() {
return requiresPeriod;
}
}
/** Set of flags that can be applied to billing events. */
public enum Flag {
ALLOCATION,
ANCHOR_TENANT,
AUTO_RENEW,
/** Landrush billing events are historical only and are no longer created. */
LANDRUSH,
/**
* This flag is used on create {@link OneTime} billing events for domains that were reserved.
*
* <p>This can happen when allocation tokens are used or superusers override a domain
* reservation. These cases can need special handling in billing/invoicing. Anchor tenants will
* never have this flag applied; they will have ANCHOR_TENANT instead.
*/
RESERVED,
SUNRISE,
/**
* This flag will be added to any {@link OneTime} events that are created via, e.g., an
* automated process to expand {@link Recurring} events.
*/
SYNTHETIC
}
/** When the cost should be billed. */
DateTime billingTime;
/**
* Sets of renewal price behaviors that can be applied to billing recurrences.
*
* <p>When a client renews a domain, they could be charged differently, depending on factors such
* as the client type and the domain itself.
* The period in years of the action being billed for, if applicable, otherwise null. Used for
* financial reporting.
*/
public enum RenewalPriceBehavior {
/**
* This indicates the renewal price is the default price.
*
* <p>By default, if the domain is premium, then premium price will be used. Otherwise, the
* standard price of the TLD will be used.
*/
DEFAULT,
/**
* This indicates the domain will be renewed at standard price even if it's a premium domain.
*
* <p>We chose to name this "NONPREMIUM" rather than simply "STANDARD" to avoid confusion
* between "STANDARD" and "DEFAULT".
*
* <p>This price behavior is used with anchor tenants.
*/
NONPREMIUM,
/**
* This indicates that the renewalPrice in {@link Recurring} will be used for domain renewal.
*
* <p>The renewalPrice has a non-null value iff the price behavior is set to "SPECIFIED". This
* behavior is used with internal registrations.
*/
SPECIFIED
Integer periodYears;
/**
* For {@link Flag#SYNTHETIC} events, when this event was persisted to the database (i.e. the
* cursor position at the time the recurrence expansion job was last run). In the event a job
* needs to be undone, a query on this field will return the complete set of potentially bad
* events.
*/
DateTime syntheticCreationTime;
/**
* For {@link Flag#SYNTHETIC} events, a {@link VKey} to the {@link BillingRecurrence} from which
* this {@link google.registry.model.billing.BillingEvent} was created. This is needed in order to
* properly match billing events against {@link BillingCancellation}s.
*/
@Column(name = "cancellation_matching_billing_recurrence_id")
VKey<BillingRecurrence> cancellationMatchingBillingEvent;
/**
* For {@link Flag#SYNTHETIC} events, the {@link DomainHistory} revision ID of the {@link
* BillingRecurrence} from which this {@link google.registry.model.billing.BillingEvent} was
* created. This is needed in order to recreate the {@link VKey} when reading from SQL.
*/
@Column(name = "recurrence_history_revision_id")
Long recurrenceHistoryRevisionId;
/**
* The {@link AllocationToken} used in the creation of this event, or null if one was not used.
*/
@Nullable VKey<AllocationToken> allocationToken;
public Money getCost() {
return cost;
}
/** Entity id. */
@IdAllocation @Id Long id;
/** The registrar to bill. */
@Column(name = "registrarId", nullable = false)
String clientId;
/** Revision id of the entry in DomainHistory table that ths bill belongs to. */
@Column(nullable = false)
Long domainHistoryRevisionId;
/** ID of the EPP resource that the bill is for. */
@Column(nullable = false)
String domainRepoId;
/** When this event was created. For recurring events, this is also the recurrence start time. */
@Column(nullable = false)
DateTime eventTime;
/** The reason for the bill. */
@Enumerated(EnumType.STRING)
@Column(nullable = false)
Reason reason;
/** The fully qualified domain name of the domain that the bill is for. */
@Column(name = "domain_name", nullable = false)
String targetId;
@Nullable Set<Flag> flags;
public String getRegistrarId() {
return clientId;
public DateTime getBillingTime() {
return billingTime;
}
public long getDomainHistoryRevisionId() {
return domainHistoryRevisionId;
public Integer getPeriodYears() {
return periodYears;
}
public String getDomainRepoId() {
return domainRepoId;
public DateTime getSyntheticCreationTime() {
return syntheticCreationTime;
}
public DateTime getEventTime() {
return eventTime;
public VKey<BillingRecurrence> getCancellationMatchingBillingEvent() {
return cancellationMatchingBillingEvent;
}
public long getId() {
return id;
public Long getRecurrenceHistoryRevisionId() {
return recurrenceHistoryRevisionId;
}
public Reason getReason() {
return reason;
}
public String getTargetId() {
return targetId;
}
public HistoryEntryId getHistoryEntryId() {
return new HistoryEntryId(domainRepoId, domainHistoryRevisionId);
}
public ImmutableSet<Flag> getFlags() {
return nullToEmptyImmutableCopy(flags);
public Optional<VKey<AllocationToken>> getAllocationToken() {
return Optional.ofNullable(allocationToken);
}
@Override
public abstract VKey<? extends BillingEvent> createVKey();
public VKey<google.registry.model.billing.BillingEvent> createVKey() {
return createVKey(getId());
}
public static VKey<google.registry.model.billing.BillingEvent> createVKey(long id) {
return VKey.create(google.registry.model.billing.BillingEvent.class, id);
}
/** Override Buildable.asBuilder() to give this method stronger typing. */
@Override
public abstract Builder<?, ?> asBuilder();
public Builder asBuilder() {
return new Builder(clone(this));
}
/** An abstract builder for {@link BillingEvent}. */
public abstract static class Builder<T extends BillingEvent, B extends Builder<?, ?>>
extends GenericBuilder<T, B> {
/** A builder for {@link google.registry.model.billing.BillingEvent} since it is immutable. */
public static class Builder
extends BillingBase.Builder<google.registry.model.billing.BillingEvent, Builder> {
protected Builder() {}
public Builder() {}
protected Builder(T instance) {
private Builder(google.registry.model.billing.BillingEvent instance) {
super(instance);
}
public B setReason(Reason reason) {
getInstance().reason = reason;
return thisCastToDerived();
public Builder setCost(Money cost) {
getInstance().cost = cost;
return this;
}
public B setId(long id) {
getInstance().id = id;
return thisCastToDerived();
public Builder setPeriodYears(Integer periodYears) {
checkNotNull(periodYears);
checkArgument(periodYears > 0);
getInstance().periodYears = periodYears;
return this;
}
public B setRegistrarId(String registrarId) {
getInstance().clientId = registrarId;
return thisCastToDerived();
public Builder setBillingTime(DateTime billingTime) {
getInstance().billingTime = billingTime;
return this;
}
public B setEventTime(DateTime eventTime) {
getInstance().eventTime = eventTime;
return thisCastToDerived();
public Builder setSyntheticCreationTime(DateTime syntheticCreationTime) {
getInstance().syntheticCreationTime = syntheticCreationTime;
return this;
}
public B setTargetId(String targetId) {
getInstance().targetId = targetId;
return thisCastToDerived();
public Builder setCancellationMatchingBillingEvent(
BillingRecurrence cancellationMatchingBillingEvent) {
getInstance().cancellationMatchingBillingEvent =
cancellationMatchingBillingEvent.createVKey();
getInstance().recurrenceHistoryRevisionId =
cancellationMatchingBillingEvent.getDomainHistoryRevisionId();
return this;
}
public B setFlags(ImmutableSet<Flag> flags) {
getInstance().flags = forceEmptyToNull(checkArgumentNotNull(flags, "flags"));
return thisCastToDerived();
}
public B setDomainHistoryId(HistoryEntryId domainHistoryId) {
getInstance().domainHistoryRevisionId = domainHistoryId.getRevisionId();
getInstance().domainRepoId = domainHistoryId.getRepoId();
return thisCastToDerived();
}
public B setDomainHistory(DomainHistory domainHistory) {
return setDomainHistoryId(domainHistory.getHistoryEntryId());
public Builder setAllocationToken(@Nullable VKey<AllocationToken> allocationToken) {
getInstance().allocationToken = allocationToken;
return this;
}
@Override
public T build() {
T instance = getInstance();
checkNotNull(instance.reason, "Reason must be set");
checkNotNull(instance.clientId, "Registrar ID must be set");
checkNotNull(instance.eventTime, "Event time must be set");
checkNotNull(instance.targetId, "Target ID must be set");
checkNotNull(instance.domainHistoryRevisionId, "Domain History Revision ID must be set");
checkNotNull(instance.domainRepoId, "Domain Repo ID must be set");
public google.registry.model.billing.BillingEvent build() {
google.registry.model.billing.BillingEvent instance = getInstance();
checkNotNull(instance.billingTime);
checkNotNull(instance.cost);
checkState(!instance.cost.isNegative(), "Costs should be non-negative.");
// TODO(mcilwain): Enforce this check on all billing events (not just more recent ones)
// post-migration after we add the missing period years values in SQL.
if (instance.eventTime.isAfter(DateTime.parse("2019-01-01T00:00:00Z"))) {
checkState(
instance.reason.hasPeriodYears() == (instance.periodYears != null),
"Period years must be set if and only if reason is "
+ "CREATE, FEE_EARLY_ACCESS, RENEW, RESTORE or TRANSFER.");
}
checkState(
instance.getFlags().contains(Flag.SYNTHETIC) == (instance.syntheticCreationTime != null),
"Synthetic creation time must be set if and only if the SYNTHETIC flag is set.");
checkState(
instance.getFlags().contains(Flag.SYNTHETIC)
== (instance.cancellationMatchingBillingEvent != null),
"Cancellation matching billing event must be set if and only if the SYNTHETIC flag "
+ "is set.");
return super.build();
}
}
/** A one-time billable event. */
@Entity(name = "BillingEvent")
@Table(
indexes = {
@Index(columnList = "registrarId"),
@Index(columnList = "eventTime"),
@Index(columnList = "billingTime"),
@Index(columnList = "syntheticCreationTime"),
@Index(columnList = "domainRepoId"),
@Index(columnList = "allocationToken"),
@Index(columnList = "cancellation_matching_billing_recurrence_id")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_event_id"))
@WithVKey(Long.class)
public static class OneTime extends BillingEvent {
/** The billable value. */
@Type(type = JodaMoneyType.TYPE_NAME)
@Columns(columns = {@Column(name = "cost_amount"), @Column(name = "cost_currency")})
Money cost;
/** When the cost should be billed. */
DateTime billingTime;
/**
* The period in years of the action being billed for, if applicable, otherwise null. Used for
* financial reporting.
*/
Integer periodYears;
/**
* For {@link Flag#SYNTHETIC} events, when this event was persisted to the database (i.e. the
* cursor position at the time the recurrence expansion job was last run). In the event a job
* needs to be undone, a query on this field will return the complete set of potentially bad
* events.
*/
DateTime syntheticCreationTime;
/**
* For {@link Flag#SYNTHETIC} events, a {@link VKey} to the {@link Recurring} from which this
* {@link OneTime} was created. This is needed in order to properly match billing events against
* {@link Cancellation}s.
*/
@Column(name = "cancellation_matching_billing_recurrence_id")
VKey<Recurring> cancellationMatchingBillingEvent;
/**
* For {@link Flag#SYNTHETIC} events, the {@link DomainHistory} revision ID of the {@link
* Recurring} from which this {@link OneTime} was created. This is needed in order to recreate
* the {@link VKey} when reading from SQL.
*/
@Column(name = "recurrence_history_revision_id")
Long recurringEventHistoryRevisionId;
/**
* The {@link AllocationToken} used in the creation of this event, or null if one was not used.
*/
@Nullable VKey<AllocationToken> allocationToken;
public Money getCost() {
return cost;
}
public DateTime getBillingTime() {
return billingTime;
}
public Integer getPeriodYears() {
return periodYears;
}
public DateTime getSyntheticCreationTime() {
return syntheticCreationTime;
}
public VKey<Recurring> getCancellationMatchingBillingEvent() {
return cancellationMatchingBillingEvent;
}
public Long getRecurringEventHistoryRevisionId() {
return recurringEventHistoryRevisionId;
}
public Optional<VKey<AllocationToken>> getAllocationToken() {
return Optional.ofNullable(allocationToken);
}
@Override
public VKey<OneTime> createVKey() {
return createVKey(getId());
}
public static VKey<OneTime> createVKey(long id) {
return VKey.create(OneTime.class, id);
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
}
/** A builder for {@link OneTime} since it is immutable. */
public static class Builder extends BillingEvent.Builder<OneTime, Builder> {
public Builder() {}
private Builder(OneTime instance) {
super(instance);
}
public Builder setCost(Money cost) {
getInstance().cost = cost;
return this;
}
public Builder setPeriodYears(Integer periodYears) {
checkNotNull(periodYears);
checkArgument(periodYears > 0);
getInstance().periodYears = periodYears;
return this;
}
public Builder setBillingTime(DateTime billingTime) {
getInstance().billingTime = billingTime;
return this;
}
public Builder setSyntheticCreationTime(DateTime syntheticCreationTime) {
getInstance().syntheticCreationTime = syntheticCreationTime;
return this;
}
public Builder setCancellationMatchingBillingEvent(
Recurring cancellationMatchingBillingEvent) {
getInstance().cancellationMatchingBillingEvent =
cancellationMatchingBillingEvent.createVKey();
getInstance().recurringEventHistoryRevisionId =
cancellationMatchingBillingEvent.getDomainHistoryRevisionId();
return this;
}
public Builder setAllocationToken(@Nullable VKey<AllocationToken> allocationToken) {
getInstance().allocationToken = allocationToken;
return this;
}
@Override
public OneTime build() {
OneTime instance = getInstance();
checkNotNull(instance.billingTime);
checkNotNull(instance.cost);
checkState(!instance.cost.isNegative(), "Costs should be non-negative.");
// TODO(mcilwain): Enforce this check on all billing events (not just more recent ones)
// post-migration after we add the missing period years values in SQL.
if (instance.eventTime.isAfter(DateTime.parse("2019-01-01T00:00:00Z"))) {
checkState(
instance.reason.hasPeriodYears() == (instance.periodYears != null),
"Period years must be set if and only if reason is "
+ "CREATE, FEE_EARLY_ACCESS, RENEW, RESTORE or TRANSFER.");
}
checkState(
instance.getFlags().contains(Flag.SYNTHETIC)
== (instance.syntheticCreationTime != null),
"Synthetic creation time must be set if and only if the SYNTHETIC flag is set.");
checkState(
instance.getFlags().contains(Flag.SYNTHETIC)
== (instance.cancellationMatchingBillingEvent != null),
"Cancellation matching billing event must be set if and only if the SYNTHETIC flag "
+ "is set.");
return super.build();
}
}
}
/**
* A recurring billable event.
*
* <p>Unlike {@link OneTime} events, these do not store an explicit cost, since the cost of the
* recurring event might change and each time we bill for it, we need to bill at the current cost,
* not the value that was in use at the time the recurrence was created.
*/
@Entity(name = "BillingRecurrence")
@Table(
indexes = {
@Index(columnList = "registrarId"),
@Index(columnList = "eventTime"),
@Index(columnList = "domainRepoId"),
@Index(columnList = "recurrenceEndTime"),
@Index(columnList = "recurrenceLastExpansion"),
@Index(columnList = "recurrence_time_of_year")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_recurrence_id"))
@WithVKey(Long.class)
public static class Recurring extends BillingEvent {
/**
* The billing event recurs every year between {@link #eventTime} and this time on the [month,
* day, time] specified in {@link #recurrenceTimeOfYear}.
*/
DateTime recurrenceEndTime;
/**
* The most recent {@link DateTime} when this recurrence was expanded.
*
* <p>We only bother checking recurrences for potential expansion if this is at least one year
* in the past. If it's more recent than that, it means that the recurrence was already expanded
* too recently to need to be checked again (as domains autorenew each year).
*/
@Column(nullable = false)
DateTime recurrenceLastExpansion;
/**
* The eventTime recurs every year on this [month, day, time] between {@link #eventTime} and
* {@link #recurrenceEndTime}, inclusive of the start but not of the end.
*
* <p>This field is denormalized from {@link #eventTime} to allow for an efficient index, but it
* always has the same data as that field.
*
* <p>Note that this is a recurrence of the event time, not the billing time. The billing time
* can be calculated by adding the relevant grace period length to this date. The reason for
* this requirement is that the event time recurs on a {@link org.joda.time.Period} schedule
* (same day of year, which can be 365 or 366 days later) which is what {@link TimeOfYear} can
* model, whereas the billing time is a fixed {@link org.joda.time.Duration} later.
*/
@Embedded
@AttributeOverrides(
@AttributeOverride(name = "timeString", column = @Column(name = "recurrence_time_of_year")))
TimeOfYear recurrenceTimeOfYear;
/**
* The renewal price for domain renewal if and only if it's specified.
*
* <p>This price column remains null except when the renewal price behavior of the billing is
* SPECIFIED. This column is used for internal registrations.
*/
@Nullable
@Type(type = JodaMoneyType.TYPE_NAME)
@Columns(
columns = {@Column(name = "renewalPriceAmount"), @Column(name = "renewalPriceCurrency")})
Money renewalPrice;
@Enumerated(EnumType.STRING)
@Column(name = "renewalPriceBehavior", nullable = false)
RenewalPriceBehavior renewalPriceBehavior = RenewalPriceBehavior.DEFAULT;
public DateTime getRecurrenceEndTime() {
return recurrenceEndTime;
}
public DateTime getRecurrenceLastExpansion() {
return recurrenceLastExpansion;
}
public TimeOfYear getRecurrenceTimeOfYear() {
return recurrenceTimeOfYear;
}
public RenewalPriceBehavior getRenewalPriceBehavior() {
return renewalPriceBehavior;
}
public Optional<Money> getRenewalPrice() {
return Optional.ofNullable(renewalPrice);
}
@Override
public VKey<Recurring> createVKey() {
return createVKey(getId());
}
public static VKey<Recurring> createVKey(Long id) {
return VKey.create(Recurring.class, id);
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
}
/** A builder for {@link Recurring} since it is immutable. */
public static class Builder extends BillingEvent.Builder<Recurring, Builder> {
public Builder() {}
private Builder(Recurring instance) {
super(instance);
}
public Builder setRecurrenceEndTime(DateTime recurrenceEndTime) {
getInstance().recurrenceEndTime = recurrenceEndTime;
return this;
}
public Builder setRecurrenceLastExpansion(DateTime recurrenceLastExpansion) {
getInstance().recurrenceLastExpansion = recurrenceLastExpansion;
return this;
}
public Builder setRenewalPriceBehavior(RenewalPriceBehavior renewalPriceBehavior) {
getInstance().renewalPriceBehavior = renewalPriceBehavior;
return this;
}
public Builder setRenewalPrice(@Nullable Money renewalPrice) {
getInstance().renewalPrice = renewalPrice;
return this;
}
@Override
public Recurring build() {
Recurring instance = getInstance();
checkNotNull(instance.eventTime);
checkNotNull(instance.reason);
// Don't require recurrenceLastExpansion to be individually set on every new Recurrence.
// The correct default value if not otherwise set is the event time of the recurrence minus
// 1 year.
instance.recurrenceLastExpansion =
Optional.ofNullable(instance.recurrenceLastExpansion)
.orElse(instance.eventTime.minusYears(1));
checkArgument(
instance.renewalPriceBehavior == RenewalPriceBehavior.SPECIFIED
^ instance.renewalPrice == null,
"Renewal price can have a value if and only if the renewal price behavior is"
+ " SPECIFIED");
instance.recurrenceTimeOfYear = TimeOfYear.fromDateTime(instance.eventTime);
instance.recurrenceEndTime =
Optional.ofNullable(instance.recurrenceEndTime).orElse(END_OF_TIME);
return super.build();
}
}
}
/**
* An event representing a cancellation of one of the other two billable event types.
*
* <p>This is implemented as a separate event rather than a bit on BillingEvent in order to
* preserve the immutability of billing events.
*/
@Entity(name = "BillingCancellation")
@Table(
indexes = {
@Index(columnList = "registrarId"),
@Index(columnList = "eventTime"),
@Index(columnList = "domainRepoId"),
@Index(columnList = "billingTime"),
@Index(columnList = "billing_event_id"),
@Index(columnList = "billing_recurrence_id")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_cancellation_id"))
@WithVKey(Long.class)
public static class Cancellation extends BillingEvent {
/** The billing time of the charge that is being cancelled. */
DateTime billingTime;
/**
* The one-time billing event to cancel, or null for autorenew cancellations.
*
* <p>Although the type is {@link VKey} the name "ref" is preserved for historical reasons.
*/
@Column(name = "billing_event_id")
VKey<OneTime> refOneTime;
/**
* The recurring billing event to cancel, or null for non-autorenew cancellations.
*
* <p>Although the type is {@link VKey} the name "ref" is preserved for historical reasons.
*/
@Column(name = "billing_recurrence_id")
VKey<Recurring> refRecurring;
public DateTime getBillingTime() {
return billingTime;
}
public VKey<? extends BillingEvent> getEventKey() {
return firstNonNull(refOneTime, refRecurring);
}
/** The mapping from billable grace period types to originating billing event reasons. */
static final ImmutableMap<GracePeriodStatus, Reason> GRACE_PERIOD_TO_REASON =
ImmutableMap.of(
GracePeriodStatus.ADD, Reason.CREATE,
GracePeriodStatus.AUTO_RENEW, Reason.RENEW,
GracePeriodStatus.RENEW, Reason.RENEW,
GracePeriodStatus.TRANSFER, Reason.TRANSFER);
/**
* Creates a cancellation billing event (parented on the provided history key, and with the
* corresponding event time) that will cancel out the provided grace period's billing event,
* using the supplied targetId and deriving other metadata (clientId, billing time, and the
* cancellation reason) from the grace period.
*/
public static Cancellation forGracePeriod(
GracePeriod gracePeriod,
DateTime eventTime,
HistoryEntryId domainHistoryId,
String targetId) {
checkArgument(
gracePeriod.hasBillingEvent(),
"Cannot create cancellation for grace period without billing event");
Builder builder =
new Builder()
.setReason(checkNotNull(GRACE_PERIOD_TO_REASON.get(gracePeriod.getType())))
.setTargetId(targetId)
.setRegistrarId(gracePeriod.getRegistrarId())
.setEventTime(eventTime)
// The charge being cancelled will take place at the grace period's expiration time.
.setBillingTime(gracePeriod.getExpirationTime())
.setDomainHistoryId(domainHistoryId);
// Set the grace period's billing event using the appropriate Cancellation builder method.
if (gracePeriod.getOneTimeBillingEvent() != null) {
builder.setOneTimeEventKey(gracePeriod.getOneTimeBillingEvent());
} else if (gracePeriod.getRecurringBillingEvent() != null) {
builder.setRecurringEventKey(gracePeriod.getRecurringBillingEvent());
}
return builder.build();
}
@Override
public VKey<Cancellation> createVKey() {
return createVKey(getId());
}
public static VKey<Cancellation> createVKey(long id) {
return VKey.create(Cancellation.class, id);
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
}
/** A builder for {@link Cancellation} since it is immutable. */
public static class Builder extends BillingEvent.Builder<Cancellation, Builder> {
public Builder() {}
private Builder(Cancellation instance) {
super(instance);
}
public Builder setBillingTime(DateTime billingTime) {
getInstance().billingTime = billingTime;
return this;
}
public Builder setOneTimeEventKey(VKey<OneTime> eventKey) {
getInstance().refOneTime = eventKey;
return this;
}
public Builder setRecurringEventKey(VKey<Recurring> eventKey) {
getInstance().refRecurring = eventKey;
return this;
}
@Override
public Cancellation build() {
Cancellation instance = getInstance();
checkNotNull(instance.billingTime, "Must set billing time");
checkNotNull(instance.reason, "Must set reason");
checkState(
(instance.refOneTime == null) != (instance.refRecurring == null),
"Cancellations must have exactly one billing event key set");
return super.build();
}
}
}
}
@@ -0,0 +1,199 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.billing;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import google.registry.model.common.TimeOfYear;
import google.registry.persistence.VKey;
import google.registry.persistence.WithVKey;
import google.registry.persistence.converter.JodaMoneyType;
import java.util.Optional;
import javax.annotation.Nullable;
import javax.persistence.AttributeOverride;
import javax.persistence.AttributeOverrides;
import javax.persistence.Column;
import javax.persistence.Embedded;
import javax.persistence.Entity;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.Index;
import javax.persistence.Table;
import org.hibernate.annotations.Columns;
import org.hibernate.annotations.Type;
import org.joda.money.Money;
import org.joda.time.DateTime;
/**
* A recurring billable event.
*
* <p>Unlike {@link BillingEvent} events, these do not store an explicit cost, since the cost of the
* recurring event might change and each time we bill for it, we need to bill at the current cost,
* not the value that was in use at the time the recurrence was created.
*/
@Entity
@Table(
indexes = {
@Index(columnList = "registrarId"),
@Index(columnList = "eventTime"),
@Index(columnList = "domainRepoId"),
@Index(columnList = "recurrenceEndTime"),
@Index(columnList = "recurrenceLastExpansion"),
@Index(columnList = "recurrence_time_of_year")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_recurrence_id"))
@WithVKey(Long.class)
public class BillingRecurrence extends BillingBase {
/**
* The billing event recurs every year between {@link #eventTime} and this time on the [month,
* day, time] specified in {@link #recurrenceTimeOfYear}.
*/
DateTime recurrenceEndTime;
/**
* The most recent {@link DateTime} when this recurrence was expanded.
*
* <p>We only bother checking recurrences for potential expansion if this is at least one year in
* the past. If it's more recent than that, it means that the recurrence was already expanded too
* recently to need to be checked again (as domains autorenew each year).
*/
@Column(nullable = false)
DateTime recurrenceLastExpansion;
/**
* The eventTime recurs every year on this [month, day, time] between {@link #eventTime} and
* {@link #recurrenceEndTime}, inclusive of the start but not of the end.
*
* <p>This field is denormalized from {@link #eventTime} to allow for an efficient index, but it
* always has the same data as that field.
*
* <p>Note that this is a recurrence of the event time, not the billing time. The billing time can
* be calculated by adding the relevant grace period length to this date. The reason for this
* requirement is that the event time recurs on a {@link org.joda.time.Period} schedule (same day
* of year, which can be 365 or 366 days later) which is what {@link TimeOfYear} can model,
* whereas the billing time is a fixed {@link org.joda.time.Duration} later.
*/
@Embedded
@AttributeOverrides(
@AttributeOverride(name = "timeString", column = @Column(name = "recurrence_time_of_year")))
TimeOfYear recurrenceTimeOfYear;
/**
* The renewal price for domain renewal if and only if it's specified.
*
* <p>This price column remains null except when the renewal price behavior of the billing is
* SPECIFIED. This column is used for internal registrations.
*/
@Nullable
@Type(type = JodaMoneyType.TYPE_NAME)
@Columns(columns = {@Column(name = "renewalPriceAmount"), @Column(name = "renewalPriceCurrency")})
Money renewalPrice;
@Enumerated(EnumType.STRING)
@Column(name = "renewalPriceBehavior", nullable = false)
RenewalPriceBehavior renewalPriceBehavior = RenewalPriceBehavior.DEFAULT;
public DateTime getRecurrenceEndTime() {
return recurrenceEndTime;
}
public DateTime getRecurrenceLastExpansion() {
return recurrenceLastExpansion;
}
public TimeOfYear getRecurrenceTimeOfYear() {
return recurrenceTimeOfYear;
}
public RenewalPriceBehavior getRenewalPriceBehavior() {
return renewalPriceBehavior;
}
public Optional<Money> getRenewalPrice() {
return Optional.ofNullable(renewalPrice);
}
@Override
public VKey<google.registry.model.billing.BillingRecurrence> createVKey() {
return createVKey(getId());
}
public static VKey<google.registry.model.billing.BillingRecurrence> createVKey(Long id) {
return VKey.create(google.registry.model.billing.BillingRecurrence.class, id);
}
@Override
public Builder asBuilder() {
return new Builder(clone(this));
}
/**
* A builder for {@link google.registry.model.billing.BillingRecurrence} since it is immutable.
*/
public static class Builder
extends BillingBase.Builder<google.registry.model.billing.BillingRecurrence, Builder> {
public Builder() {}
private Builder(google.registry.model.billing.BillingRecurrence instance) {
super(instance);
}
public Builder setRecurrenceEndTime(DateTime recurrenceEndTime) {
getInstance().recurrenceEndTime = recurrenceEndTime;
return this;
}
public Builder setRecurrenceLastExpansion(DateTime recurrenceLastExpansion) {
getInstance().recurrenceLastExpansion = recurrenceLastExpansion;
return this;
}
public Builder setRenewalPriceBehavior(RenewalPriceBehavior renewalPriceBehavior) {
getInstance().renewalPriceBehavior = renewalPriceBehavior;
return this;
}
public Builder setRenewalPrice(@Nullable Money renewalPrice) {
getInstance().renewalPrice = renewalPrice;
return this;
}
@Override
public google.registry.model.billing.BillingRecurrence build() {
google.registry.model.billing.BillingRecurrence instance = getInstance();
checkNotNull(instance.eventTime);
checkNotNull(instance.reason);
// Don't require recurrenceLastExpansion to be individually set on every new Recurrence.
// The correct default value if not otherwise set is the event time of the recurrence minus
// 1 year.
instance.recurrenceLastExpansion =
Optional.ofNullable(instance.recurrenceLastExpansion)
.orElse(instance.eventTime.minusYears(1));
checkArgument(
instance.renewalPriceBehavior == RenewalPriceBehavior.SPECIFIED
^ instance.renewalPrice == null,
"Renewal price can have a value if and only if the renewal price behavior is"
+ " SPECIFIED");
instance.recurrenceTimeOfYear = TimeOfYear.fromDateTime(instance.eventTime);
instance.recurrenceEndTime =
Optional.ofNullable(instance.recurrenceEndTime).orElse(END_OF_TIME);
return super.build();
}
}
}
@@ -1,275 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.common;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import com.github.benmanes.caffeine.cache.LoadingCache;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableMultimap;
import com.google.common.collect.ImmutableSortedMap;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryEnvironment;
import google.registry.model.CacheUtils;
import google.registry.model.annotations.DeleteAfterMigration;
import java.time.Duration;
import java.util.Arrays;
import javax.persistence.Entity;
import javax.persistence.PersistenceException;
import org.joda.time.DateTime;
/**
* A wrapper object representing the stage-to-time mapping of the Registry 3.0 Cloud SQL migration.
*
* <p>The entity is stored in SQL throughout the entire migration so as to have a single point of
* access.
*/
@DeleteAfterMigration
@Entity
public class DatabaseMigrationStateSchedule extends CrossTldSingleton {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static boolean useUncachedForTest = false;
public enum PrimaryDatabase {
CLOUD_SQL,
DATASTORE
}
public enum ReplayDirection {
NO_REPLAY,
DATASTORE_TO_SQL,
SQL_TO_DATASTORE
}
/**
* The current phase of the migration plus information about which database to use and whether or
* not the phase is read-only.
*/
public enum MigrationState {
/** Datastore is the only DB being used. */
DATASTORE_ONLY(PrimaryDatabase.DATASTORE, false, ReplayDirection.NO_REPLAY),
/** Datastore is the primary DB, with changes replicated to Cloud SQL. */
DATASTORE_PRIMARY(PrimaryDatabase.DATASTORE, false, ReplayDirection.DATASTORE_TO_SQL),
/** Datastore is the primary DB, with replication, and async actions are disallowed. */
DATASTORE_PRIMARY_NO_ASYNC(PrimaryDatabase.DATASTORE, false, ReplayDirection.DATASTORE_TO_SQL),
/** Datastore is the primary DB, with replication, and all mutating actions are disallowed. */
DATASTORE_PRIMARY_READ_ONLY(PrimaryDatabase.DATASTORE, true, ReplayDirection.DATASTORE_TO_SQL),
/**
* Cloud SQL is the primary DB, with replication back to Datastore, and all mutating actions are
* disallowed.
*/
SQL_PRIMARY_READ_ONLY(PrimaryDatabase.CLOUD_SQL, true, ReplayDirection.SQL_TO_DATASTORE),
/** Cloud SQL is the primary DB, with changes replicated to Datastore. */
SQL_PRIMARY(PrimaryDatabase.CLOUD_SQL, false, ReplayDirection.SQL_TO_DATASTORE),
/** Cloud SQL is the only DB being used. */
SQL_ONLY(PrimaryDatabase.CLOUD_SQL, false, ReplayDirection.NO_REPLAY),
/** Toggles SQL Sequence based allocateId */
SEQUENCE_BASED_ALLOCATE_ID(PrimaryDatabase.CLOUD_SQL, false, ReplayDirection.NO_REPLAY),
/** Use SQL-based Nordn upload flow instead of the pull queue-based one. */
NORDN_SQL(PrimaryDatabase.CLOUD_SQL, false, ReplayDirection.NO_REPLAY),
/** Use SQL-based DNS update flow instead of the pull queue-based one. */
DNS_SQL(PrimaryDatabase.CLOUD_SQL, false, ReplayDirection.NO_REPLAY);
private final PrimaryDatabase primaryDatabase;
private final boolean isReadOnly;
private final ReplayDirection replayDirection;
public PrimaryDatabase getPrimaryDatabase() {
return primaryDatabase;
}
public boolean isReadOnly() {
return isReadOnly;
}
public ReplayDirection getReplayDirection() {
return replayDirection;
}
MigrationState(
PrimaryDatabase primaryDatabase, boolean isReadOnly, ReplayDirection replayDirection) {
this.primaryDatabase = primaryDatabase;
this.isReadOnly = isReadOnly;
this.replayDirection = replayDirection;
}
}
/**
* Cache of the current migration schedule. The key is meaningless; this is essentially a memoized
* Supplier that can be reset for testing purposes and after writes.
*/
@VisibleForTesting
public static final LoadingCache<
Class<DatabaseMigrationStateSchedule>, TimedTransitionProperty<MigrationState>>
// Each instance should cache the migration schedule for five minutes before reloading
CACHE =
CacheUtils.newCacheBuilder(Duration.ofMinutes(5))
.build(singletonClazz -> DatabaseMigrationStateSchedule.getUncached());
// Restrictions on the state transitions, e.g. no going from DATASTORE_ONLY to SQL_ONLY
private static final ImmutableMultimap<MigrationState, MigrationState> VALID_STATE_TRANSITIONS =
createValidStateTransitions();
/**
* The valid state transitions. Generally, one can advance the state one step or move backward any
* number of steps, as long as the step we're moving back to has the same primary database as the
* one we're in. Otherwise, we must move to the corresponding READ_ONLY stage first.
*/
private static ImmutableMultimap<MigrationState, MigrationState> createValidStateTransitions() {
ImmutableMultimap.Builder<MigrationState, MigrationState> builder =
new ImmutableMultimap.Builder<MigrationState, MigrationState>()
.put(MigrationState.DATASTORE_ONLY, MigrationState.DATASTORE_PRIMARY)
.putAll(
MigrationState.DATASTORE_PRIMARY,
MigrationState.DATASTORE_ONLY,
MigrationState.DATASTORE_PRIMARY_NO_ASYNC)
.putAll(
MigrationState.DATASTORE_PRIMARY_NO_ASYNC,
MigrationState.DATASTORE_ONLY,
MigrationState.DATASTORE_PRIMARY,
MigrationState.DATASTORE_PRIMARY_READ_ONLY)
.putAll(
MigrationState.DATASTORE_PRIMARY_READ_ONLY,
MigrationState.DATASTORE_ONLY,
MigrationState.DATASTORE_PRIMARY,
MigrationState.DATASTORE_PRIMARY_NO_ASYNC,
MigrationState.SQL_PRIMARY_READ_ONLY,
MigrationState.SQL_PRIMARY)
.putAll(
MigrationState.SQL_PRIMARY_READ_ONLY,
MigrationState.DATASTORE_PRIMARY_READ_ONLY,
MigrationState.SQL_PRIMARY)
.putAll(
MigrationState.SQL_PRIMARY,
MigrationState.SQL_PRIMARY_READ_ONLY,
MigrationState.SQL_ONLY)
.putAll(
MigrationState.SQL_ONLY,
MigrationState.SQL_PRIMARY_READ_ONLY,
MigrationState.SQL_PRIMARY)
.putAll(MigrationState.SQL_ONLY, MigrationState.SEQUENCE_BASED_ALLOCATE_ID)
.putAll(MigrationState.SEQUENCE_BASED_ALLOCATE_ID, MigrationState.NORDN_SQL)
.putAll(
MigrationState.NORDN_SQL,
MigrationState.SEQUENCE_BASED_ALLOCATE_ID,
MigrationState.DNS_SQL)
.putAll(MigrationState.DNS_SQL, MigrationState.NORDN_SQL);
// In addition, we can always transition from a state to itself (useful when updating the map).
Arrays.stream(MigrationState.values()).forEach(state -> builder.put(state, state));
return builder.build();
}
// Default map to return if we have never saved any -- only use Datastore.
@VisibleForTesting
public static final TimedTransitionProperty<MigrationState> DEFAULT_TRANSITION_MAP =
TimedTransitionProperty.fromValueMap(
ImmutableSortedMap.of(START_OF_TIME, MigrationState.DATASTORE_ONLY));
@VisibleForTesting
public TimedTransitionProperty<MigrationState> migrationTransitions =
TimedTransitionProperty.withInitialValue(MigrationState.DATASTORE_ONLY);
// Required for Hibernate initialization
protected DatabaseMigrationStateSchedule() {}
@VisibleForTesting
public DatabaseMigrationStateSchedule(
TimedTransitionProperty<MigrationState> migrationTransitions) {
this.migrationTransitions = migrationTransitions;
}
/** Sets and persists to SQL the provided migration transition schedule. */
public static void set(ImmutableSortedMap<DateTime, MigrationState> migrationTransitionMap) {
tm().assertInTransaction();
TimedTransitionProperty<MigrationState> transitions =
TimedTransitionProperty.make(
migrationTransitionMap,
VALID_STATE_TRANSITIONS,
"validStateTransitions",
MigrationState.DATASTORE_ONLY,
"migrationTransitionMap must start with DATASTORE_ONLY");
validateTransitionAtCurrentTime(transitions);
tm().put(new DatabaseMigrationStateSchedule(transitions));
CACHE.invalidateAll();
}
@VisibleForTesting
public static void useUncachedForTest() {
useUncachedForTest = true;
}
/** Loads the currently-set migration schedule from the cache, or the default if none exists. */
public static TimedTransitionProperty<MigrationState> get() {
return CACHE.get(DatabaseMigrationStateSchedule.class);
}
/** Returns the database migration status at the given time. */
public static MigrationState getValueAtTime(DateTime dateTime) {
return useUncachedForTest
? getUncached().getValueAtTime(dateTime)
: get().getValueAtTime(dateTime);
}
/** Loads the currently-set migration schedule from SQL, or the default if none exists. */
@VisibleForTesting
static TimedTransitionProperty<MigrationState> getUncached() {
return tm().transact(
() -> {
try {
return tm().loadSingleton(DatabaseMigrationStateSchedule.class)
.map(s -> s.migrationTransitions)
.orElse(DEFAULT_TRANSITION_MAP);
} catch (PersistenceException e) {
if (!RegistryEnvironment.get().equals(RegistryEnvironment.UNITTEST)) {
throw e;
}
logger.atWarning().withCause(e).log(
"Error when retrieving migration schedule; this should only happen in tests.");
return DEFAULT_TRANSITION_MAP;
}
});
}
/**
* A provided map of transitions may be valid by itself (i.e. it shifts states properly, doesn't
* skip states, and doesn't backtrack incorrectly) while still being invalid. In addition to the
* transitions in the map being valid, the single transition from the current map at the current
* time to the new map at the current time must also be valid.
*/
private static void validateTransitionAtCurrentTime(
TimedTransitionProperty<MigrationState> newTransitions) {
MigrationState currentValue = getUncached().getValueAtTime(tm().getTransactionTime());
MigrationState nextCurrentValue = newTransitions.getValueAtTime(tm().getTransactionTime());
checkArgument(
VALID_STATE_TRANSITIONS.get(currentValue).contains(nextCurrentValue),
"Cannot transition from current state-as-of-now %s to new state-as-of-now %s",
currentValue,
nextCurrentValue);
}
}
@@ -18,7 +18,7 @@ import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import google.registry.dns.DnsConstants.TargetType;
import google.registry.dns.DnsUtils.TargetType;
import google.registry.dns.PublishDnsUpdatesAction;
import google.registry.model.ImmutableObject;
import google.registry.persistence.VKey;
@@ -44,7 +44,7 @@ import com.google.gson.annotations.Expose;
import google.registry.flows.ResourceFlowUtils;
import google.registry.model.EppResource;
import google.registry.model.EppResource.ResourceWithTransferData;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.contact.Contact;
import google.registry.model.domain.launch.LaunchNotice;
import google.registry.model.domain.rgp.GracePeriodStatus;
@@ -200,7 +200,7 @@ public class DomainBase extends EppResource
* should be created, and this field should be updated to point to the new one.
*/
@Column(name = "billing_recurrence_id")
VKey<Recurring> autorenewBillingEvent;
VKey<BillingRecurrence> autorenewBillingEvent;
/**
* The recurring poll message associated with this domain's autorenewals.
@@ -286,7 +286,7 @@ public class DomainBase extends EppResource
return deletePollMessage;
}
public VKey<Recurring> getAutorenewBillingEvent() {
public VKey<BillingRecurrence> getAutorenewBillingEvent() {
return autorenewBillingEvent;
}
@@ -520,7 +520,7 @@ public class DomainBase extends EppResource
builder
.setRegistrationExpirationTime(newExpirationTime)
.addGracePeriod(
GracePeriod.createForRecurring(
GracePeriod.createForRecurrence(
GracePeriodStatus.AUTO_RENEW,
domain.getRepoId(),
lastAutorenewTime.plus(Tld.get(domain.getTld()).getAutoRenewGracePeriodLength()),
@@ -847,7 +847,7 @@ public class DomainBase extends EppResource
return thisCastToDerived();
}
public B setAutorenewBillingEvent(VKey<Recurring> autorenewBillingEvent) {
public B setAutorenewBillingEvent(VKey<BillingRecurrence> autorenewBillingEvent) {
getInstance().autorenewBillingEvent = autorenewBillingEvent;
return thisCastToDerived();
}
@@ -20,7 +20,7 @@ import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.google.common.annotations.VisibleForTesting;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.persistence.VKey;
@@ -60,23 +60,23 @@ public class GracePeriod extends GracePeriodBase {
String domainRepoId,
DateTime expirationTime,
String registrarId,
@Nullable VKey<BillingEvent.OneTime> billingEventOneTime,
@Nullable VKey<BillingEvent.Recurring> billingEventRecurring,
@Nullable VKey<BillingEvent> billingEvent,
@Nullable VKey<BillingRecurrence> billingRecurrence,
@Nullable Long gracePeriodId) {
checkArgument(
billingEventOneTime == null || billingEventRecurring == null,
billingEvent == null || billingRecurrence == null,
"A grace period can have at most one billing event");
checkArgument(
(billingEventRecurring != null) == GracePeriodStatus.AUTO_RENEW.equals(type),
"Recurring billing events must be present on (and only on) autorenew grace periods");
(billingRecurrence != null) == GracePeriodStatus.AUTO_RENEW.equals(type),
"BillingRecurrences must be present on (and only on) autorenew grace periods");
GracePeriod instance = new GracePeriod();
instance.gracePeriodId = gracePeriodId == null ? allocateId() : gracePeriodId;
instance.type = checkArgumentNotNull(type);
instance.domainRepoId = checkArgumentNotNull(domainRepoId);
instance.expirationTime = checkArgumentNotNull(expirationTime);
instance.clientId = checkArgumentNotNull(registrarId);
instance.billingEventOneTime = billingEventOneTime;
instance.billingEventRecurring = billingEventRecurring;
instance.billingEvent = billingEvent;
instance.billingRecurrence = billingRecurrence;
return instance;
}
@@ -92,7 +92,7 @@ public class GracePeriod extends GracePeriodBase {
String domainRepoId,
DateTime expirationTime,
String registrarId,
@Nullable VKey<BillingEvent.OneTime> billingEventOneTime) {
@Nullable VKey<BillingEvent> billingEventOneTime) {
return createInternal(
type, domainRepoId, expirationTime, registrarId, billingEventOneTime, null, null);
}
@@ -111,7 +111,7 @@ public class GracePeriod extends GracePeriodBase {
String domainRepoId,
DateTime expirationTime,
String registrarId,
@Nullable VKey<BillingEvent.OneTime> billingEventOneTime,
@Nullable VKey<BillingEvent> billingEventOneTime,
@Nullable Long gracePeriodId) {
return createInternal(
type, domainRepoId, expirationTime, registrarId, billingEventOneTime, null, gracePeriodId);
@@ -123,40 +123,40 @@ public class GracePeriod extends GracePeriodBase {
history.domainRepoId,
history.expirationTime,
history.clientId,
history.billingEventOneTime,
history.billingEventRecurring,
history.billingEvent,
history.billingRecurrence,
history.gracePeriodId);
}
/** Creates a GracePeriod for a Recurring billing event. */
public static GracePeriod createForRecurring(
/** Creates a GracePeriod for a Recurrence billing event. */
public static GracePeriod createForRecurrence(
GracePeriodStatus type,
String domainRepoId,
DateTime expirationTime,
String registrarId,
VKey<Recurring> billingEventRecurring) {
checkArgumentNotNull(billingEventRecurring, "billingEventRecurring cannot be null");
VKey<BillingRecurrence> billingEventRecurrence) {
checkArgumentNotNull(billingEventRecurrence, "billingEventRecurrence cannot be null");
return createInternal(
type, domainRepoId, expirationTime, registrarId, null, billingEventRecurring, null);
type, domainRepoId, expirationTime, registrarId, null, billingEventRecurrence, null);
}
/** Creates a GracePeriod for a Recurring billing event and a given {@link #gracePeriodId}. */
/** Creates a GracePeriod for a Recurrence billing event and a given {@link #gracePeriodId}. */
@VisibleForTesting
public static GracePeriod createForRecurring(
public static GracePeriod createForRecurrence(
GracePeriodStatus type,
String domainRepoId,
DateTime expirationTime,
String registrarId,
VKey<Recurring> billingEventRecurring,
VKey<BillingRecurrence> billingEventRecurrence,
@Nullable Long gracePeriodId) {
checkArgumentNotNull(billingEventRecurring, "billingEventRecurring cannot be null");
checkArgumentNotNull(billingEventRecurrence, "billingEventRecurrence cannot be null");
return createInternal(
type,
domainRepoId,
expirationTime,
registrarId,
null,
billingEventRecurring,
billingEventRecurrence,
gracePeriodId);
}
@@ -168,7 +168,7 @@ public class GracePeriod extends GracePeriodBase {
/** Constructs a GracePeriod of the given type from the provided one-time BillingEvent. */
public static GracePeriod forBillingEvent(
GracePeriodStatus type, String domainRepoId, BillingEvent.OneTime billingEvent) {
GracePeriodStatus type, String domainRepoId, BillingEvent billingEvent) {
return create(
type,
domainRepoId,
@@ -205,8 +205,8 @@ public class GracePeriod extends GracePeriodBase {
instance.domainRepoId = gracePeriod.domainRepoId;
instance.expirationTime = gracePeriod.expirationTime;
instance.clientId = gracePeriod.clientId;
instance.billingEventOneTime = gracePeriod.billingEventOneTime;
instance.billingEventRecurring = gracePeriod.billingEventRecurring;
instance.billingEvent = gracePeriod.billingEvent;
instance.billingRecurrence = gracePeriod.billingRecurrence;
return instance;
}
}
@@ -17,6 +17,7 @@ package google.registry.model.domain;
import google.registry.model.ImmutableObject;
import google.registry.model.UnsafeSerializable;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.persistence.VKey;
import javax.persistence.Access;
@@ -56,21 +57,21 @@ public class GracePeriodBase extends ImmutableObject implements UnsafeSerializab
/**
* The one-time billing event corresponding to the action that triggered this grace period, or
* null if not applicable. Not set for autorenew grace periods (which instead use the field {@code
* billingEventRecurring}) or for redemption grace periods (since deletes have no cost).
* billingEventRecurrence}) or for redemption grace periods (since deletes have no cost).
*/
// NB: Would @IgnoreSave(IfNull.class), but not allowed for @Embed collections.
@Access(AccessType.FIELD)
@Column(name = "billing_event_id")
VKey<BillingEvent.OneTime> billingEventOneTime = null;
VKey<BillingEvent> billingEvent = null;
/**
* The recurring billing event corresponding to the action that triggered this grace period, if
* applicable - i.e. if the action was an autorenew - or null in all other cases.
* The Recurrence corresponding to the action that triggered this grace period, if applicable -
* i.e. if the action was an autorenew - or null in all other cases.
*/
// NB: Would @IgnoreSave(IfNull.class), but not allowed for @Embed collections.
@Access(AccessType.FIELD)
@Column(name = "billing_recurrence_id")
VKey<BillingEvent.Recurring> billingEventRecurring = null;
VKey<BillingRecurrence> billingRecurrence = null;
public long getGracePeriodId() {
return gracePeriodId;
@@ -100,22 +101,22 @@ public class GracePeriodBase extends ImmutableObject implements UnsafeSerializab
/** Returns true if this GracePeriod has an associated BillingEvent; i.e. if it's refundable. */
public boolean hasBillingEvent() {
return billingEventOneTime != null || billingEventRecurring != null;
return billingEvent != null || billingRecurrence != null;
}
/**
* Returns the one time billing event. The value will only be non-null if the type of this grace
* period is not AUTO_RENEW.
*/
public VKey<BillingEvent.OneTime> getOneTimeBillingEvent() {
return billingEventOneTime;
public VKey<BillingEvent> getBillingEvent() {
return billingEvent;
}
/**
* Returns the recurring billing event. The value will only be non-null if the type of this grace
* period is AUTO_RENEW.
* Returns the Recurrence. The value will only be non-null if the type of this grace period is
* AUTO_RENEW.
*/
public VKey<BillingEvent.Recurring> getRecurringBillingEvent() {
return billingEventRecurring;
public VKey<BillingRecurrence> getBillingRecurrence() {
return billingRecurrence;
}
}
@@ -43,7 +43,7 @@ import google.registry.model.Buildable;
import google.registry.model.CacheUtils;
import google.registry.model.CreateAutoTimestamp;
import google.registry.model.UpdateAutoTimestampEntity;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.common.TimedTransitionProperty;
import google.registry.model.domain.fee.FeeQueryCommandExtensionItem.CommandName;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
@@ -17,6 +17,7 @@ package google.registry.model.reporting;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import google.registry.batch.ExpandBillingRecurrencesAction;
import google.registry.model.Buildable;
import google.registry.model.EppResource;
import google.registry.model.ImmutableObject;
@@ -81,7 +82,7 @@ public abstract class HistoryEntry extends ImmutableObject
DOMAIN_ALLOCATE,
/**
* Used for domain registration autorenews explicitly logged by {@link
* google.registry.batch.ExpandRecurringBillingEventsAction}.
* ExpandBillingRecurrencesAction}.
*/
DOMAIN_AUTORENEW,
DOMAIN_CREATE,
@@ -671,18 +671,18 @@ public class Tld extends ImmutableObject implements Buildable, UnsafeSerializabl
}
/** Returns the time to live for A and AAAA records. */
public Duration getDnsAPlusAaaaTtl() {
return dnsAPlusAaaaTtl;
public Optional<Duration> getDnsAPlusAaaaTtl() {
return Optional.ofNullable(dnsAPlusAaaaTtl);
}
/** Returns the time to live for NS records. */
public Duration getDnsNsTtl() {
return dnsNsTtl;
public Optional<Duration> getDnsNsTtl() {
return Optional.ofNullable(dnsNsTtl);
}
/** Returns the time to live for DS records. */
public Duration getDnsDsTtl() {
return dnsDsTtl;
public Optional<Duration> getDnsDsTtl() {
return Optional.ofNullable(dnsDsTtl);
}
public ImmutableSet<String> getAllowedRegistrantContactIds() {
@@ -17,7 +17,9 @@ package google.registry.model.transfer;
import static google.registry.util.CollectionUtils.isNullOrEmpty;
import com.google.common.collect.ImmutableSet;
import google.registry.model.billing.BillingCancellation;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Period;
import google.registry.model.domain.Period.Unit;
import google.registry.model.poll.PollMessage;
@@ -69,7 +71,7 @@ public class DomainTransferData extends TransferData {
DateTime transferredRegistrationExpirationTime;
@Column(name = "transfer_billing_cancellation_id")
public VKey<BillingEvent.Cancellation> billingCancellationId;
public VKey<BillingCancellation> billingCancellationId;
/**
* The regular one-time billing event that will be charged for a server-approved transfer.
@@ -78,7 +80,7 @@ public class DomainTransferData extends TransferData {
* being transferred is not a domain.
*/
@Column(name = "transfer_billing_event_id")
VKey<BillingEvent.OneTime> serverApproveBillingEvent;
VKey<BillingEvent> serverApproveBillingEvent;
/**
* The autorenew billing event that should be associated with this resource after the transfer.
@@ -87,7 +89,7 @@ public class DomainTransferData extends TransferData {
* being transferred is not a domain.
*/
@Column(name = "transfer_billing_recurrence_id")
VKey<BillingEvent.Recurring> serverApproveAutorenewEvent;
VKey<BillingRecurrence> serverApproveAutorenewEvent;
/**
* The autorenew poll message that should be associated with this resource after the transfer.
@@ -120,12 +122,12 @@ public class DomainTransferData extends TransferData {
}
@Nullable
public VKey<BillingEvent.OneTime> getServerApproveBillingEvent() {
public VKey<BillingEvent> getServerApproveBillingEvent() {
return serverApproveBillingEvent;
}
@Nullable
public VKey<BillingEvent.Recurring> getServerApproveAutorenewEvent() {
public VKey<BillingRecurrence> getServerApproveAutorenewEvent() {
return serverApproveAutorenewEvent;
}
@@ -176,9 +178,9 @@ public class DomainTransferData extends TransferData {
domainTransferData.billingCancellationId = null;
} else {
domainTransferData.billingCancellationId =
(VKey<BillingEvent.Cancellation>)
(VKey<BillingCancellation>)
serverApproveEntities.stream()
.filter(k -> k.getKind().equals(BillingEvent.Cancellation.class))
.filter(k -> k.getKind().equals(BillingCancellation.class))
.findFirst()
.orElse(null);
}
@@ -204,14 +206,13 @@ public class DomainTransferData extends TransferData {
return this;
}
public Builder setServerApproveBillingEvent(
VKey<BillingEvent.OneTime> serverApproveBillingEvent) {
public Builder setServerApproveBillingEvent(VKey<BillingEvent> serverApproveBillingEvent) {
getInstance().serverApproveBillingEvent = serverApproveBillingEvent;
return this;
}
public Builder setServerApproveAutorenewEvent(
VKey<BillingEvent.Recurring> serverApproveAutorenewEvent) {
VKey<BillingRecurrence> serverApproveAutorenewEvent) {
getInstance().serverApproveAutorenewEvent = serverApproveAutorenewEvent;
return this;
}
@@ -21,7 +21,7 @@ import google.registry.batch.CannedScriptExecutionAction;
import google.registry.batch.DeleteExpiredDomainsAction;
import google.registry.batch.DeleteLoadTestDataAction;
import google.registry.batch.DeleteProberDataAction;
import google.registry.batch.ExpandRecurringBillingEventsAction;
import google.registry.batch.ExpandBillingRecurrencesAction;
import google.registry.batch.RelockDomainAction;
import google.registry.batch.ResaveAllEppResourcesPipelineAction;
import google.registry.batch.ResaveEntityAction;
@@ -32,7 +32,6 @@ import google.registry.cron.CronModule;
import google.registry.cron.TldFanoutAction;
import google.registry.dns.DnsModule;
import google.registry.dns.PublishDnsUpdatesAction;
import google.registry.dns.ReadDnsQueueAction;
import google.registry.dns.ReadDnsRefreshRequestsAction;
import google.registry.dns.RefreshDnsAction;
import google.registry.dns.RefreshDnsOnHostRenameAction;
@@ -115,7 +114,7 @@ interface BackendRequestComponent {
DeleteProberDataAction deleteProberDataAction();
ExpandRecurringBillingEventsAction expandRecurringBillingEventsAction();
ExpandBillingRecurrencesAction expandBillingRecurrencesAction();
ExportDomainListsAction exportDomainListsAction();
@@ -143,8 +142,6 @@ interface BackendRequestComponent {
PublishSpec11ReportAction publishSpec11ReportAction();
ReadDnsQueueAction readDnsQueueAction();
ReadDnsRefreshRequestsAction readDnsRefreshRequestsAction();
RdeReportAction rdeReportAction();
@@ -14,7 +14,7 @@
package google.registry.persistence.converter;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingBase.Flag;
import java.util.Set;
import javax.persistence.AttributeConverter;
import javax.persistence.Converter;
@@ -1,37 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.persistence.converter;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.common.DatabaseMigrationStateSchedule;
import google.registry.model.common.DatabaseMigrationStateSchedule.MigrationState;
import javax.persistence.Converter;
/** JPA converter for {@link DatabaseMigrationStateSchedule} transitions. */
@DeleteAfterMigration
@Converter(autoApply = true)
public class DatabaseMigrationScheduleTransitionConverter
extends TimedTransitionPropertyConverterBase<MigrationState> {
@Override
protected String convertValueToString(MigrationState value) {
return value.name();
}
@Override
protected MigrationState convertStringToValue(String string) {
return MigrationState.valueOf(string);
}
}
@@ -21,7 +21,7 @@ import static google.registry.request.RequestParameters.extractRequiredParameter
import com.google.api.services.dataflow.Dataflow;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule.DefaultCredential;
import google.registry.config.CredentialModule.ApplicationDefaultCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.request.HttpException.BadRequestException;
import google.registry.request.Parameter;
@@ -134,7 +134,7 @@ public class ReportingModule {
/** Constructs a {@link Dataflow} API client with default settings. */
@Provides
static Dataflow provideDataflow(
@DefaultCredential GoogleCredentialsBundle credentialsBundle,
@ApplicationDefaultCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Dataflow.Builder(
credentialsBundle.getHttpTransport(),
@@ -35,7 +35,6 @@ import dagger.Lazy;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule.ApplicationDefaultCredential;
import google.registry.config.CredentialModule.DefaultCredential;
import google.registry.config.CredentialModule.LocalCredential;
import google.registry.config.CredentialModule.LocalCredentialJson;
import google.registry.config.RegistryConfig.Config;
@@ -222,10 +221,6 @@ public class AuthModule {
@Module
abstract static class LocalCredentialModule {
@Binds
@DefaultCredential
abstract GoogleCredentialsBundle provideLocalCredentialAsDefaultCredential(
@LocalCredential GoogleCredentialsBundle credential);
@Binds
@ApplicationDefaultCredential
@@ -21,6 +21,7 @@ import com.beust.jcommander.Parameter;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableSet;
import google.registry.tools.params.NameserversParameter;
import google.registry.tools.params.StringListParameter;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
@@ -53,15 +54,15 @@ abstract class CreateOrUpdateDomainCommand extends MutatingEppToolCommand {
String registrant;
@Parameter(
names = {"-a", "--admins"},
description = "Comma-separated list of admin contacts."
)
names = {"-a", "--admins"},
description = "Comma-separated list of admin contacts.",
listConverter = StringListParameter.class)
List<String> admins = new ArrayList<>();
@Parameter(
names = {"-t", "--techs"},
description = "Comma-separated list of technical contacts."
)
names = {"-t", "--techs"},
description = "Comma-separated list of technical contacts.",
listConverter = StringListParameter.class)
List<String> techs = new ArrayList<>();
@Parameter(
@@ -17,6 +17,8 @@ package google.registry.tools;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Predicates.isNull;
import static com.google.common.base.Strings.isNullOrEmpty;
import static com.google.common.base.Verify.verify;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.util.DomainNameUtils.canonicalizeHostname;
import static google.registry.util.RegistrarUtils.normalizeRegistrarName;
@@ -34,6 +36,7 @@ import google.registry.tools.params.OptionalLongParameter;
import google.registry.tools.params.OptionalPhoneNumberParameter;
import google.registry.tools.params.OptionalStringParameter;
import google.registry.tools.params.PathParameter;
import google.registry.tools.params.StringListParameter;
import google.registry.util.CidrAddressBlock;
import java.nio.file.Files;
import java.nio.file.Path;
@@ -69,12 +72,14 @@ abstract class CreateOrUpdateRegistrarCommand extends MutatingCommand {
@Parameter(
names = "--allowed_tlds",
description = "Comma-delimited list of TLDs which the registrar is allowed to use")
description = "Comma-delimited list of TLDs which the registrar is allowed to use",
listConverter = StringListParameter.class)
List<String> allowedTlds = new ArrayList<>();
@Parameter(
names = "--add_allowed_tlds",
description = "Comma-delimited list of TLDs to add to TLDs a registrar is allowed to use")
description = "Comma-delimited list of TLDs to add to TLDs a registrar is allowed to use",
listConverter = StringListParameter.class)
List<String> addAllowedTlds = new ArrayList<>();
@Nullable
@@ -130,6 +135,13 @@ abstract class CreateOrUpdateRegistrarCommand extends MutatingCommand {
validateWith = PathParameter.InputFile.class)
Path clientCertificateFilename;
@Parameter(
names = "--rotate_primary_cert",
description =
"Used together with --cert_file when updating an registrar. "
+ "If set, current cert is saved as failover.")
private Boolean rotatePrimaryCert = Boolean.FALSE;
@Nullable
@Parameter(
names = "--failover_cert_file",
@@ -139,8 +151,9 @@ abstract class CreateOrUpdateRegistrarCommand extends MutatingCommand {
@Parameter(
names = "--ip_allow_list",
description = "Comma-delimited list of IP ranges. An empty string clears the allow list.")
List<String> ipAllowList = new ArrayList<>();
description = "Comma-delimited list of IP ranges. An empty string clears the allow list.",
listConverter = StringListParameter.class)
List<String> ipAllowList;
@Nullable
@Parameter(
@@ -248,7 +261,8 @@ abstract class CreateOrUpdateRegistrarCommand extends MutatingCommand {
description =
"Comma-delimited list of RDAP servers. An empty argument clears the list."
+ " Note that for real registrars this could get overridden periodically by"
+ " ICANN-registered values.")
+ " ICANN-registered values.",
listConverter = StringListParameter.class)
List<String> rdapServers = new ArrayList<>();
/** Returns the existing registrar (for update) or null (for creates). */
@@ -322,16 +336,9 @@ abstract class CreateOrUpdateRegistrarCommand extends MutatingCommand {
}
builder.setAllowedTlds(allowedTldsBuilder.build());
}
if (!ipAllowList.isEmpty()) {
ImmutableList.Builder<CidrAddressBlock> ipAllowListBuilder = new ImmutableList.Builder<>();
if (!(ipAllowList.size() == 1 && ipAllowList.get(0).contains("null"))) {
for (String ipRange : ipAllowList) {
if (!ipRange.isEmpty()) {
ipAllowListBuilder.add(CidrAddressBlock.create(ipRange));
}
}
}
builder.setIpAddressAllowList(ipAllowListBuilder.build());
if (ipAllowList != null) {
builder.setIpAddressAllowList(
ipAllowList.stream().map(CidrAddressBlock::create).collect(toImmutableList()));
}
if (clientCertificateFilename != null) {
String asciiCert = new String(Files.readAllBytes(clientCertificateFilename), US_ASCII);
@@ -341,8 +348,20 @@ abstract class CreateOrUpdateRegistrarCommand extends MutatingCommand {
if (!asciiCert.equals("")) {
certificateChecker.validateCertificate(asciiCert);
}
if (rotatePrimaryCert) {
verify(
oldRegistrar != null,
"--rotate_primary_cert cannot be set by create_registrar command.");
verify(
oldRegistrar.getClientCertificate().isPresent(),
"Primary cert is absent. Rotation may remove a failover certificate still in use.");
builder.setFailoverClientCertificate(oldRegistrar.getClientCertificate().get(), now);
}
builder.setClientCertificate(asciiCert, now);
}
if (rotatePrimaryCert && clientCertificateFilename == null) {
throw new IllegalArgumentException("--rotate_primary_cert must be used with --cert_file.");
}
if (failoverClientCertificateFilename != null) {
String asciiCert =
@@ -20,6 +20,7 @@ import google.registry.model.tld.label.ReservedList;
import google.registry.model.tld.label.ReservedListDao;
import google.registry.tools.params.PathParameter;
import java.nio.file.Path;
import java.util.stream.Collectors;
import javax.annotation.Nullable;
/**
@@ -69,4 +70,12 @@ public abstract class CreateOrUpdateReservedListCommand extends ConfirmingComman
}
return message;
}
String outputReservedListEntries(ReservedList rl) {
return "["
+ rl.getReservedListEntries().values().stream()
.map(rle -> String.format("(%s)", rle.toString()))
.collect(Collectors.joining(", "))
+ "]";
}
}
@@ -18,7 +18,6 @@ import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.tools.UpdateOrDeleteAllocationTokensCommand.getTokenKeys;
import static google.registry.util.CollectionUtils.findDuplicates;
import static google.registry.util.CollectionUtils.isNullOrEmpty;
import static google.registry.util.DomainNameUtils.canonicalizeHostname;
import com.beust.jcommander.Parameter;
@@ -37,6 +36,7 @@ import google.registry.model.tld.label.PremiumList;
import google.registry.model.tld.label.PremiumListDao;
import google.registry.tldconfig.idn.IdnTableEnum;
import google.registry.tools.params.OptionalStringParameter;
import google.registry.tools.params.StringListParameter;
import google.registry.tools.params.TransitionListParameter.BillingCostTransitions;
import google.registry.tools.params.TransitionListParameter.TldStateTransitions;
import java.util.Arrays;
@@ -195,19 +195,22 @@ abstract class CreateOrUpdateTldCommand extends MutatingCommand {
@Nullable
@Parameter(
names = "--reserved_lists",
description = "A comma-separated list of reserved list names to be applied to the TLD")
description = "A comma-separated list of reserved list names to be applied to the TLD",
listConverter = StringListParameter.class)
List<String> reservedListNames;
@Nullable
@Parameter(
names = "--allowed_registrants",
description = "A comma-separated list of allowed registrants for the TLD")
description = "A comma-separated list of allowed registrants for the TLD",
listConverter = StringListParameter.class)
List<String> allowedRegistrants;
@Nullable
@Parameter(
names = "--allowed_nameservers",
description = "A comma-separated list of allowed nameservers for the TLD")
description = "A comma-separated list of allowed nameservers for the TLD",
listConverter = StringListParameter.class)
List<String> allowedNameservers;
@Parameter(
@@ -223,8 +226,9 @@ abstract class CreateOrUpdateTldCommand extends MutatingCommand {
@Nullable
@Parameter(
names = "--dns_writers",
description = "A comma-separated list of DnsWriter implementations to use")
names = "--dns_writers",
description = "A comma-separated list of DnsWriter implementations to use",
listConverter = StringListParameter.class)
List<String> dnsWriters;
@Nullable
@@ -262,7 +266,8 @@ abstract class CreateOrUpdateTldCommand extends MutatingCommand {
"A comma-separated list of default allocation tokens to be applied to the TLD. The"
+ " ordering of this list will determine which token is used in the case where"
+ " multiple tokens are valid for a registration. Use an empty string to clear all"
+ " present default tokens.")
+ " present default tokens.",
listConverter = StringListParameter.class)
List<String> defaultTokens;
@Nullable
@@ -271,7 +276,8 @@ abstract class CreateOrUpdateTldCommand extends MutatingCommand {
description =
"A comma-separated list of the IDN tables to use for this TLD. Specify an empty list to"
+ " remove any previously-set tables and to use the default. All elements must be"
+ " IdnTableEnum values")
+ " IdnTableEnum values",
listConverter = StringListParameter.class)
List<String> idnTables;
/** Returns the existing tld (for update) or null (for creates). */
@@ -420,12 +426,8 @@ abstract class CreateOrUpdateTldCommand extends MutatingCommand {
builder.setAllowedFullyQualifiedHostNames(getAllowedNameservers(oldTld));
if (!isNullOrEmpty(defaultTokens)) {
if (defaultTokens.equals(ImmutableList.of(""))) {
builder.setDefaultPromoTokens(ImmutableList.of());
} else {
builder.setDefaultPromoTokens(getTokenKeys(defaultTokens, null));
}
if (defaultTokens != null) {
builder.setDefaultPromoTokens(getTokenKeys(defaultTokens, null));
}
if (idnTables != null) {
if (idnTables.equals(ImmutableList.of(""))) {
@@ -28,7 +28,6 @@ import com.google.common.base.Strings;
import google.registry.model.tld.label.ReservedList;
import java.nio.file.Files;
import java.util.List;
import java.util.stream.Collectors;
import org.joda.time.DateTime;
/** Command to create a {@link ReservedList}. */
@@ -64,11 +63,8 @@ final class CreateReservedListCommand extends CreateOrUpdateReservedListCommand
.setCreationTimestamp(now)
.build();
String entries =
reservedList.getReservedListEntries().entrySet().stream()
.map(entry -> String.format("%s=%s", entry.getKey(), entry.getValue()))
.collect(Collectors.joining(", "));
return String.format("%s\nreservedListMap={%s}\n", reservedList, entries);
return String.format(
"%s\nreservedListMap=%s\n", reservedList, outputReservedListEntries(reservedList));
}
private static void validateListName(String name) {
@@ -26,8 +26,8 @@ import com.google.common.collect.Sets;
import google.registry.batch.CloudTasksUtils;
import google.registry.batch.RelockDomainAction;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.billing.BillingBase.Reason;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.RegistryLock;
@@ -397,8 +397,8 @@ public final class DomainLockUtils {
tm().update(domain);
tm().insert(domainHistory);
if (!lock.isSuperuser()) { // admin actions shouldn't affect billing
BillingEvent.OneTime oneTime =
new BillingEvent.OneTime.Builder()
BillingEvent billingEvent =
new BillingEvent.Builder()
.setReason(Reason.SERVER_STATUS)
.setTargetId(domain.getForeignKey())
.setRegistrarId(domain.getCurrentSponsorRegistrarId())
@@ -407,7 +407,7 @@ public final class DomainLockUtils {
.setBillingTime(now)
.setDomainHistory(domainHistory)
.build();
tm().insert(oneTime);
tm().insert(billingEvent);
}
}
}
@@ -17,7 +17,8 @@ package google.registry.tools;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.collect.Sets.difference;
import static google.registry.model.billing.BillingEvent.RenewalPriceBehavior.DEFAULT;
import static google.registry.model.billing.BillingBase.RenewalPriceBehavior.DEFAULT;
import static google.registry.model.domain.token.AllocationToken.TokenType.DEFAULT_PROMO;
import static google.registry.model.domain.token.AllocationToken.TokenType.PACKAGE;
import static google.registry.model.domain.token.AllocationToken.TokenType.SINGLE_USE;
import static google.registry.model.domain.token.AllocationToken.TokenType.UNLIMITED_USE;
@@ -38,7 +39,7 @@ import com.google.common.collect.ImmutableSortedMap;
import com.google.common.collect.Iterables;
import com.google.common.collect.Streams;
import com.google.common.io.Files;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.domain.fee.FeeQueryCommandExtensionItem.CommandName;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.domain.token.AllocationToken.RegistrationBehavior;
@@ -259,6 +260,10 @@ class GenerateAllocationTokensCommand implements Command {
!(UNLIMITED_USE.equals(tokenType) && CollectionUtils.isNullOrEmpty(tokenStatusTransitions)),
"For UNLIMITED_USE tokens, must specify --token_status_transitions");
checkArgument(
!(DEFAULT_PROMO.equals(tokenType) && CollectionUtils.isNullOrEmpty(tokenStatusTransitions)),
"For DEFAULT_PROMO tokens, must specify --token_status_transitions");
// A list consisting solely of the empty string means user error when formatting parameters
checkArgument(
!ImmutableList.of("").equals(allowedClientIds),
@@ -1,34 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.tools;
import com.beust.jcommander.Parameters;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.common.DatabaseMigrationStateSchedule;
import google.registry.model.common.DatabaseMigrationStateSchedule.MigrationState;
import google.registry.model.common.TimedTransitionProperty;
/** A command to check the current Registry 3.0 migration state of the database. */
@DeleteAfterMigration
@Parameters(separators = " =", commandDescription = "Check current Registry 3.0 migration state")
public class GetDatabaseMigrationStateCommand implements Command {
@Override
public void run() throws Exception {
TimedTransitionProperty<MigrationState> migrationSchedule =
DatabaseMigrationStateSchedule.get();
System.out.printf("Current migration schedule: %s%n", migrationSchedule.toValueMap());
}
}
@@ -15,17 +15,23 @@
package google.registry.tools;
import static google.registry.model.EppResourceUtils.loadByForeignKey;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.beust.jcommander.Parameter;
import com.beust.jcommander.Parameters;
import google.registry.model.domain.Domain;
import google.registry.persistence.transaction.QueryComposer.Comparator;
import google.registry.util.DomainNameUtils;
import java.util.List;
import java.util.Optional;
/** Command to show a domain resource. */
@Parameters(separators = " =", commandDescription = "Show domain resource(s)")
final class GetDomainCommand extends GetEppResourceCommand {
@Parameter(names = "--show_deleted", description = "Include deleted domains in the print out")
private boolean showDeleted = false;
@Parameter(
description = "Fully qualified domain name(s)",
required = true)
@@ -35,10 +41,24 @@ final class GetDomainCommand extends GetEppResourceCommand {
public void runAndPrint() {
for (String domainName : mainParameters) {
String canonicalDomain = DomainNameUtils.canonicalizeHostname(domainName);
printResource(
"Domain",
canonicalDomain,
loadByForeignKey(Domain.class, canonicalDomain, readTimestamp));
if (showDeleted) {
tm().transact(
() ->
tm()
.createQueryComposer(Domain.class)
.where("domainName", Comparator.EQ, canonicalDomain)
.orderBy("creationTime")
.stream()
.forEach(
d -> {
printResource("Domain", canonicalDomain, Optional.of(d));
}));
} else {
printResource(
"Domain",
canonicalDomain,
loadByForeignKey(Domain.class, canonicalDomain, readTimestamp));
}
}
}
}
@@ -32,6 +32,7 @@ import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.tools.params.OptionalPhoneNumberParameter;
import google.registry.tools.params.PathParameter;
import google.registry.tools.params.StringListParameter;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
@@ -72,8 +73,10 @@ final class RegistrarPocCommand extends MutatingCommand {
@Nullable
@Parameter(
names = "--contact_type",
description = "Type of communications for this contact; separate multiple with commas."
+ " Allowed values are ABUSE, ADMIN, BILLING, LEGAL, MARKETING, TECH, WHOIS.")
description =
"Type of communications for this contact; separate multiple with commas."
+ " Allowed values are ABUSE, ADMIN, BILLING, LEGAL, MARKETING, TECH, WHOIS.",
listConverter = StringListParameter.class)
private List<String> contactTypeNames;
@Nullable
@@ -177,15 +180,6 @@ final class RegistrarPocCommand extends MutatingCommand {
// If the contact_type parameter is not specified, we should not make any changes.
if (contactTypeNames == null) {
contactTypes = null;
// It appears that when the user specifies "--contact_type=" with no types following,
// JCommander sets contactTypeNames to a one-element list containing the empty string. This is
// strange, but we need to handle this by setting the contact types to the empty set. Also do
// this if contactTypeNames is empty, which is what I would hope JCommander would return in
// some future, better world.
} else //noinspection UnnecessaryParentheses
if (contactTypeNames.isEmpty()
|| (contactTypeNames.size() == 1 && contactTypeNames.get(0).isEmpty())) {
contactTypes = ImmutableSet.of();
} else {
contactTypes =
contactTypeNames.stream()
@@ -15,7 +15,7 @@
package google.registry.tools;
import com.google.common.collect.ImmutableMap;
import google.registry.tools.javascrap.CreateCancellationsForOneTimesCommand;
import google.registry.tools.javascrap.CreateCancellationsForBillingEventsCommand;
/** Container class to create and run remote commands against a server instance. */
public final class RegistryTool {
@@ -35,7 +35,9 @@ public final class RegistryTool {
.put("convert_idn", ConvertIdnCommand.class)
.put("count_domains", CountDomainsCommand.class)
.put("create_anchor_tenant", CreateAnchorTenantCommand.class)
.put("create_cancellations_for_one_times", CreateCancellationsForOneTimesCommand.class)
.put(
"create_cancellations_for_billing_events",
CreateCancellationsForBillingEventsCommand.class)
.put("create_cdns_tld", CreateCdnsTld.class)
.put("create_contact", CreateContactCommand.class)
.put("create_domain", CreateDomainCommand.class)
@@ -66,7 +68,6 @@ public final class RegistryTool {
.put("get_allocation_token", GetAllocationTokenCommand.class)
.put("get_claims_list", GetClaimsListCommand.class)
.put("get_contact", GetContactCommand.class)
.put("get_database_migration_state", GetDatabaseMigrationStateCommand.class)
.put("get_domain", GetDomainCommand.class)
.put("get_history_entries", GetHistoryEntriesCommand.class)
.put("get_host", GetHostCommand.class)
@@ -96,8 +97,6 @@ public final class RegistryTool {
.put("renew_domain", RenewDomainCommand.class)
.put("save_sql_credential", SaveSqlCredentialCommand.class)
.put("send_escrow_report_to_icann", SendEscrowReportToIcannCommand.class)
.put("set_database_migration_state", SetDatabaseMigrationStateCommand.class)
.put("set_num_instances", SetNumInstancesCommand.class)
.put("setup_ote", SetupOteCommand.class)
.put("uniform_rapid_suspension", UniformRapidSuspensionCommand.class)
.put("unlock_domain", UnlockDomainCommand.class)
@@ -40,7 +40,6 @@ import google.registry.request.Modules.UrlConnectionServiceModule;
import google.registry.request.Modules.UrlFetchServiceModule;
import google.registry.request.Modules.UserServiceModule;
import google.registry.tools.AuthModule.LocalCredentialModule;
import google.registry.tools.javascrap.CreateCancellationsForOneTimesCommand;
import google.registry.util.UtilsModule;
import google.registry.whois.NonCachingWhoisModule;
import javax.annotation.Nullable;
@@ -92,8 +91,6 @@ interface RegistryToolComponent {
void inject(CreateAnchorTenantCommand command);
void inject(CreateCancellationsForOneTimesCommand command);
void inject(CreateCdnsTld command);
void inject(CreateContactCommand command);
@@ -144,8 +141,6 @@ interface RegistryToolComponent {
void inject(SendEscrowReportToIcannCommand command);
void inject(SetNumInstancesCommand command);
void inject(SetupOteCommand command);
void inject(UniformRapidSuspensionCommand command);
@@ -24,7 +24,7 @@ import com.google.api.client.util.GenericData;
import com.google.auth.oauth2.UserCredentials;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule.DefaultCredential;
import google.registry.config.CredentialModule.ApplicationDefaultCredential;
import google.registry.config.RegistryConfig;
import google.registry.config.RegistryConfig.Config;
import google.registry.util.GoogleCredentialsBundle;
@@ -56,7 +56,7 @@ class RequestFactoryModule {
@Provides
static HttpRequestFactory provideHttpRequestFactory(
@DefaultCredential GoogleCredentialsBundle credentialsBundle,
@ApplicationDefaultCredential GoogleCredentialsBundle credentialsBundle,
@Config("iapClientId") Optional<String> iapClientId) {
if (RegistryConfig.areServersLocal()) {
return new NetHttpTransport()
@@ -85,7 +85,7 @@ public class ServiceConnection {
private String internalSend(
String endpoint, Map<String, ?> params, MediaType contentType, @Nullable byte[] payload)
throws IOException {
GenericUrl url = new GenericUrl(String.format("%s%s", getServer(), endpoint));
GenericUrl url = new GenericUrl(String.format("%s%s", getServer(service), endpoint));
url.putAll(params);
HttpRequest request =
(payload != null)
@@ -141,7 +141,7 @@ public class ServiceConnection {
return (Map<String, Object>) JSONValue.parse(response.substring(JSON_SAFETY_PREFIX.length()));
}
public URL getServer() {
public static URL getServer(Service service) {
switch (service) {
case DEFAULT:
return RegistryConfig.getDefaultServer();
@@ -1,70 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.tools;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.beust.jcommander.Parameter;
import com.beust.jcommander.Parameters;
import com.google.common.collect.ImmutableSortedMap;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.common.DatabaseMigrationStateSchedule;
import google.registry.model.common.DatabaseMigrationStateSchedule.MigrationState;
import google.registry.tools.params.TransitionListParameter.MigrationStateTransitions;
import org.joda.time.DateTime;
/** Command to set the Registry 3.0 database migration state schedule. */
@DeleteAfterMigration
@Parameters(
separators = " =",
commandDescription = "Set the current database migration state schedule.")
public class SetDatabaseMigrationStateCommand extends ConfirmingCommand {
private static final String WARNING_MESSAGE =
"Attempting to change the schedule with an effect that would take place within the next 10 "
+ "minutes. The cache expiration duration is 5 minutes so this MAY BE DANGEROUS.\n";
@Parameter(
names = "--migration_schedule",
converter = MigrationStateTransitions.class,
validateWith = MigrationStateTransitions.class,
required = true,
description =
"Comma-delimited list of database transitions, of the form"
+ " <time>=<migration-state>[,<time>=<migration-state>]*")
ImmutableSortedMap<DateTime, MigrationState> transitionSchedule;
@Override
protected String prompt() {
return tm().transact(
() -> {
StringBuilder result = new StringBuilder();
DateTime now = tm().getTransactionTime();
DateTime nextTransition = transitionSchedule.ceilingKey(now);
if (nextTransition != null && nextTransition.isBefore(now.plusMinutes(10))) {
result.append(WARNING_MESSAGE);
}
return result
.append(String.format("Set new migration state schedule %s?", transitionSchedule))
.toString();
});
}
@Override
protected String execute() {
tm().transact(() -> DatabaseMigrationStateSchedule.set(transitionSchedule));
return String.format("Successfully set new migration state schedule %s", transitionSchedule);
}
}
@@ -1,172 +0,0 @@
// Copyright 2018 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.tools;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSetMultimap.flatteningToImmutableSetMultimap;
import static google.registry.util.CollectionUtils.nullToEmpty;
import com.beust.jcommander.Parameter;
import com.beust.jcommander.Parameters;
import com.google.api.services.appengine.v1.Appengine;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSetMultimap;
import com.google.common.collect.Maps;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryConfig.Config;
import google.registry.request.Action.Service;
import google.registry.util.AppEngineServiceUtils;
import java.io.IOException;
import java.util.List;
import javax.inject.Inject;
/** A command to set the number of instances for an App Engine service. */
@Parameters(
separators = " =",
commandDescription =
"Set the number of instances for a given service and version. "
+ "Note that this command only works for manual scaling service.")
final class SetNumInstancesCommand implements Command {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final ImmutableList<Service> ALL_DEPLOYED_SERVICES =
ImmutableList.copyOf(Service.values());
private static final ImmutableMap<String, Service> SERVICE_ID_TO_SERVICE =
Maps.uniqueIndex(ALL_DEPLOYED_SERVICES, Service::getServiceId);
@Parameter(
names = {"-s", "--services"},
description =
"Comma-delimited list of App Engine services to set. "
+ "Allowed values: [DEFAULT, TOOLS, BACKEND, PUBAPI]")
private List<Service> services = ImmutableList.of();
@Parameter(
names = {"-v", "--versions"},
description =
"Comma-delimited list of App Engine versions to set, e.g., canary. "
+ "Cannot be set if --non_live_versions is set.")
private List<String> versions = ImmutableList.of();
@Parameter(
names = {"-n", "--num_instances"},
description =
"The new number of instances for the given versions "
+ "or for all non-live versions if --non_live_versions is set.",
required = true)
private Long numInstances;
@Parameter(
names = "--non_live_versions",
description = "Whether to set number of instances for all non-live versions.",
arity = 1)
private Boolean nonLiveVersions = false;
@Inject AppEngineServiceUtils appEngineServiceUtils;
@Inject Appengine appengine;
@Inject
@Config("projectId")
String projectId;
@Override
public void run() {
if (nonLiveVersions) {
checkArgument(versions.isEmpty(), "--versions cannot be set if --non_live_versions is set");
services = services.isEmpty() ? ALL_DEPLOYED_SERVICES : services;
ImmutableSetMultimap<Service, String> allLiveVersionsMap = getAllLiveVersionsMap(services);
ImmutableSetMultimap<Service, String> manualScalingVersionsMap =
getManualScalingVersionsMap(services);
// Set number of instances for versions which are manual scaling and non-live
manualScalingVersionsMap.forEach(
(service, versionId) -> {
if (!allLiveVersionsMap.containsEntry(service, versionId)) {
setNumInstances(service, versionId, numInstances);
}
});
} else {
checkArgument(!services.isEmpty(), "Service must be specified");
checkArgument(!versions.isEmpty(), "Version must be specified");
checkArgument(numInstances > 0, "Number of instances must be greater than zero");
ImmutableSetMultimap<Service, String> manualScalingVersionsMap =
getManualScalingVersionsMap(services);
for (Service service : services) {
for (String versionId : versions) {
checkArgument(
manualScalingVersionsMap.containsEntry(service, versionId),
"Version %s of service %s is not managed through manual scaling",
versionId,
service);
setNumInstances(service, versionId, numInstances);
}
}
}
}
private void setNumInstances(Service service, String version, long numInstances) {
appEngineServiceUtils.setNumInstances(service.getServiceId(), version, numInstances);
logger.atInfo().log(
"Successfully set version %s of service %s to %d instances.",
version, service, numInstances);
}
private ImmutableSetMultimap<Service, String> getAllLiveVersionsMap(List<Service> services) {
try {
return nullToEmpty(appengine.apps().services().list(projectId).execute().getServices())
.stream()
.filter(
service ->
services.contains(SERVICE_ID_TO_SERVICE.getOrDefault(service.getId(), null)))
.collect(
flatteningToImmutableSetMultimap(
service -> SERVICE_ID_TO_SERVICE.get(service.getId()),
service -> nullToEmpty(service.getSplit().getAllocations()).keySet().stream()));
} catch (IOException e) {
throw new RuntimeException(e);
}
}
private ImmutableSetMultimap<Service, String> getManualScalingVersionsMap(
List<Service> services) {
return services.stream()
.collect(
flatteningToImmutableSetMultimap(
service -> service,
service -> {
try {
return nullToEmpty(
appengine
.apps()
.services()
.versions()
.list(projectId, service.getServiceId())
.execute()
.getVersions())
.stream()
.filter(version -> version.getManualScaling() != null)
.map(version -> version.getId());
} catch (IOException e) {
throw new RuntimeException(e);
}
}));
}
}
@@ -32,8 +32,7 @@ import com.google.common.collect.ImmutableMultimap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Sets;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.Period;
@@ -205,7 +204,7 @@ class UnrenewDomainCommand extends ConfirmingCommand {
.setEventTime(now)
.build();
// Create a new autorenew billing event and poll message starting at the new expiration time.
BillingEvent.Recurring newAutorenewEvent =
BillingRecurrence newAutorenewEvent =
newAutorenewBillingEvent(domain)
.setEventTime(newExpirationTime)
.setDomainHistory(domainHistory)
@@ -216,9 +215,9 @@ class UnrenewDomainCommand extends ConfirmingCommand {
.setHistoryEntry(domainHistory)
.build();
// End the old autorenew billing event and poll message now.
Recurring existingRecurring = tm().loadByKey(domain.getAutorenewBillingEvent());
BillingRecurrence existingBillingRecurrence = tm().loadByKey(domain.getAutorenewBillingEvent());
updateAutorenewRecurrenceEndTime(
domain, existingRecurring, now, domainHistory.getHistoryEntryId());
domain, existingBillingRecurrence, now, domainHistory.getHistoryEntryId());
Domain newDomain =
domain
.asBuilder()
@@ -28,12 +28,13 @@ import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedMap;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.domain.fee.FeeQueryCommandExtensionItem.CommandName;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.domain.token.AllocationToken.RegistrationBehavior;
import google.registry.model.domain.token.AllocationToken.TokenStatus;
import google.registry.model.domain.token.AllocationToken.TokenType;
import google.registry.tools.params.StringListParameter;
import google.registry.tools.params.TransitionListParameter.TokenStatusTransitions;
import java.util.List;
import java.util.Map;
@@ -54,21 +55,24 @@ final class UpdateAllocationTokensCommand extends UpdateOrDeleteAllocationTokens
names = {"--allowed_client_ids"},
description =
"Comma-separated list of allowed client IDs. Use the empty string to clear the "
+ "existing list.")
+ "existing list.",
listConverter = StringListParameter.class)
private List<String> allowedClientIds;
@Parameter(
names = {"--allowed_tlds"},
description =
"Comma-separated list of allowed TLDs. Use the empty string to clear the "
+ "existing list.")
+ "existing list.",
listConverter = StringListParameter.class)
private List<String> allowedTlds;
@Parameter(
names = {"--allowed_epp_actions"},
description =
"Comma-separated list of allowed EPP actions. Use an empty string to clear the existing"
+ " list.")
+ " list.",
listConverter = StringListParameter.class)
private List<String> allowedEppActions;
@Parameter(
@@ -128,18 +132,6 @@ final class UpdateAllocationTokensCommand extends UpdateOrDeleteAllocationTokens
@Override
public void init() {
// A single entry with the empty string means that the user passed an empty argument to the
// lists, so we should clear them
if (ImmutableList.of("").equals(allowedClientIds)) {
allowedClientIds = ImmutableList.of();
}
if (ImmutableList.of("").equals(allowedTlds)) {
allowedTlds = ImmutableList.of();
}
if (ImmutableList.of("").equals(allowedEppActions)) {
allowedEppActions = ImmutableList.of();
}
if (tokenStatusTransitions != null
&& (tokenStatusTransitions.containsValue(TokenStatus.ENDED)
|| tokenStatusTransitions.containsValue(TokenStatus.CANCELLED))) {
@@ -190,7 +190,7 @@ final class UpdateDomainCommand extends CreateOrUpdateDomainCommand {
checkArgument(
!domain.getStatusValues().contains(SERVER_UPDATE_PROHIBITED),
"The domain '%s' has status SERVER_UPDATE_PROHIBITED. Verify that you are allowed "
+ "to make updates, and if so, use the domain_unlock command to enable updates.",
+ "to make updates, and if so, use the unlock_domain command to enable updates.",
domainName);
checkArgument(
!domain.getStatusValues().contains(PENDING_DELETE) || forceInPendingDelete,
@@ -25,8 +25,8 @@ import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import google.registry.model.EppResourceUtils;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.billing.BillingBase.RenewalPriceBehavior;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.reporting.HistoryEntry;
@@ -38,7 +38,7 @@ import org.joda.money.Money;
import org.joda.time.DateTime;
/**
* Command to update {@link Recurring} billing events with a new behavior and/or price.
* Command to update {@link BillingRecurrence} billing events with a new behavior and/or price.
*
* <p>More specifically, this closes the existing recurrence object and creates a new, similar,
* object as well as a corresponding synthetic {@link DomainHistory} object. This is done to
@@ -85,11 +85,11 @@ public class UpdateRecurrenceCommand extends ConfirmingCommand {
+ " SPECIFIED");
}
}
ImmutableMap<Domain, Recurring> domainsAndRecurrings =
tm().transact(this::loadDomainsAndRecurrings);
ImmutableMap<Domain, BillingRecurrence> domainsAndRecurrences =
tm().transact(this::loadDomainsAndRecurrences);
if (renewalPriceBehavior == null) {
// Allow users to specify only a price only if all renewals are already SPECIFIED
domainsAndRecurrings.forEach(
domainsAndRecurrences.forEach(
(d, r) ->
checkArgument(
r.getRenewalPriceBehavior().equals(RenewalPriceBehavior.SPECIFIED),
@@ -103,41 +103,45 @@ public class UpdateRecurrenceCommand extends ConfirmingCommand {
"Update the following with behavior %s%s?\n%s",
renewalPriceBehavior,
specifiedPriceString,
Joiner.on('\n').withKeyValueSeparator(':').join(domainsAndRecurrings));
Joiner.on('\n').withKeyValueSeparator(':').join(domainsAndRecurrences));
}
@Override
protected String execute() throws Exception {
ImmutableList<Recurring> newRecurrings = tm().transact(this::internalExecute);
return "Updated new recurring(s): " + newRecurrings;
ImmutableList<BillingRecurrence> newBillingRecurrences = tm().transact(this::internalExecute);
return "Updated new recurrence(s): " + newBillingRecurrences;
}
private ImmutableList<Recurring> internalExecute() {
ImmutableMap<Domain, Recurring> domainsAndRecurrings = loadDomainsAndRecurrings();
private ImmutableList<BillingRecurrence> internalExecute() {
ImmutableMap<Domain, BillingRecurrence> domainsAndRecurrences = loadDomainsAndRecurrences();
DateTime now = tm().getTransactionTime();
ImmutableList.Builder<Recurring> resultBuilder = new ImmutableList.Builder<>();
domainsAndRecurrings.forEach(
(domain, existingRecurring) -> {
// Make a new history ID to break the (recurring, history, domain) circular dep chain
ImmutableList.Builder<BillingRecurrence> resultBuilder = new ImmutableList.Builder<>();
domainsAndRecurrences.forEach(
(domain, existingRecurrence) -> {
// Make a new history ID to break the (recurrence, history, domain) circular dep chain
long newHistoryId = allocateId();
HistoryEntryId newDomainHistoryId = new HistoryEntryId(domain.getRepoId(), newHistoryId);
Recurring endingNow = existingRecurring.asBuilder().setRecurrenceEndTime(now).build();
Recurring.Builder newRecurringBuilder =
existingRecurring
BillingRecurrence endingNow =
existingRecurrence.asBuilder().setRecurrenceEndTime(now).build();
BillingRecurrence.Builder newRecurrenceBuilder =
existingRecurrence
.asBuilder()
// set the ID to be 0 (null) to create a new object
.setId(0)
.setDomainHistoryId(newDomainHistoryId);
if (renewalPriceBehavior != null) {
newRecurringBuilder.setRenewalPriceBehavior(renewalPriceBehavior);
newRecurringBuilder.setRenewalPrice(null);
newRecurrenceBuilder.setRenewalPriceBehavior(renewalPriceBehavior);
newRecurrenceBuilder.setRenewalPrice(null);
}
if (specifiedRenewalPrice != null) {
newRecurringBuilder.setRenewalPrice(specifiedRenewalPrice);
newRecurrenceBuilder.setRenewalPrice(specifiedRenewalPrice);
}
Recurring newRecurring = newRecurringBuilder.build();
BillingRecurrence newBillingRecurrence = newRecurrenceBuilder.build();
Domain newDomain =
domain.asBuilder().setAutorenewBillingEvent(newRecurring.createVKey()).build();
domain
.asBuilder()
.setAutorenewBillingEvent(newBillingRecurrence.createVKey())
.build();
DomainHistory newDomainHistory =
new DomainHistory.Builder()
.setRevisionId(newDomainHistoryId.getRevisionId())
@@ -149,14 +153,14 @@ public class UpdateRecurrenceCommand extends ConfirmingCommand {
.setType(HistoryEntry.Type.SYNTHETIC)
.setModificationTime(now)
.build();
tm().putAll(endingNow, newRecurring, newDomain, newDomainHistory);
resultBuilder.add(newRecurring);
tm().putAll(endingNow, newBillingRecurrence, newDomain, newDomainHistory);
resultBuilder.add(newBillingRecurrence);
});
return resultBuilder.build();
}
private ImmutableMap<Domain, Recurring> loadDomainsAndRecurrings() {
ImmutableMap.Builder<Domain, Recurring> result = new ImmutableMap.Builder<>();
private ImmutableMap<Domain, BillingRecurrence> loadDomainsAndRecurrences() {
ImmutableMap.Builder<Domain, BillingRecurrence> result = new ImmutableMap.Builder<>();
DateTime now = tm().getTransactionTime();
for (String domainName : mainParameters) {
Domain domain =
@@ -183,12 +187,12 @@ public class UpdateRecurrenceCommand extends ConfirmingCommand {
"Domain %s autorenew ended prior to now at %s",
domainName,
endTime));
Recurring recurring = tm().loadByKey(domain.getAutorenewBillingEvent());
BillingRecurrence billingRecurrence = tm().loadByKey(domain.getAutorenewBillingEvent());
checkArgument(
recurring.getRecurrenceEndTime().equals(END_OF_TIME),
billingRecurrence.getRecurrenceEndTime().equals(END_OF_TIME),
"Domain %s's recurrence's end date is not END_OF_TIME",
domainName);
result.put(domain, recurring);
result.put(domain, billingRecurrence);
}
return result.build();
}
@@ -22,7 +22,6 @@ import com.google.common.base.Strings;
import google.registry.model.tld.label.ReservedList;
import java.nio.file.Files;
import java.util.List;
import java.util.stream.Collectors;
/** Command to safely update {@link ReservedList}. */
@Parameters(separators = " =", commandDescription = "Update a ReservedList.")
@@ -52,17 +51,11 @@ final class UpdateReservedListCommand extends CreateOrUpdateReservedListCommand
if (!existingReservedList
.getReservedListEntries()
.equals(reservedList.getReservedListEntries())) {
String oldEntries =
existingReservedList.getReservedListEntries().entrySet().stream()
.map(entry -> String.format("%s=%s", entry.getKey(), entry.getValue()))
.collect(Collectors.joining(", "));
String newEntries =
reservedList.getReservedListEntries().entrySet().stream()
.map(entry -> String.format("%s=%s", entry.getKey(), entry.getValue()))
.collect(Collectors.joining(", "));
return String.format(
"Update reserved list for %s?\nOld List: %s\n New List: %s",
name, oldEntries, newEntries);
"Update reserved list for %s?\nOld list: %s\n New list: %s",
name,
outputReservedListEntries(existingReservedList),
outputReservedListEntries(reservedList));
}
return "No entity changes to apply.";
}
@@ -28,6 +28,7 @@ import com.google.common.collect.Maps;
import google.registry.config.RegistryEnvironment;
import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldState;
import google.registry.tools.params.StringListParameter;
import java.util.List;
import java.util.Map;
import java.util.Optional;
@@ -39,40 +40,47 @@ import org.joda.time.DateTimeZone;
/** Command to update a TLD. */
@Parameters(separators = " =", commandDescription = "Update existing TLD(s)")
public class UpdateTldCommand extends CreateOrUpdateTldCommand {
@Nullable
@Parameter(
names = "--add_reserved_lists",
description = "A comma-separated list of reserved list names to be added to the TLD")
description = "A comma-separated list of reserved list names to be added to the TLD",
listConverter = StringListParameter.class)
List<String> reservedListsAdd;
@Nullable
@Parameter(
names = "--remove_reserved_lists",
description = "A comma-separated list of reserved list names to be removed from the TLD")
description = "A comma-separated list of reserved list names to be removed from the TLD",
listConverter = StringListParameter.class)
List<String> reservedListsRemove;
@Nullable
@Parameter(
names = "--add_allowed_registrants",
description = "A comma-separated list of allowed registrants to be added to the TLD")
description = "A comma-separated list of allowed registrants to be added to the TLD",
listConverter = StringListParameter.class)
List<String> allowedRegistrantsAdd;
@Nullable
@Parameter(
names = "--remove_allowed_registrants",
description = "A comma-separated list of allowed registrants to be removed from the TLD")
description = "A comma-separated list of allowed registrants to be removed from the TLD",
listConverter = StringListParameter.class)
List<String> allowedRegistrantsRemove;
@Nullable
@Parameter(
names = "--add_allowed_nameservers",
description = "A comma-separated list of allowed nameservers to be added to the TLD")
description = "A comma-separated list of allowed nameservers to be added to the TLD",
listConverter = StringListParameter.class)
List<String> allowedNameserversAdd;
@Nullable
@Parameter(
names = "--remove_allowed_nameservers",
description = "A comma-separated list of allowed nameservers to be removed from the TLD")
description = "A comma-separated list of allowed nameservers to be removed from the TLD",
listConverter = StringListParameter.class)
List<String> allowedNameserversRemove;
@Nullable
@@ -19,9 +19,9 @@ import static google.registry.persistence.transaction.TransactionManagerFactory.
import com.beust.jcommander.Parameter;
import com.beust.jcommander.Parameters;
import com.google.common.collect.ImmutableSet;
import google.registry.model.billing.BillingBase;
import google.registry.model.billing.BillingCancellation;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Cancellation;
import google.registry.model.billing.BillingEvent.OneTime;
import google.registry.persistence.VKey;
import google.registry.persistence.transaction.QueryComposer.Comparator;
import google.registry.tools.ConfirmingCommand;
@@ -29,14 +29,16 @@ import google.registry.tools.params.LongParameter;
import java.util.List;
/**
* Command to create {@link Cancellation}s for specified {@link OneTime} billing events.
* Command to create {@link BillingCancellation}s for specified {@link BillingEvent} billing events.
*
* <p>This can be used to fix situations where we've inadvertently billed registrars. It's generally
* easier and better to issue cancellations within the Nomulus system than to attempt to issue
* refunds after the fact.
*/
@Parameters(separators = " =", commandDescription = "Manually create Cancellations for OneTimes.")
public class CreateCancellationsForOneTimesCommand extends ConfirmingCommand {
@Parameters(
separators = " =",
commandDescription = "Manually create Cancellations for BillingEvents.")
public class CreateCancellationsForBillingEventsCommand extends ConfirmingCommand {
@Parameter(
description = "Space-delimited billing event ID(s) to cancel",
@@ -44,80 +46,83 @@ public class CreateCancellationsForOneTimesCommand extends ConfirmingCommand {
validateWith = LongParameter.class)
private List<Long> mainParameters;
private ImmutableSet<OneTime> oneTimesToCancel;
private ImmutableSet<BillingEvent> billingEventsToCancel;
@Override
protected void init() {
ImmutableSet.Builder<Long> missingIdsBuilder = new ImmutableSet.Builder<>();
ImmutableSet.Builder<Long> alreadyCancelledIdsBuilder = new ImmutableSet.Builder<>();
ImmutableSet.Builder<OneTime> oneTimesBuilder = new ImmutableSet.Builder<>();
ImmutableSet.Builder<BillingEvent> billingEventsBuilder = new ImmutableSet.Builder<>();
tm().transact(
() -> {
for (Long billingEventId : ImmutableSet.copyOf(mainParameters)) {
VKey<OneTime> key = VKey.create(OneTime.class, billingEventId);
VKey<BillingEvent> key = VKey.create(BillingEvent.class, billingEventId);
if (tm().exists(key)) {
OneTime oneTime = tm().loadByKey(key);
if (alreadyCancelled(oneTime)) {
BillingEvent billingEvent = tm().loadByKey(key);
if (alreadyCancelled(billingEvent)) {
alreadyCancelledIdsBuilder.add(billingEventId);
} else {
oneTimesBuilder.add(oneTime);
billingEventsBuilder.add(billingEvent);
}
} else {
missingIdsBuilder.add(billingEventId);
}
}
});
oneTimesToCancel = oneTimesBuilder.build();
System.out.printf("Found %d OneTime(s) to cancel\n", oneTimesToCancel.size());
billingEventsToCancel = billingEventsBuilder.build();
System.out.printf("Found %d BillingEvent(s) to cancel\n", billingEventsToCancel.size());
ImmutableSet<Long> missingIds = missingIdsBuilder.build();
if (!missingIds.isEmpty()) {
System.out.printf("Missing OneTime(s) for IDs %s\n", missingIds);
System.out.printf("Missing BillingEvent(s) for IDs %s\n", missingIds);
}
ImmutableSet<Long> alreadyCancelledIds = alreadyCancelledIdsBuilder.build();
if (!alreadyCancelledIds.isEmpty()) {
System.out.printf(
"The following OneTime IDs were already cancelled: %s\n", alreadyCancelledIds);
"The following BillingEvent IDs were already cancelled: %s\n", alreadyCancelledIds);
}
}
@Override
protected String prompt() {
return String.format("Create cancellations for %d OneTime(s)?", oneTimesToCancel.size());
return String.format(
"Create cancellations for %d BillingEvent(s)?", billingEventsToCancel.size());
}
@Override
protected String execute() throws Exception {
int cancelledOneTimes = 0;
for (OneTime oneTime : oneTimesToCancel) {
cancelledOneTimes +=
int cancelledBillingEvents = 0;
for (BillingEvent billingEvent : billingEventsToCancel) {
cancelledBillingEvents +=
tm().transact(
() -> {
if (alreadyCancelled(oneTime)) {
if (alreadyCancelled(billingEvent)) {
System.out.printf(
"OneTime %d already cancelled, this is unexpected.\n", oneTime.getId());
"BillingEvent %d already cancelled, this is unexpected.\n",
billingEvent.getId());
return 0;
}
tm().put(
new Cancellation.Builder()
.setOneTimeEventKey(oneTime.createVKey())
.setBillingTime(oneTime.getBillingTime())
.setDomainHistoryId(oneTime.getHistoryEntryId())
.setRegistrarId(oneTime.getRegistrarId())
.setEventTime(oneTime.getEventTime())
.setReason(BillingEvent.Reason.ERROR)
.setTargetId(oneTime.getTargetId())
new BillingCancellation.Builder()
.setBillingEvent(billingEvent.createVKey())
.setBillingTime(billingEvent.getBillingTime())
.setDomainHistoryId(billingEvent.getHistoryEntryId())
.setRegistrarId(billingEvent.getRegistrarId())
.setEventTime(billingEvent.getEventTime())
.setReason(BillingBase.Reason.ERROR)
.setTargetId(billingEvent.getTargetId())
.build());
System.out.printf(
"Added Cancellation for OneTime with ID %d\n", oneTime.getId());
"Added BillingCancellation for BillingEvent with ID %d\n",
billingEvent.getId());
return 1;
});
}
return String.format("Created %d Cancellation event(s)", cancelledOneTimes);
return String.format("Created %d BillingCancellation event(s)", cancelledBillingEvents);
}
private boolean alreadyCancelled(OneTime oneTime) {
return tm().createQueryComposer(Cancellation.class)
.where("refOneTime", Comparator.EQ, oneTime.getId())
private boolean alreadyCancelled(BillingEvent billingEvent) {
return tm().createQueryComposer(BillingCancellation.class)
.where("billingEvent", Comparator.EQ, billingEvent.getId())
.first()
.isPresent();
}

Some files were not shown because too many files have changed in this diff Show More