1
0
mirror of https://github.com/google/nomulus synced 2026-01-31 01:52:28 +00:00

Compare commits

...

99 Commits

Author SHA1 Message Date
Weimin Yu
f9659af3b2 Remove aggressive check in RegistryJpaIO.Write (#1889) 2022-12-22 17:12:09 -05:00
Ben McIlwain
0aeb92ee16 Standardize hostname handling in URS command (#1886) 2022-12-19 16:22:52 -05:00
sarahcaseybot
4ede5f0c8a Prevent saving duplicate rows in spec11 pipeline (#1810)
* Prevent saving duplicate rows in spec11 pipeline

* Chain applies together
2022-12-15 15:51:28 -05:00
Lai Jiang
2292bfcaed Remove pipline servlet endpoint mapping (#1885) 2022-12-14 16:28:57 -05:00
sarahcaseybot
b056d2945f Add package monitoring for active domain limits (#1867)
* Add monitoring for package active domain limits

* Reformat action class

* Fix a bunch of nits

* Change native query to HQL
2022-12-14 12:10:10 -05:00
Lai Jiang
b8b1dce40a Remove TransactionManagerFactory.jpaTm() (#1883)
Since JPA is the only TM now, there's no point distinguishing tm() from
jpaTm().
2022-12-13 14:37:46 -05:00
Lai Jiang
d7e008a4af Move SQL files to resources from java (#1879)
This is similar to where we store the SQL files for beam pipelines, and
frankly makes more sense. Also streamlined the use of the API to read
SQL files from a jar.
2022-12-12 16:32:27 -05:00
Lai Jiang
d943ebd423 Add CodeQL workflow (#1884)
This is based on #1864. Also removes the LGTM setup as it is turning
down.
2022-12-12 15:52:47 -05:00
gbrodman
0ff9543efa Use standard Java thread creation in Concurrent (#1880)
The AppEngine thread factory is only useful if we can't create our own
(this is no longer the case) or if we need access to AppEngine APIs
(this is no longer the case).

The Concurrent class is only used by the DNS writer and the
CreateGroupsAction.
2022-12-12 15:42:02 -05:00
gbrodman
bb54ace0c0 Change the cookie auth mechanism to use IAP-provided JWTs (#1877) 2022-12-12 13:51:33 -05:00
Lai Jiang
cfee4713ed Remove sharding parameter from RegistryJpaIO (#1856)
This parameter is misleading and does not do what it purports to do.
Namely, it does not impact the level of parallelism. Given the input n for this
parameter, and m for the batch size, the elements are divided (keyed) into n
groups, each of which are then spread evenly across all threads, which
are eventually in turn batched into batches with size m:

https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupIntoBatches.java#L227

This is also evident in the implementation itself, where the ShardedKey
is determined by the unique number for a worker/thread combo and the
original key:

https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupIntoBatches.java#L268

Using a more concrete example, suppose we have 100 elements and 10
worker threads, with a target batch size of 5. If the "shard" number is set to
1, we first spread the 100 elements across 10 threads, resulting in 10
elements per thread, each thread then batches the elements into 2
batches of size 5.

If the "shard" number is set to 2, the 100 elements are first divided into 2
"shards" of 50 each. Each "shard" is then distributed within the 10
threads, resulting in 5 elements per "shard" per thread. They then get
turned into 1 batch per "shard" per thread. In the end, each thread
still processes 2 batches, even though they are from 2 different "shards".

Therefore this "shard" number does not perform horizontal partitioning
that one normally associates with sharding, and provides no
performance benefits but rather confuses the user.

It is also suggested that using withShardedKey() alone is already
sufficient to achieve auto-sharding within the keyed group. There is no
need to manually divide the input by keying them differently based on
the "shard" number specified:

https://youtu.be/jses0W4Zalc?t=967
2022-12-12 11:55:24 -05:00
Lai Jiang
dc7d123f6d Remove @EmptySetToNull (#1878)
This annotation was only used in cross-database comparison.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1878)
<!-- Reviewable:end -->
2022-12-09 15:07:00 -05:00
Lai Jiang
717334aa89 Remove the mention of ofy in most places (#1872)
There are still some mentions of ofy after this PR, but mostly in places
that would need to be modified later anyway.
2022-12-08 20:38:57 -05:00
Lai Jiang
215a70feba Remove the use of datastore.Text (#1875)
There doesn't seem to be any reason to do this any more. This data is
no longer persisted to datastore.
2022-12-08 17:17:16 -05:00
Lai Jiang
82f636a21e Remove some appengine dependencies (#1874)
Some retriers are no longer needed because transactions are
automatically retried by the JPA transaction manager when there's a
transient exception.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1874)
<!-- Reviewable:end -->
2022-12-08 11:46:47 -05:00
Lai Jiang
87e8cf4165 Remove unused endpoints on the Tools service (#1873)
We no longer use App Engine Remote API as of #1858.

The pipeline endpoint is only for GAE mapreduce, which we stopped doing
for a while.

TESTED=deployed to alpha and used nomulus tool built from master to
connect to the tools service.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1873)
<!-- Reviewable:end -->
2022-12-08 11:15:24 -05:00
Lai Jiang
55dcd65ffd Replace appengine repackaged class with the proper class (#1871) 2022-12-07 12:35:29 -05:00
Lai Jiang
9088a8d0ac Remove unused nomulus tool command (#1870)
This command was used to compare mapreduce-generated escrow deposits
with those generated by dataflow.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1870)
<!-- Reviewable:end -->
2022-12-07 12:33:38 -05:00
sarahcaseybot
342ae7a5de Add defaultPromoTokens to Registry (#1850)
* Add defaultPromoTokens to Registry

* Remove flyway files from this PR

* Fix merge conflicts

* Add back flyway file

* Add more info to error messages

* Change to a list

* Fix javadoc

* Change error message

* Add note to field declaration
2022-12-06 12:22:43 -05:00
gbrodman
9bf1bf47dd Take anchor tenant tokens into account in domain check flows (#1868)
These were always properly reflected in the actual creations, but when
running a check flow it would still show a non-zero cost even when using
an ANCHOR_TENANT allocation token. This changes it so that we accurately
show the $0.00 cost.
2022-12-05 16:14:53 -05:00
gbrodman
6dc1ca0279 Remove usage of the AppEngine remote API (#1858)
This is only used for contacting Datastore. With the removal of:
1. All standard usages of Datastore
2. Usage of Datastore for allocation of object IDs
3. Usage of Datastore for GAE user IDs

we can remove the remote API without affecting functionality.

This also allows us to just use SQL for every command since it's lazily
supplied. This simplifies the SQL setup and means that we remove a
possible situation where we forget the SQL setup.
2022-12-05 13:23:18 -05:00
Lai Jiang
1d7dfe4e07 Remove Ofy (#1863)
So long, farewell, adios, ciao, sayonara, 再见!

TESTED=deployed to alpha and used `nomulus list_tlds` to confirm that the web app can receive and serve requests.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1863)
<!-- Reviewable:end -->
2022-12-02 22:28:33 -05:00
Lai Jiang
601aed388c Fix javadoc build (#1866)
With newer versions of Java 11, javadoc fails to build due to unknown
tags in package-info.java files. These files are not important so we
exclude them.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1866)
<!-- Reviewable:end -->
2022-12-02 13:37:56 -05:00
Weimin Yu
46a7956f77 Fix nomulus GetEppResourceCommand (#1865)
* Fix nomulus GetEppResourceCommand

Fixes a bug in read_timestamp validation.

Fixes string representation of Collection fields in Epp Resources.
2022-12-01 18:35:15 -05:00
Lai Jiang
63d3453848 Re-add parenthesis (#1862)
Apparently IntelliJ doesn't like the extra parens, but our own
ErrorProne checks want it for clarity.
2022-11-30 10:45:12 -05:00
Lai Jiang
85272a30a2 Use login email instead of GAE user ID for RegistrarPoc (#1852)
Switch to using the login email address instead of GAE user ID to
identify console users. The primary use cases are:

1) When the user logged in the registrar console, need to figure out
   which registrars they have access to (in
   AuthenticatedReigstrarAccess).

2) When a user tries to apply a registry lock, needs to know if they
   can (in RegistryLockGetAction).

Both cases are tested in alpha with a personal email address to ensure
it does not get the permission due to being a GAE admin account.

Also verified that the soy templates includes the hidden login email
form field instead of GAE user ID when registrars are displayed on the
console; and consequently when a contact update is posted to the server,
the login email is part of the JSON payload. Even though it does not
look like it is used in any way by RegistrarSettingsAction, which
receives the POST request. Like GAE user ID, the field is hidden, so
cannot be changed by the user from the console, it is also not used to
identify the RegistryPoc entity, whose composite keys are the contact
email and the registrar ID associated with it.

The login email address is backfilled for all RegistrarPocs that have a
non-null GAE user ID. The backfilled addresses converted to the same ID
as stored in the database.
2022-11-29 17:16:19 -05:00
gbrodman
e3944d5d52 Rename AppEngineConnection to ServiceConnection (#1857)
It doesn't actually use any App Engine libraries or code -- it's just a
generic connection with authentication to a service. This also involves
changing that block of config to be "gcpProject" instead of "appEngine"
since it's more generic.

Note: this will require an internal PR as well to change the
corresponding private config block
2022-11-28 15:46:51 -05:00
sarahcaseybot
124a3d83ba Remove package token on manual transfer approval (#1819)
* Remove package token on manual transfer approval

* remove extra variables

* Add back white space

* Don't overwrite existingDomain

* Format fixes, use available helper variables

* Use PACKAGE allocation tokens in tests

* Refactor

* Fix merge conflicts

* Dont overwrite existingRecurring
2022-11-28 15:30:55 -05:00
Pavlo Tkach
99cbb862dc remove jpaTransactionManagerType rde pipeline param (#1860) 2022-11-28 12:13:45 -05:00
sarahcaseybot
4e3151ca02 Remove names from packages on automatic transfers (#1827)
* Remove names from packages on automatic transfers

* Add more tests

* Remove unneccesary local variable

* Eliminate unnecessary api call

* Reformat if blocks

* Don't overwrite existingRecurring
2022-11-23 15:48:59 -05:00
sarahcaseybot
292bc788fb Flyway files for defaultPromoTokens (#1851)
* Flyway files for defaultPromoTokens

* Add flyway file
2022-11-22 14:51:08 -05:00
Lai Jiang
b1ee1e8441 Delete DatastoreEntityExtension (#1855)
All entities are Ofy key free and therefore do not this hack any more.
2022-11-21 14:39:01 -05:00
Pavlo Tkach
f3f00d0ae0 Restore SelfAllocateId for RDE pipeline (#1853) 2022-11-18 11:13:41 -05:00
Lai Jiang
ca213aa8d7 Fix missing autorenew onetime billing events (#1854)
This PR fixes the issue where the onetime billing event for an autorenew
is not correctly created if the recurrence of the autorenew is closed
during the autorenew grace period, such as the case if a manual renew
happens during the same grace period.

The detailed analysis of the issue is captured in b/258822640. Note that
this is a quick and dirty fix to make ongoing billing event expanse work
correctly in the future. It does not fix the missing events in the past,
nor can it be used to reconstruct the missing ones (by providing a
different cursor time), due to timeout when triggering the action from
nomulus curl.

Per Weimin, the recurrences that fits the new condition along, based on
the current cursor, would increase from 382k to 430k, a 12% increase. I
checked last nights cron job run, which starts on 22:00 EST and seemed
to finish at 22:15 EST (when the last log for this request was
recorded), so it should definitely still finish in time for the nightly
runs with the new condition.
2022-11-17 12:04:34 -05:00
Lai Jiang
592dadd12e Add a field to save the login email of a RegistrarPoc (#1849)
This will replace the gaeUserId field. For now the field is not used and
only added to alter the schema.
2022-11-11 15:15:39 -05:00
Pavlo Tkach
9d7e3cdf79 Switch to SQL sequence based allocateId, remove SelfAllocateId (#1831) 2022-11-10 18:25:40 -05:00
sarahcaseybot
cf0560607e Send email for packages over create limit (#1835)
* Send email for packages over create limit

* Small change to query

* Fix small nits
2022-11-10 18:08:27 -05:00
Lai Jiang
78ca14e426 Remove JpaDemoPipeline (#1848)
<!-- Reviewable:start -->
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1848)
<!-- Reviewable:end -->
2022-11-10 10:46:11 -05:00
Weimin Yu
72da4cc493 Use keyless delegated credential (#1847) 2022-11-10 10:44:25 -05:00
Lai Jiang
961f9e7844 Re-add RefreshDnsOnHostRenameAction (#1845)
This class was accidentally deleted in #1661. This PR recreates it by
mostly re-adding its SQL-based code flow:

https://cs.opensource.google/nomulus/nomulus/+/master:core/src/test/java/google/registry/batch/RefreshDnsOnHostRenameActionTest.java;drc=9912e35ea297e969a428efdb1f8f01c86d794305;bpv=0;bpt=0

It does away with a pull queue due to incompatibility with Cloud Tasks.
Given what we have seen (about 700 tasks enqueued since May 2022), it
does not add much value in batching this operation anyway.

Also deleted AsyncTaskMetrics, which is not used any more. I don't think
we need to re-add metrics for this class either.
2022-11-09 17:21:20 -05:00
Lai Jiang
d2b9ebafc8 Remove references to Objectify (#1846)
This is not a complete removal of ofy as we still a dependency on it
(GaeUserIdConverter). But this PR removed it from a lot of places where
it's no longer needed.
2022-11-09 11:31:00 -05:00
Pavlo Tkach
9546408a3a Allow status, crDate, upDate, trDate, exDate and ns fields for all Domain Info responses (#1842) 2022-11-08 16:40:50 -05:00
Weimin Yu
0ee15b3a59 Fix Gradle dependency version pinning (#1843)
* Fix Gradle dependency version pinning

In Gradle 7, version labels require '!!' at the end to be free from
any forced upgrade.

Hibernate min version needs to be advanced past 5.6.12, which is buggy.

Upgraded most dependencies to the latest version.
2022-11-08 11:59:39 -05:00
Lai Jiang
01464e8e05 Remove async-delete-pull queue (#1841)
It is not being used anymore.
2022-11-07 22:16:49 -05:00
Weimin Yu
52b0574c73 Use GoogleCredentials for tools Cloud SQL access (#1844) 2022-11-04 17:20:21 -04:00
Lai Jiang
a0f177b71f The only remaining Ofy entity is GaeUserConverter after this PR. (#1838)
Therefore this PR also removed several classes and related tests that
support the setup and verification of Ofy entities.

In addition, support for creating a VKey from a string is limited to
VKey<? extends EppResource> only because it is the only use case (to
pass a key to an EPP resource in a web safe way to facilitate resave),
and we do not want to keep an extra simple name to class mapping, in
addition to what persistence.xml contains. I looked into using
PersistenceXmlUtility to obtain the mapping, but the xml file contains
classes with the same simple name (namely OneTime from both PollMessage
and BillingEvent). It doesn't seem like a worthwhile investment to write
more code to deal with that, when the fact is that we only need to
consider EppResource.
2022-11-04 12:47:11 -04:00
Weimin Yu
e01448b52e Fix list_cursors command for globals (#1840) 2022-11-04 11:18:20 -04:00
Weimin Yu
4da474e094 Remove Cloud KMS from Nomulus Server (#1839)
* Remove Cloud KMS from Nomulus Server

Removed Cloud KMS from the Nomulus (:core) since it is no longer used.

Renamed remaining classes to reflect their use of the SecretManager.

Updated the config instructions to use a new codename for the keyring:
KMS to CSM. This PR works with both codenames. Will drop 'KMS' after
the internal repo is updated.
2022-11-04 11:17:15 -04:00
Weimin Yu
e273a18b4a Implement Keyless Delegated credential (#1836)
Add a implementation of Delegated credential without using downloaded private key.

This is a stop-gap implementation while waiting for a solution from the Java auth library.

Also added a verifier action to test the new credential in production. Testing is helpful because:

Configuration is per-environment, therefore, success in alpha does not fully validate prod.
The relevant use case is triggered by low-frequency activities. Problem may not pop out for hours or longer.
2022-11-03 11:45:23 -04:00
Weimin Yu
8275bc45b9 Switch MetricReporter to App default credential (#1837) 2022-11-02 20:02:52 -04:00
Pavlo Tkach
0b6805531a Add flyway files for allocateId sequence alternative (#1830) 2022-11-02 12:06:40 -04:00
Lai Jiang
592454d97d Remove ofy support from HistoryEntry (#1823)
This PR removes all Ofy related cruft around `HistoryEntry` and its three subclasses in order to support dual-write to datastore and SQL. The class structure was refactored to take advantage of inheritance to reduce code duplication and improve clarity.

Note that for the embedded EPP resources, either their columns are all empty (for pre-3.0 entities imported into SQL), including their unique foreign key (domain name, host name, contact id) and the update timestamp; or they are filled as expected (for entities that were written since dual writing was implemented).

Therefore the check for foreign key column nullness in the various `@PostLoad` methods in the original code is an no-op as the EPP resource would have been loaded as null. In another word, there is no case where the update timestamp is null but other columns are not.

See the following query for the most recent entries in each table where the foreign key column or the update timestamp are null -- they are the same.

```
[I]postgres=> select MAX(history_modification_time) from "DomainHistory" where update_timestamp is null;
            max
----------------------------
 2021-09-27 15:56:52.502+00
(1 row)

[I]postgres=> select MAX(history_modification_time) from "DomainHistory" where domain_name is null;
            max
----------------------------
 2021-09-27 15:56:52.502+00
(1 row)

[I]postgres=> select MAX(history_modification_time) from "ContactHistory" where update_timestamp is null;
            max
----------------------------
 2021-09-27 15:56:04.311+00
(1 row)

[I]postgres=> select MAX(history_modification_time) from "ContactHistory" where contact_id is null;
            max
----------------------------
 2021-09-27 15:56:04.311+00
(1 row)

[I]postgres=> select MAX(history_modification_time) from "HostHistory" where update_timestamp is null;
            max
----------------------------
 2021-09-27 15:52:16.517+00
(1 row)

[I]postgres=> select MAX(history_modification_time) from "HostHistory" where host_name is null;
            max
----------------------------
 2021-09-27 15:52:16.517+00
(1 row)
```
2022-11-01 21:17:20 -04:00
Weimin Yu
671e42474c Document alternative method to deploy schema 2022-11-01 12:58:11 -04:00
Lai Jiang
1c90a6648e Remove bulk query entities (#1834)
These alternative ORMs are introduced as a way to make querying large number of
domains and domain histories more efficient through bulk loading from several
to-be-joined tables separately, then in-memory re-assembly of the final entity,
bypassing the need to query multiple tables each time an entity is queried.

Their primary use case is loading these entities for comparison between
datastore and SQL during the migration, which has been completed. The
code remain unused as of now and their existence makes refactoring and
general maintenance more complicated than necessary due to the need to keep
them up to date.

Therefore we remove the related code.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1834)
<!-- Reviewable:end -->
2022-10-28 12:25:57 -04:00
Lai Jiang
3f68ad5ea3 Rename BackupGroupRoot (#1829)
Also removed the ability to disable update timestamp auto update as it
was only needed during the migration.

Lastly, rectified the use of raw Coder in RegistryJpaIO.
2022-10-28 12:22:53 -04:00
gbrodman
9c6c210e21 Check for entity nonexistence in SqlBatchWriter (#1824)
Passing in an already-existing instance is an antipattern because it can
lead to race conditions where something else modified the object in
between when the pipeline loaded it and when you're saving it. The Write
action should only be writing new entities.

We cannot check IDs for the objects (some IDs are not autogenerated so
they might exist already). We also cannot call `insert` on the objects
because the underlying JPA `persist` call adds the input object to the
persistence context, meaning that any modifications (e.g.
updateTimestamp) are reflected in the input object. Beam doesn't allow
modification of input objects.
2022-10-27 14:46:26 -04:00
sarahcaseybot
ca60ca159f Add DEFAULT_PROMO token type (#1832)
* Add DEFAULT_PROMO token type

* Fix test error message check
2022-10-27 13:08:15 -04:00
Lai Jiang
82092b3516 Remove ofy-only functions in TransactionManager (#1826)
Also remove the use of auditedOfy in places other than the
GaeUserIdConverter.
2022-10-25 15:52:00 -04:00
sarahcaseybot
0746d28e0c Check token type of currentPackageToken (#1825)
* Check currentPackageToken TokenType

* Check TokenType of currentPackageToken

* Check that token already exists
2022-10-25 12:39:33 -04:00
Lai Jiang
aaa311ec40 Remove the mechanism to compare objects across database (#1822)
The migration is done.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1822)
<!-- Reviewable:end -->
2022-10-20 13:19:48 -04:00
Lai Jiang
addef17904 Does not self allocate IDs in Beam by default. (#1809)
* Does not self allocate IDs in Beam by default.

Per b/250948425, it is dangerous to implicitly allow all Beam pipelines
to create buildables by self allocating the IDs. This change makes it so
that one has to explicitly request self allocation in Beam.

A boolean is added to the pipeline option so that it can be passed to
the beam worker initializer that controls the behavior of the JVM on
each worker. Note that we did not add the option in the metadata.json file
because we did not want people to use the override at run time when launching
a pipeline, due to the risk. As shown in RdePipeline.java, we instead
explicitly hard-code the option in the pipeline. There is nothing that
stops one to supply that option when launching the pipeline, but it's
not advised.

Tested=deployed the pipeline alpha and ran it.
2022-10-19 20:44:06 -04:00
Weimin Yu
8fe3c08069 Properly create and use default credential (#1818)
* Properly create and use default credential

This PR consists of the following changes:

- Stopped adding scopes to the default credential when using it to access other
  non-workspace GCP APIs. Scopes are not needed here.

- Started applying scopes to the default credential when using to access
  Drive and Sheets APIs.
  - Upgraded Drive access from the deprecated credential lib to the
    up-to-date one
  - Switched Sheet access from the exported json credential to the
    scoped default credential.

This PR requires that the affected files be writable to the default
service account (project-name@appspot.gserviceaccount.com) of the
project.

- This is already the case for exported files (premium terms, reserved
  terms, and domain list).

- The registrar sync sheets in alpha, sandbox, and production have been
  updated with the new permissions.

All impacted operations have been tested in alpha.

* Properly create and use default credential

This PR consists of the following changes:

- Added a new method to generate scope-less default credential when using it to
  access other non-workspace GCP APIs. Scopes are not needed here.

  - Started to use the new credential in the SecreteManager.
  - Will migrate other usages to this new credential gradually.
  - Marked the old DefaultCredential as deprecated.

- Started applying scopes to the default credential when using to access Drive
  and Sheets APIs.

  - Upgraded Drive access from the deprecated credentials lib
  - Switched Sheet access from the exported json credential to the scoped
    default credential.

This PR requires that the affected files be writable to the default service
account (project-name@appspot.gserviceaccount.com) of the project.

- This is already the case for exported files (premium terms, reserved terms,
  and domain list).

- The registrar sync sheets in alpha, sandbox, and production have been
  updated with the new permissions.

All impacted operations have been tested in alpha.
2022-10-18 20:20:36 -04:00
sarahcaseybot
5dc796b1f7 Add monitoring for package max create limit (#1798)
* Add action for checking package domain create limit compliance

* Add create limit monitoring

* Change variable name

* Add more logging
2022-10-18 12:39:53 -04:00
Ben McIlwain
8bddf35d0d Revert "Upgrade App Engine Standard to Java 17 w/ bundled APIs (#1816)" (#1817)
This reverts commit 1ab077d267.

Apparently the new version of Spinnaker that is compatible with this doesn't
work for our release, so we need to roll this back for now. (Again!)
2022-10-13 10:05:47 -04:00
Pavlo Tkach
7b9c16ca3e Update conditions when domain update flow triggers dns publish task (#1811)
Addressing b/246375161
2022-10-12 10:25:33 -04:00
Ben McIlwain
1ab077d267 Upgrade App Engine Standard to Java 17 w/ bundled APIs (#1816) 2022-10-11 20:06:37 -04:00
gbrodman
ca65fbcc79 Refactor createSynthetic to be a command instead of a pipeline (#1813) 2022-10-11 12:23:31 -04:00
sarahcaseybot
0cfa7f8081 Remove allocation token check for transfering package domains (#1814) 2022-10-11 11:37:52 -04:00
Lai Jiang
9e31047c3a Fix nomulus command (#1812)
go/r3pr/1805 introduced an injectable clock in a few commands, but we
forgot to add the corresponding injector in the component. This PR fixes
it.
2022-10-09 16:45:42 -04:00
Pavlo Tkach
b7c2e8fba5 Limit environments allowed to send emails out (#1807) 2022-10-07 12:12:57 -04:00
Pavlo Tkach
a299df3005 Add fallback for Spec11 ThreatMatch parser (#1806) 2022-10-06 13:54:43 +00:00
Pavlo Tkach
a9b35c163d Revert "Do not enqueue DNS updates when flow doesn't affect nameservers (#1785)" (#1808)
This reverts commit 775f672f2a.
2022-10-05 14:13:52 -04:00
gbrodman
9da24d114c Use injected times in URSC and CommandTestCase (#1805)
We started getting failures because some of the tests used October. In
general we should freeze the clock for testing as much as possible.

Same thing with the Get*Commands
2022-10-04 15:36:41 -04:00
Lai Jiang
7dd5876315 Refactor VKeyConverter (#1794)
Remove the redundant composite key boolean and simply the annotation
structure a bit.
2022-10-03 15:49:18 -04:00
gbrodman
d1a259f63a Modify the CreateSynthetic pipeline to run over all non-deleted domains (#1803) 2022-10-03 15:15:41 -04:00
sarahcaseybot
8c5d2e9d92 Don't allow package tokens to discount premium names (#1804) 2022-10-03 14:27:10 -04:00
gbrodman
cca1306b09 Change some READ_COMMITTED levels to REPEATABLE_READ (#1802)
Basically, any time we're loading a bunch of linked objects that might
change, we want to have REPEATABLE_READ so that another transaction
doesn't come along and smush whatever we think we're loading.

The following instances of READ_COMMITTED haven't changed:
- RdePipeline (it only loads immutable objects like histories)
- Invoicing pipeline (only immutable objects like BillingEvents)
- Spec11 (doesn't use any linked info from Domain)

This also changes the PersistenceModule to use REPEATABLE_READ by
default on the replica JPA TM, for the standard reasoning.
2022-09-30 14:44:50 -04:00
Weimin Yu
47071b0fbb Restore log4j exclusion in gradle build (#1801) 2022-09-30 14:04:00 -04:00
Weimin Yu
d83565d37e Add a new allowed license string (#1800)
There are sporadic errors when building on desktop using maven central.
2022-09-30 14:03:17 -04:00
Weimin Yu
a557b3f376 Disable the cron job for ResaveAllEppResourcesPipelineAction (#1799)
See b/249863289 for more information.
2022-09-30 12:05:29 -04:00
sarahcaseybot
f4a49864b5 Add a get_package_promotion Command (#1793)
* Add a get_package_promotion Command

* add changes to loadByTokenString

* Fix test
2022-09-29 15:02:16 -04:00
gbrodman
acdecca181 Don't create unnecessary synthetic History objects (#1796) 2022-09-26 13:41:57 -04:00
gbrodman
5264ab3fc3 Create pipeline to save synthetic DomainHistory objects (#1795)
This runs over all domains that weren't deleted as of September 5. This
will fix most of b/248112997, which is itself caused by b/245940594 --
creating synthetic history objects means that the RDE pipeline will look
at those instead of the potentially-no-longer-valid data in the old
history objects.
2022-09-22 14:58:50 -04:00
sarahcaseybot
a9d59e4d6e Fix id generation in PackagePromotion (#1788)
* Fix id generation in PackagePromotion

* Fix update command tests
2022-09-21 15:19:49 -04:00
sarahcaseybot
1d3738da27 Add mutating commands for PackagePromotion (#1769)
* Add mutating commands for PackagePromotion

* Add checkAllocationToken methods

* Remove abstract methods

* Add better comments

* Small fixes

* Remove unneccesary init method

* Only assert in transaction in helper method
2022-09-21 12:38:09 -04:00
Lai Jiang
82a3a49268 Rename various fields and classes after migration (#1784)
Also fixed a bug introduced in #1785 where identity checked were performed instead of equality. This resulted in two sets containing the same elements not being regarded as equal and subsequent DNS updated being unnecessarily enqueued.
2022-09-21 11:49:22 -04:00
Pavlo Tkach
5bbad483e4 Fail genenerate invoices job when billing events not finished expanding (#1791) 2022-09-21 09:20:05 -04:00
Pavlo Tkach
f6e9dae58d Add REMOVEPACKAGE token functionality to domain transfer flow (#1792) 2022-09-19 15:11:36 -04:00
Lai Jiang
c4c1c72306 Refactor ForeignKeyIndex into ForeignKeyUtils (#1783)
The old class is modeled after datastore with some logic jammed in for it to work with SQL as well. As of #1777, the ofy related logic is deleted, however the general structure of the class remained datastore oriented.

This PR refactors the existing class into a ForeignKeyUtils helper class that does away wit the index subclasses and provides static helper methods to do the same, in a SQL-idiomatic fashion.

Some minor changes are made to the EPP resource classes to make it possible to create them in a SQL only environment in tests.
2022-09-19 14:41:19 -04:00
Pavlo Tkach
775f672f2a Do not enqueue DNS updates when flow doesn't affect nameservers (#1785) 2022-09-16 16:59:04 -04:00
gbrodman
372c854268 Create a scrap command to cancel OneTime billing events by ID (#1790)
This allows us to correct situations where we have erroneously charged
registrars for an action, without explicitly issuing a refund.
2022-09-16 16:17:31 -04:00
Lai Jiang
edbca15bf4 Remove generics from TransferData (#1787)
`TransferData` is currently a generic class with a complicated type parameter that designate the `Builder` class of its concrete subclass, on order to facilitate returning the said `Builder` from an instance loosely typed to the superclass (`TransferData`) itself.

While this works, in most all places that a `TransferData` is used, the raw, un-generic type is declared, resulting a lot of warnings, not to mention the fact that type safety not actually checked when raw type is used.

In this PR, we make it so that the concrete `Builder` is returned through a protected abstract method that is implemented by the subclasses. The type information therefore no longer needs to be embedded in the superclass type signature, and reflection is not necessary to create the `Builder` either. Overall, it makes `TransferData` a much cleaner class without the messiness of generics.
2022-09-15 14:07:38 -04:00
sarahcaseybot
5f41adf843 Flyway file for autogenerated PackagePromotion id fix (#1789)
* Flyway file for autogenerated PackagePromotion id fix

* Actually include the flyway file
2022-09-15 13:28:46 -04:00
Lai Jiang
e21f64b745 Delete EppResourceIndex and EppResourceIndexBucket (#1774) 2022-09-15 10:50:22 -04:00
sarahcaseybot
0dee97934a Prevent creation of package domains for more than 1 year (#1786)
* Prevent creation of package domains for more than 1 year

* Fix docs test
2022-09-14 14:49:56 -04:00
gbrodman
1070173264 Load, project, and save in one txn in ResaveAERP (#1780) 2022-09-13 15:59:49 -04:00
Pavlo Tkach
b9a3c0cd96 Add dry run test for remove package token (#1782) 2022-09-13 11:20:53 -04:00
sarahcaseybot
120456d138 Increase dns update failure max retry count (#1781) 2022-09-12 16:17:31 -04:00
gbrodman
66736d52f0 Add a cookie-based OAuth2 authenticator (#1761)
This uses the GoogleIdTokenVerifier to verify ID tokens passed in
(presumably from a front end) via cookies. This isn't used anywhere yet
but it will be used for front-end API calls for the new console.
2022-09-12 15:03:05 -04:00
682 changed files with 15054 additions and 22025 deletions

View File

@@ -6,5 +6,4 @@ node_modules/
repos/**
**/.idea/
*.jar
!third_party/**/*.jar
!/gradle/wrapper/**/*.jar

63
.github/workflows/codeql.yml vendored Normal file
View File

@@ -0,0 +1,63 @@
name: "CodeQL"
on:
push:
branches: [ 'master' ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ 'master' ]
schedule:
- cron: '24 4 * * 2'
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: [ 'java', 'javascript', 'python' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
steps:
- name: Checkout repository
uses: actions/checkout@v3
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
queries: security-and-quality
# Autobuild attempts to build any compiled languages (C/C++, C#, Go, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2
# Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
# If the Autobuild fails above, remove it and uncomment the following three lines.
# modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.
# - run: |
# echo "Run, Build Application using script"
# ./location_of_script_within_repo/buildscript.sh
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
with:
category: "/language:${{matrix.language}}"

1
.gitignore vendored
View File

@@ -14,7 +14,6 @@ gjf.out
*.jar
*.war
*.ear
!/third_party/**/*.jar
# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
hs_err_pid*

View File

@@ -1,6 +0,0 @@
extraction:
java:
prepare:
packages: "npm"
index:
java_version: "11"

View File

@@ -1,8 +1,8 @@
# Nomulus
| Internal Build | FOSS Build | LGTM | License | Code Search |
|:--------------:|:----------:|:----:|:-------:|:-----------:|
|[![Build Status for Google Registry internal build](https://storage.googleapis.com/domain-registry-kokoro/internal/build.svg)](https://storage.googleapis.com/domain-registry-kokoro/internal/index.html)|[![Build Status for the open source build](https://storage.googleapis.com/domain-registry-kokoro/foss/build.svg)](https://storage.googleapis.com/domain-registry-kokoro/foss/index.html)|[![Total alerts](https://img.shields.io/lgtm/alerts/g/google/nomulus.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/google/nomulus/alerts/)|[![License for this repo](https://img.shields.io/github/license/google/nomulus.svg)](https://github.com/google/nomulus/blob/master/LICENSE)|[![Link to Code Search](https://www.gstatic.com/devopsconsole/images/oss/favicons/oss-32x32.png)](https://cs.opensource.google/nomulus/nomulus)|
| Internal Build | FOSS Build | License | Code Search |
|:--------------:|:----------:|:-------:|:-----------:|
|[![Build Status for Google Registry internal build](https://storage.googleapis.com/domain-registry-kokoro/internal/build.svg)](https://storage.googleapis.com/domain-registry-kokoro/internal/index.html)|[![Build Status for the open source build](https://storage.googleapis.com/domain-registry-kokoro/foss/build.svg)](https://storage.googleapis.com/domain-registry-kokoro/foss/index.html)|[![License for this repo](https://img.shields.io/github/license/google/nomulus.svg)](https://github.com/google/nomulus/blob/master/LICENSE)|[![Link to Code Search](https://www.gstatic.com/devopsconsole/images/oss/favicons/oss-32x32.png)](https://cs.opensource.google/nomulus/nomulus)|
![Nomulus logo](./nomulus-logo.png)

View File

@@ -355,8 +355,6 @@ subprojects {
}
}
if (project.name == 'third_party') return
project.tasks.test.dependsOn runPresubmits
def commonlyExcludedResources = ['**/*.java', '**/BUILD']
@@ -528,6 +526,10 @@ task javaIncrementalFormatApply {
task javadoc(type: Javadoc) {
source javadocSource
// Java 11.0.17 has the following bug that affects annotation handling on
// package-info.java:
// https://bugs.openjdk.org/browse/JDK-8222091
exclude "**/package-info.java"
classpath = files(javadocClasspath)
destinationDir = file("${buildDir}/docs/javadoc")
options.encoding = "UTF-8"

View File

@@ -4,31 +4,38 @@
antlr:antlr:2.7.7=checkstyle
aopalliance:aopalliance:1.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
args4j:args4j:2.0.23=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson.core:jackson-core:2.13.3=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson:jackson-bom:2.13.3=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson.core:jackson-core:2.14.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson:jackson-bom:2.14.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.github.ben-manes.caffeine:caffeine:2.7.0=annotationProcessor,testAnnotationProcessor
com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor
com.google.api-client:google-api-client:1.35.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-common-protos:2.9.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-iam-v1:1.4.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:api-common:2.2.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax-httpjson:0.103.2=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax:2.18.2=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-storage:v1-rev20220705-1.32.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auth:google-auth-library-credentials:1.7.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auth:google-auth-library-oauth2-http:1.7.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auto.value:auto-value-annotations:1.9=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auto.value:auto-value:1.9=annotationProcessor
com.google.android:annotations:4.1.1.4=testRuntimeClasspath
com.google.api-client:google-api-client:2.1.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:gapic-google-cloud-storage-v2:2.16.0-alpha=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-storage-v2:2.16.0-alpha=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-iam-v1:1.6.22=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-storage-v2:2.16.0-alpha=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-common-protos:2.11.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-iam-v1:1.6.22=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:api-common:2.2.2=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax-grpc:2.20.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax-httpjson:0.105.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax:2.20.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-storage:v1-rev20220705-2.0.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auth:google-auth-library-credentials:1.13.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auth:google-auth-library-oauth2-http:1.13.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auto.value:auto-value-annotations:1.10.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auto.value:auto-value:1.10.1=annotationProcessor
com.google.auto:auto-common:0.10=annotationProcessor,testAnnotationProcessor
com.google.cloud:google-cloud-core-http:2.8.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-core:2.8.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-storage:2.10.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-core-grpc:2.9.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-core-http:2.9.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-core:2.9.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-storage:2.16.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.code.findbugs:jFormatString:3.0.0=annotationProcessor,testAnnotationProcessor
com.google.code.findbugs:jsr305:3.0.2=annotationProcessor,checkstyle,compileClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.code.gson:gson:2.9.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.code.gson:gson:2.10=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.common.html.types:types:1.0.6=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.errorprone:error_prone_annotation:2.3.4=annotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_annotations:2.11.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.errorprone:error_prone_annotations:2.16=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.errorprone:error_prone_annotations:2.3.4=annotationProcessor,checkstyle,testAnnotationProcessor
com.google.errorprone:error_prone_check_api:2.3.4=annotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_core:2.3.4=annotationProcessor,testAnnotationProcessor
@@ -39,20 +46,21 @@ com.google.guava:guava:27.0.1-jre=annotationProcessor,testAnnotationProcessor
com.google.guava:guava:29.0-jre=checkstyle
com.google.guava:guava:31.1-jre=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,compileClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-apache-v2:1.42.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-appengine:1.42.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-gson:1.42.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-jackson2:1.42.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client:1.42.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-apache-v2:1.42.3=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-appengine:1.42.3=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-gson:1.42.3=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-jackson2:1.42.3=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client:1.42.3=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.inject.extensions:guice-multibindings:4.1.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.inject:guice:4.1.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.j2objc:j2objc-annotations:1.1=annotationProcessor,testAnnotationProcessor
com.google.j2objc:j2objc-annotations:1.3=checkstyle,compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.jsinterop:jsinterop-annotations:1.0.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.oauth-client:google-oauth-client:1.34.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.protobuf:protobuf-java-util:3.21.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.protobuf:protobuf-java:3.21.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.protobuf:protobuf-java-util:3.21.10=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.protobuf:protobuf-java:3.21.10=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.protobuf:protobuf-java:3.4.0=annotationProcessor,testAnnotationProcessor
com.google.re2j:re2j:1.6=testRuntimeClasspath
com.google.template:soy:2021-02-01=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.google.truth.extensions:truth-java8-extension:1.1.3=testCompileClasspath,testRuntimeClasspath
com.google.truth:truth:1.1.3=testCompileClasspath,testRuntimeClasspath
@@ -60,32 +68,48 @@ com.googlecode.java-diff-utils:diffutils:1.3.0=annotationProcessor,testAnnotatio
com.ibm.icu:icu4j:57.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
com.puppycrawl.tools:checkstyle:8.37=checkstyle
commons-beanutils:commons-beanutils:1.9.4=checkstyle
commons-codec:commons-codec:1.11=compileClasspath,testCompileClasspath,testRuntimeClasspath
commons-codec:commons-codec:1.15=compileClasspath,testCompileClasspath,testRuntimeClasspath
commons-collections:commons-collections:3.2.2=checkstyle
commons-logging:commons-logging:1.2=compileClasspath,testCompileClasspath,testRuntimeClasspath
info.picocli:picocli:4.5.2=checkstyle
io.grpc:grpc-context:1.47.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-alts:1.51.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-api:1.51.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-auth:1.51.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-context:1.51.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-core:1.51.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-googleapis:1.51.0=testRuntimeClasspath
io.grpc:grpc-grpclb:1.51.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-netty-shaded:1.51.0=testRuntimeClasspath
io.grpc:grpc-protobuf-lite:1.51.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-protobuf:1.51.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-services:1.51.0=testRuntimeClasspath
io.grpc:grpc-stub:1.51.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-xds:1.51.0=testRuntimeClasspath
io.opencensus:opencensus-api:0.31.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-contrib-http-util:0.31.1=compileClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-proto:0.2.0=testRuntimeClasspath
io.perfmark:perfmark-api:0.26.0=testRuntimeClasspath
javax.annotation:javax.annotation-api:1.3.2=compileClasspath,testCompileClasspath,testRuntimeClasspath
javax.annotation:jsr250-api:1.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
javax.inject:javax.inject:1=compileClasspath,testCompileClasspath,testRuntimeClasspath
junit:junit:4.13.2=testCompileClasspath,testRuntimeClasspath
net.bytebuddy:byte-buddy-agent:1.12.10=testCompileClasspath,testRuntimeClasspath
net.bytebuddy:byte-buddy:1.12.10=testCompileClasspath,testRuntimeClasspath
net.bytebuddy:byte-buddy-agent:1.12.16=testCompileClasspath,testRuntimeClasspath
net.bytebuddy:byte-buddy:1.12.16=testCompileClasspath,testRuntimeClasspath
net.sf.saxon:Saxon-HE:10.3=checkstyle
org.antlr:antlr4-runtime:4.8-1=checkstyle
org.apache.commons:commons-lang3:3.11=compileClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-text:1.9=compileClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-lang3:3.12.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-text:1.10.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.httpcomponents:httpclient:4.5.13=compileClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.httpcomponents:httpcore:4.4.15=compileClasspath,testCompileClasspath,testRuntimeClasspath
org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath
org.checkerframework:checker-qual:2.11.1=checkstyle
org.checkerframework:checker-qual:3.0.0=annotationProcessor,testAnnotationProcessor
org.checkerframework:checker-qual:3.22.2=compileClasspath,testCompileClasspath,testRuntimeClasspath
org.checkerframework:checker-qual:3.28.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
org.checkerframework:dataflow:3.0.0=annotationProcessor,testAnnotationProcessor
org.checkerframework:javacutil:3.0.0=annotationProcessor,testAnnotationProcessor
org.codehaus.mojo:animal-sniffer-annotations:1.17=annotationProcessor,testAnnotationProcessor
org.codehaus.mojo:animal-sniffer-annotations:1.22=testRuntimeClasspath
org.conscrypt:conscrypt-openjdk-uber:2.5.2=compileClasspath,testCompileClasspath,testRuntimeClasspath
org.hamcrest:hamcrest-core:1.3=testCompileClasspath,testRuntimeClasspath
org.jacoco:org.jacoco.agent:0.8.6=jacocoAgent,jacocoAnt
org.jacoco:org.jacoco.ant:0.8.6=jacocoAnt
@@ -93,13 +117,13 @@ org.jacoco:org.jacoco.core:0.8.6=jacocoAnt
org.jacoco:org.jacoco.report:0.8.6=jacocoAnt
org.javassist:javassist:3.26.0-GA=checkstyle
org.json:json:20160212=compileClasspath,testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-api:5.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-engine:5.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-commons:1.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-engine:1.9.0=testCompileClasspath,testRuntimeClasspath
org.junit:junit-bom:5.9.0=testCompileClasspath,testRuntimeClasspath
org.mockito:mockito-core:4.6.1=testCompileClasspath,testRuntimeClasspath
org.objenesis:objenesis:3.2=testRuntimeClasspath
org.junit.jupiter:junit-jupiter-api:5.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-engine:5.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-commons:1.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-engine:1.9.1=testCompileClasspath,testRuntimeClasspath
org.junit:junit-bom:5.9.1=testCompileClasspath,testRuntimeClasspath
org.mockito:mockito-core:4.9.0=testCompileClasspath,testRuntimeClasspath
org.objenesis:objenesis:3.3=testRuntimeClasspath
org.opentest4j:opentest4j:1.2.0=testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-analysis:7.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-analysis:8.0.1=jacocoAnt
@@ -116,5 +140,5 @@ org.plumelib:plume-util:1.0.6=annotationProcessor,testAnnotationProcessor
org.plumelib:reflection-util:0.0.2=annotationProcessor,testAnnotationProcessor
org.plumelib:require-javadoc:0.1.0=annotationProcessor,testAnnotationProcessor
org.reflections:reflections:0.9.12=checkstyle
org.threeten:threetenbp:1.6.0=compileClasspath,testCompileClasspath,testRuntimeClasspath
org.threeten:threetenbp:1.6.4=compileClasspath,testCompileClasspath,testRuntimeClasspath
empty=

View File

@@ -32,7 +32,7 @@ commons-collections:commons-collections:3.2.2=checkstyle
info.picocli:picocli:4.5.2=checkstyle
io.github.java-diff-utils:java-diff-utils:4.12=compileClasspath,default,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
javax.inject:javax.inject:1=compileClasspath,default,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
joda-time:joda-time:2.10.14=compileClasspath,default,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
joda-time:joda-time:2.12.2=compileClasspath,default,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
junit:junit:4.13.2=default,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
net.sf.saxon:Saxon-HE:10.3=checkstyle
org.antlr:antlr4-runtime:4.8-1=checkstyle
@@ -50,11 +50,11 @@ org.jacoco:org.jacoco.ant:0.8.6=jacocoAnt
org.jacoco:org.jacoco.core:0.8.6=jacocoAnt
org.jacoco:org.jacoco.report:0.8.6=jacocoAnt
org.javassist:javassist:3.26.0-GA=checkstyle
org.junit.jupiter:junit-jupiter-api:5.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-engine:5.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-commons:1.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-engine:1.9.0=testCompileClasspath,testRuntimeClasspath
org.junit:junit-bom:5.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-api:5.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-engine:5.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-commons:1.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-engine:1.9.1=testCompileClasspath,testRuntimeClasspath
org.junit:junit-bom:5.9.1=testCompileClasspath,testRuntimeClasspath
org.opentest4j:opentest4j:1.2.0=testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-analysis:8.0.1=jacocoAnt
org.ow2.asm:asm-commons:8.0.1=jacocoAnt

View File

@@ -43,12 +43,6 @@ by Joshua Bloch in his book Effective Java -->
<property name="message" value='Your Javadocs appear to use invalid &lt;a&gt; tag syntax in @see tags. Please use the correct syntax: @see &lt;a href="http(s)://your_url"&gt;url_description&lt;/a&gt;'/>
</module>
<!-- Checks that our Ofy wrapper is used instead of the "real" ofy(). -->
<module name="RegexpSingleline">
<property name="format" value="com\.googlecode\.objectify\.ObjectifyService\.ofy"/>
<property name="message" value="Use google.registry.model.ofy.ObjectifyService.ofy(). Do not use com.googlecode.objectify.v4.ObjectifyService.ofy()."/>
</module>
<!-- Checks that java.util.Optional is used instead of Guava's Optional. -->
<module name="RegexpSingleline">
<property name="format" value="com\.google\.common\.base\.Optional"/>
@@ -58,7 +52,7 @@ by Joshua Bloch in his book Effective Java -->
<!-- Checks that the deprecated ExpectedException is not used. -->
<module name="RegexpSingleline">
<property name="format" value="org\.junit\.rules\.ExpectedException"/>
<property name="message" value="Use assertThrows and expectThrows from JUnitBackports instead of the deprecated methods on ExpectedException."/>
<property name="message" value="Use assertThrows and expectThrows instead of the deprecated methods on ExpectedException."/>
</module>
<module name="LineLength">

View File

@@ -9,6 +9,4 @@
<suppress files="[/\\].*[/\\]generated.*[/\\]" checks="."/>
<!-- Ignore Javadoc checks in test files -->
<suppress files="[/\\].*[/\\]src/test/java/.*[/\\]" checks="JavadocType"/>
<!-- ofy() regex check doesn't apply to these files -->
<suppress files="AugmentedDeleter.java|AugmentedSaver.java|Ofy.java" checks="RegexpSingleline"/>
</suppressions>

View File

@@ -207,6 +207,9 @@
{
"moduleLicense": "GNU Library General Public License v2.1 or later"
},
{
"moduleLicense": "GNU Lesser General Public License v3.0"
},
// This is just 3-clause BSD.
{
"moduleLicense": "Go License"

View File

@@ -109,15 +109,6 @@ PRESUBMITS = {
"System.(out|err).println is only allowed in tools/ packages. Please "
"use a logger instead.",
# ObjectifyService.register is restricted to main/ or AppEngineExtension.
PresubmitCheck(
r".*\bObjectifyService\.register", "java", {
"/build/", "/generated/", "node_modules/", "src/main/",
"AppEngineExtension.java"
}):
"ObjectifyService.register(...) is not allowed in tests. Please use "
"AppEngineExtension.register(...) instead.",
# PostgreSQLContainer instantiation must specify docker tag
# TODO(b/204572437): Fix the pattern to pass DatabaseSnapshotTest.java
PresubmitCheck(

View File

@@ -163,11 +163,6 @@ configurations {
dependencies {
def deps = rootProject.dependencyMap
// Custom-built objectify jar at commit ecd5165, included in Nomulus
// release.
implementation files(
"${rootDir}/third_party/objectify/v4_1/objectify-4.1.3.jar")
testRuntimeOnly files(sourceSets.test.resources.srcDirs)
implementation deps['com.beust:jcommander']
@@ -186,7 +181,6 @@ dependencies {
implementation deps['com.google.apis:google-api-services-admin-directory']
implementation deps['com.google.apis:google-api-services-appengine']
implementation deps['com.google.apis:google-api-services-bigquery']
implementation deps['com.google.apis:google-api-services-cloudkms']
implementation deps['com.google.apis:google-api-services-dataflow']
implementation deps['com.google.apis:google-api-services-dns']
implementation deps['com.google.apis:google-api-services-drive']
@@ -195,9 +189,6 @@ dependencies {
implementation deps['com.google.apis:google-api-services-sheets']
implementation deps['com.google.apis:google-api-services-storage']
testImplementation deps['com.google.appengine:appengine-api-stubs']
implementation deps['com.google.appengine.tools:appengine-gcs-client']
implementation deps['com.google.appengine.tools:appengine-pipeline']
implementation deps['com.google.appengine:appengine-remote-api']
implementation deps['com.google.auth:google-auth-library-credentials']
implementation deps['com.google.auth:google-auth-library-oauth2-http']
implementation deps['com.google.cloud.bigdataoss:util']
@@ -268,7 +259,7 @@ dependencies {
testImplementation deps['org.apache.sshd:sshd-sftp']
testImplementation deps['org.apache.tomcat:tomcat-annotations-api']
implementation deps['org.bouncycastle:bcpg-jdk15on']
testImplementation deps['org.bouncycastle:bcpkix-jdk15on']
implementation deps['org.bouncycastle:bcpkix-jdk15on']
implementation deps['org.bouncycastle:bcprov-jdk15on']
testImplementation deps['com.fasterxml.jackson.core:jackson-databind']
runtime deps['org.glassfish.jaxb:jaxb-runtime']
@@ -705,7 +696,8 @@ createToolTask(
sourceSets.nonprod)
createToolTask(
'jpaDemoPipeline', 'google.registry.beam.common.JpaDemoPipeline')
'createSyntheticDomainHistories',
'google.registry.tools.javascrap.CreateSyntheticDomainHistoriesPipeline')
project.tasks.create('generateSqlSchema', JavaExec) {
classpath = sourceSets.nonprod.runtimeClasspath

View File

@@ -8,13 +8,14 @@ args4j:args4j:2.0.26=css
cglib:cglib-nodep:2.2=css
com.101tec:zkclient:0.10=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.beust:jcommander:1.60=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson.core:jackson-annotations:2.13.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson.core:jackson-core:2.13.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson.core:jackson-databind:2.13.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson:jackson-bom:2.13.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson.core:jackson-annotations:2.14.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson.core:jackson-core:2.14.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson.core:jackson-databind:2.14.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson.dataformat:jackson-dataformat-toml:2.14.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.14.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.14.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.14.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson:jackson-bom:2.14.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml:classmate:1.5.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.ben-manes.caffeine:caffeine:2.7.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.github.ben-manes.caffeine:caffeine:2.9.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
@@ -36,72 +37,73 @@ com.google.api-client:google-api-client-jackson2:1.32.2=compileClasspath,default
com.google.api-client:google-api-client-java6:1.35.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api-client:google-api-client-servlet:1.35.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api-client:google-api-client:1.35.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.136.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:gapic-google-cloud-storage-v2:2.16.0-alpha=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.23.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.147.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.147.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.6.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.98.4=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.5.4=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.23.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.23.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.23.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-storage-v2:2.2.2-alpha=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-common-protos:2.8.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.12.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.136.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.136.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.6.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.6.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.10=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-firestore-v1:3.1.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.98.4=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.5.4=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.3.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.3.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.23.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.23.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-spanner-v1:6.23.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-storage-v2:2.2.2-alpha=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-tasks-v2:2.3.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.93.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.93.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-common-protos:2.9.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-iam-v1:1.4.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:api-common:2.2.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax-grpc:2.18.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax-httpjson:0.103.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax:2.18.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.14.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.102.20=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.7.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.31.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.31.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.31.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-storage-v2:2.16.0-alpha=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-common-protos:2.9.6=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-iam-v1:1.6.22=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.23.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.147.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.147.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.14.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.14.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-datastore-v1:0.102.5=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-firestore-v1:3.6.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0=compileClasspath,nonprodCompileClasspath,testCompileClasspath
com.google.api.grpc:proto-google-cloud-monitoring-v3:3.4.6=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.102.20=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.7.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.6.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.6.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.31.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.31.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-spanner-v1:6.31.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-storage-v2:2.16.0-alpha=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-tasks-v2:2.6.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.96.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.96.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-common-protos:2.11.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-iam-v1:1.6.22=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:api-common:2.2.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax-grpc:2.20.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax-httpjson:0.105.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax:2.20.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-cloudkms:v1-rev20220701-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-appengine:v1-rev20221205-2.0.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-bigquery:v2-rev20220924-2.0.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-clouddebugger:v2-rev20220318-2.0.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20220828-2.0.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-dataflow:v1b3-rev20220920-2.0.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-drive:v2-rev393-1.25.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-groupssettings:v1-rev20210624-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-groupssettings:v1-rev20210624-2.0.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-healthcare:v1-rev20220818-2.0.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-monitoring:v3-rev20220715-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220623-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-storage:v1-rev20220705-1.32.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine.tools:appengine-gcs-client:0.8.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine.tools:appengine-pipeline:0.2.13=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-api-1.0-sdk:2.0.5=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-api-stubs:2.0.5=testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-remote-api:2.0.5=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-monitoring:v3-rev20221205-2.0.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-pubsub:v1-rev20220904-2.0.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-sheets:v4-rev20220927-2.0.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-sqladmin:v1beta4-rev20221017-2.0.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-storage:v1-rev20220705-2.0.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-api-1.0-sdk:1.9.86=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath
com.google.appengine:appengine-api-1.0-sdk:2.0.10=testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-api-stubs:2.0.10=testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-testing:1.9.86=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auth:google-auth-library-credentials:1.8.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auth:google-auth-library-oauth2-http:1.8.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auth:google-auth-library-credentials:1.13.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auth:google-auth-library-oauth2-http:1.13.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auto.service:auto-service:1.0.1=annotationProcessor
com.google.auto.value:auto-value-annotations:1.9=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auto.value:auto-value:1.9=annotationProcessor,default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testRuntimeClasspath
com.google.auto.value:auto-value-annotations:1.10.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auto.value:auto-value:1.10.1=annotationProcessor,testAnnotationProcessor
com.google.auto.value:auto-value:1.9=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auto:auto-common:0.10=errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.auto:auto-common:1.2=annotationProcessor
com.google.closure-stylesheets:closure-stylesheets:1.5.0=css
@@ -109,38 +111,40 @@ com.google.cloud.bigdataoss:gcsio:2.2.6=compileClasspath,default,deploy_jar,nonp
com.google.cloud.bigdataoss:util:2.2.6=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.bigtable:bigtable-client-core:1.26.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.bigtable:bigtable-metrics-api:1.26.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.datastore:datastore-v1-proto-client:2.2.10=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.sql:jdbc-socket-factory-core:1.6.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.sql:postgres-socket-factory:1.6.2=default,deploy_jar,runtimeClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-bigquerystorage:2.12.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-bigtable:2.6.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-core-grpc:2.6.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-core-http:2.8.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-core:2.8.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-firestore:3.1.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-monitoring:1.82.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-nio:0.124.10=testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-pubsub:1.116.4=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-pubsublite:1.5.4=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-secretmanager:2.3.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-spanner:6.23.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-storage:2.10.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-tasks:2.3.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:grpc-gcp:1.1.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.1.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.datastore:datastore-v1-proto-client:2.9.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.sql:jdbc-socket-factory-core:1.7.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.sql:postgres-socket-factory:1.7.2=default,deploy_jar,runtimeClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-bigquerystorage:2.23.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-bigtable-stats:2.14.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-bigtable:2.14.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-core-grpc:2.9.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-core-http:2.9.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-core:2.9.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-firestore:3.6.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-monitoring:1.82.0=compileClasspath,nonprodCompileClasspath,testCompileClasspath
com.google.cloud:google-cloud-monitoring:3.4.6=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-nio:0.125.0=testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-pubsub:1.120.20=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-pubsublite:1.7.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-secretmanager:2.6.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-spanner:6.31.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-storage:2.16.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-tasks:2.6.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:grpc-gcp:1.2.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.6.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.code.findbugs:jFormatString:3.0.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.code.findbugs:jsr305:3.0.1=css
com.google.code.findbugs:jsr305:3.0.2=annotationProcessor,checkstyle,compileClasspath,default,deploy_jar,errorprone,nonprodAnnotationProcessor,nonprodCompileClasspath,nonprodRuntime,nonprodRuntimeClasspath,runtime,runtimeClasspath,soy,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.code.gson:gson:2.10=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.code.gson:gson:2.7=css,soy
com.google.code.gson:gson:2.9.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.common.html.types:types:1.0.6=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,soy,testCompileClasspath,testRuntimeClasspath
com.google.dagger:dagger-compiler:2.43=annotationProcessor,testAnnotationProcessor
com.google.dagger:dagger-producers:2.43=annotationProcessor,testAnnotationProcessor
com.google.dagger:dagger-spi:2.43=annotationProcessor,testAnnotationProcessor
com.google.dagger:dagger:2.43=annotationProcessor,compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.dagger:dagger-compiler:2.44.2=annotationProcessor,testAnnotationProcessor
com.google.dagger:dagger-producers:2.44.2=annotationProcessor,testAnnotationProcessor
com.google.dagger:dagger-spi:2.44.2=annotationProcessor,testAnnotationProcessor
com.google.dagger:dagger:2.44.2=annotationProcessor,compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.devtools.ksp:symbol-processing-api:1.7.0-1.0.6=annotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_annotation:2.3.4=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_annotations:2.14.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.errorprone:error_prone_annotations:2.16=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.errorprone:error_prone_annotations:2.3.4=checkstyle,errorprone,nonprodAnnotationProcessor,soy
com.google.errorprone:error_prone_annotations:2.7.1=annotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_check_api:2.3.4=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
@@ -163,12 +167,12 @@ com.google.guava:guava:31.0.1-jre=annotationProcessor,testAnnotationProcessor
com.google.guava:guava:31.1-jre=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,compileClasspath,default,deploy_jar,errorprone,nonprodAnnotationProcessor,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,soy,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.gwt:gwt-user:2.10.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-apache-v2:1.42.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-appengine:1.42.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-gson:1.42.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-jackson2:1.42.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-protobuf:1.41.7=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client:1.42.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-apache-v2:1.42.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-appengine:1.42.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-gson:1.42.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-jackson2:1.42.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-protobuf:1.41.8=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client:1.42.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.inject.extensions:guice-multibindings:4.1.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,soy,testCompileClasspath,testRuntimeClasspath
com.google.inject:guice:4.1.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.inject:guice:5.1.0=soy
@@ -187,31 +191,32 @@ com.google.oauth-client:google-oauth-client-java6:1.34.1=compileClasspath,defaul
com.google.oauth-client:google-oauth-client-jetty:1.34.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.oauth-client:google-oauth-client-servlet:1.34.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.oauth-client:google-oauth-client:1.34.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.protobuf:protobuf-java-util:3.21.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.protobuf:protobuf-java-util:3.21.10=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.protobuf:protobuf-java:2.5.0=css
com.google.protobuf:protobuf-java:3.21.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.protobuf:protobuf-java:3.21.10=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.protobuf:protobuf-java:3.4.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.protobuf:protobuf-java:4.0.0-rc-2=soy
com.google.re2j:re2j:1.7=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.re2j:re2j:1.6=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.template:soy:2021-02-01=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,soy,testCompileClasspath,testRuntimeClasspath
com.google.truth.extensions:truth-java8-extension:1.1.3=testCompileClasspath,testRuntimeClasspath
com.google.truth:truth:1.1.3=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.googlecode.java-diff-utils:diffutils:1.3.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.googlecode.json-simple:json-simple:1.1.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.ibm.icu:icu4j:57.1=compileClasspath,nonprodCompileClasspath,soy,testCompileClasspath
com.ibm.icu:icu4j:71.1=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
com.ibm.icu:icu4j:72.1=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
com.jcraft:jsch:0.1.55=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.lmax:disruptor:3.4.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.puppycrawl.tools:checkstyle:8.37=checkstyle
com.squareup.okhttp3:okhttp:3.11.0=testCompileClasspath,testRuntimeClasspath
com.squareup.okio:okio:1.14.0=testCompileClasspath,testRuntimeClasspath
com.squareup:javapoet:1.13.0=annotationProcessor,testAnnotationProcessor
com.squareup:kotlinpoet:1.11.0=annotationProcessor,testAnnotationProcessor
com.sun.activation:jakarta.activation:1.2.2=jaxb
com.sun.activation:javax.activation:1.2.0=jaxb
com.sun.istack:istack-commons-runtime:3.0.7=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.sun.istack:istack-commons-runtime:4.1.1=nonprodRuntime,runtime
com.sun.xml.bind:jaxb-impl:2.3.3=jaxb
com.sun.xml.bind:jaxb-osgi:4.0.0=jaxb
com.sun.xml.bind:jaxb-osgi:4.0.1=jaxb
com.sun.xml.bind:jaxb-xjc:2.3.3=jaxb
com.sun.xml.fastinfoset:FastInfoset:1.2.15=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.thoughtworks.paranamer:paranamer:2.7=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
@@ -221,7 +226,7 @@ commons-beanutils:commons-beanutils:1.9.4=checkstyle
commons-codec:commons-codec:1.15=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
commons-collections:commons-collections:3.2.2=checkstyle
commons-logging:commons-logging:1.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
dnsjava:dnsjava:3.5.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
dnsjava:dnsjava:3.5.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
guru.nidi.com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0=nonprodRuntime,runtime,testRuntimeClasspath
guru.nidi.com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0=nonprodRuntime,runtime,testRuntimeClasspath
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0=nonprodRuntime,runtime,testRuntimeClasspath
@@ -237,44 +242,41 @@ io.confluent:kafka-schema-registry-client:5.3.2=compileClasspath,default,deploy_
io.dropwizard.metrics:metrics-core:3.1.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.github.classgraph:classgraph:4.8.104=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.github.java-diff-utils:java-diff-utils:4.12=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-alts:1.45.1=compileClasspath,nonprodCompileClasspath,testCompileClasspath
io.grpc:grpc-alts:1.47.0=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.grpc:grpc-api:1.47.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-auth:1.45.1=compileClasspath,nonprodCompileClasspath,testCompileClasspath
io.grpc:grpc-auth:1.47.0=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.grpc:grpc-census:1.45.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-context:1.47.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-core:1.45.1=compileClasspath,nonprodCompileClasspath,testCompileClasspath
io.grpc:grpc-core:1.47.0=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.grpc:grpc-googleapis:1.47.0=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.grpc:grpc-grpclb:1.45.1=compileClasspath,nonprodCompileClasspath,testCompileClasspath
io.grpc:grpc-grpclb:1.47.0=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.grpc:grpc-netty-shaded:1.45.1=compileClasspath,nonprodCompileClasspath,testCompileClasspath
io.grpc:grpc-netty-shaded:1.47.0=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.grpc:grpc-netty:1.45.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-protobuf-lite:1.47.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-protobuf:1.47.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-services:1.45.1=compileClasspath,nonprodCompileClasspath,testCompileClasspath
io.grpc:grpc-services:1.47.0=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.grpc:grpc-stub:1.47.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-xds:1.45.1=compileClasspath,nonprodCompileClasspath,testCompileClasspath
io.grpc:grpc-xds:1.47.0=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.netty:netty-buffer:4.1.72.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-codec-http2:4.1.72.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-codec-http:4.1.72.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-codec-socks:4.1.72.Final=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.netty:netty-codec:4.1.72.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-common:4.1.72.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-handler-proxy:4.1.72.Final=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.netty:netty-handler:4.1.72.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-resolver:4.1.72.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-tcnative-boringssl-static:2.0.46.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-tcnative-classes:2.0.46.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-transport:4.1.72.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-alts:1.51.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-api:1.51.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-auth:1.51.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-census:1.49.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-context:1.51.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-core:1.51.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-googleapis:1.51.0=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.grpc:grpc-grpclb:1.51.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-netty-shaded:1.51.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-netty:1.49.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-protobuf-lite:1.51.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-protobuf:1.51.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-rls:1.49.2=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.grpc:grpc-services:1.49.2=compileClasspath,nonprodCompileClasspath,testCompileClasspath
io.grpc:grpc-services:1.51.0=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.grpc:grpc-stub:1.51.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-xds:1.49.2=compileClasspath,nonprodCompileClasspath,testCompileClasspath
io.grpc:grpc-xds:1.51.0=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.netty:netty-buffer:4.1.77.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-codec-http2:4.1.77.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-codec-http:4.1.77.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-codec-socks:4.1.77.Final=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.netty:netty-codec:4.1.77.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-common:4.1.77.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-handler-proxy:4.1.77.Final=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.netty:netty-handler:4.1.77.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-resolver:4.1.77.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-tcnative-boringssl-static:2.0.52.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-tcnative-classes:2.0.52.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-transport-native-unix-common:4.1.77.Final=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.netty:netty-transport:4.1.77.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-api:0.31.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-contrib-exemplar-util:0.31.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-contrib-grpc-util:0.31.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-contrib-grpc-util:0.31.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-contrib-http-util:0.31.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-contrib-resource-util:0.31.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-exporter-metrics-util:0.31.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
@@ -282,7 +284,7 @@ io.opencensus:opencensus-exporter-stats-stackdriver:0.31.0=compileClasspath,defa
io.opencensus:opencensus-impl-core:0.31.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-impl:0.31.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-proto:0.2.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.perfmark:perfmark-api:0.25.0=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.perfmark:perfmark-api:0.26.0=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
jakarta.activation:jakarta.activation-api:2.1.0=jaxb,nonprodRuntime,runtime
jakarta.xml.bind:jakarta.xml.bind-api:4.0.0=jaxb,nonprodRuntime,runtime
javacc:javacc:4.1=css
@@ -302,9 +304,8 @@ jline:jline:1.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonp
joda-time:joda-time:2.10.10=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
junit:junit:4.13.2=default,nonprodCompileClasspath,nonprodRuntimeClasspath,testCompileClasspath,testRuntimeClasspath
net.arnx:nashorn-promise:0.1.1=nonprodRuntime,runtime,testRuntimeClasspath
net.bytebuddy:byte-buddy-agent:1.12.10=testCompileClasspath,testRuntimeClasspath
net.bytebuddy:byte-buddy:1.12.10=testCompileClasspath,testRuntimeClasspath
net.bytebuddy:byte-buddy:1.12.9=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath
net.bytebuddy:byte-buddy-agent:1.12.16=testCompileClasspath,testRuntimeClasspath
net.bytebuddy:byte-buddy:1.12.18=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
net.java.dev.javacc:javacc:4.1=css
net.java.dev.jna:jna:5.8.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
net.ltgt.gradle.incap:incap:0.2=annotationProcessor,testAnnotationProcessor
@@ -314,31 +315,29 @@ org.apache.arrow:arrow-format:5.0.0=compileClasspath,default,deploy_jar,nonprodC
org.apache.arrow:arrow-memory-core:5.0.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.arrow:arrow-vector:5.0.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.avro:avro:1.8.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-model-fn-execution:2.40.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-model-job-management:2.40.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-model-pipeline:2.40.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-core-construction-java:2.40.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-core-java:2.40.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-direct-java:2.40.0=testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.40.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-java-fn-execution:2.40.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-core:2.40.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-expansion-service:2.40.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-extensions-arrow:2.40.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.40.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-extensions-protobuf:2.40.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-fn-execution:2.40.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-io-kafka:2.40.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-vendor-grpc-1_43_2:0.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-model-fn-execution:2.43.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-model-job-management:2.43.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-model-pipeline:2.43.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-core-construction-java:2.43.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-core-java:2.43.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-direct-java:2.43.0=testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.43.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-java-fn-execution:2.43.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-core:2.43.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-expansion-service:2.43.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-extensions-arrow:2.43.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.43.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-extensions-protobuf:2.43.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-fn-execution:2.43.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.43.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-io-kafka:2.43.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-vendor-grpc-1_48_1:0.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-vendor-guava-26_0-jre:0.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-compress:1.21=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-compress:1.22=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-csv:1.9.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-exec:1.3=nonprodRuntime,runtime,testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-lang3:3.11=testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-lang3:3.12.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath
org.apache.commons:commons-text:1.9=testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-lang3:3.12.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-text:1.10.0=testCompileClasspath,testRuntimeClasspath
org.apache.ftpserver:ftplet-api:1.2.0=testCompileClasspath,testRuntimeClasspath
org.apache.ftpserver:ftpserver-core:1.2.0=testCompileClasspath,testRuntimeClasspath
org.apache.httpcomponents:httpclient:4.5.13=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
@@ -347,7 +346,7 @@ org.apache.mina:mina-core:2.1.6=testCompileClasspath,testRuntimeClasspath
org.apache.sshd:sshd-core:2.0.0=testCompileClasspath,testRuntimeClasspath
org.apache.sshd:sshd-scp:2.0.0=testCompileClasspath,testRuntimeClasspath
org.apache.sshd:sshd-sftp:2.0.0=testCompileClasspath,testRuntimeClasspath
org.apache.tomcat:tomcat-annotations-api:10.1.0-M17=testCompileClasspath,testRuntimeClasspath
org.apache.tomcat:tomcat-annotations-api:11.0.0-M1=testCompileClasspath,testRuntimeClasspath
org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath
org.bouncycastle:bcpg-jdk15on:1.67=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.bouncycastle:bcpkix-jdk15on:1.67=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
@@ -357,23 +356,23 @@ org.checkerframework:checker-compat-qual:2.5.5=annotationProcessor,compileClassp
org.checkerframework:checker-qual:2.11.1=checkstyle
org.checkerframework:checker-qual:3.0.0=errorprone,nonprodAnnotationProcessor
org.checkerframework:checker-qual:3.12.0=annotationProcessor,testAnnotationProcessor
org.checkerframework:checker-qual:3.22.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.checkerframework:checker-qual:3.28.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.checkerframework:checker-qual:3.5.0=nonprodRuntime,runtime,soy
org.checkerframework:dataflow:3.0.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
org.checkerframework:javacutil:3.0.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
org.codehaus.jackson:jackson-core-asl:1.9.13=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.codehaus.jackson:jackson-mapper-asl:1.9.13=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.codehaus.mojo:animal-sniffer-annotations:1.17=errorprone,nonprodAnnotationProcessor
org.codehaus.mojo:animal-sniffer-annotations:1.21=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
org.conscrypt:conscrypt-openjdk-uber:2.5.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.codehaus.mojo:animal-sniffer-annotations:1.22=default,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
org.conscrypt:conscrypt-openjdk-uber:2.5.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.easymock:easymock:3.0=css
org.eclipse.angus:angus-activation:1.0.0=nonprodRuntime,runtime
org.flywaydb:flyway-core:9.0.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.glassfish.jaxb:jaxb-core:4.0.0=nonprodRuntime,runtime
org.flywaydb:flyway-core:9.10.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.glassfish.jaxb:jaxb-core:4.0.1=nonprodRuntime,runtime
org.glassfish.jaxb:jaxb-runtime:2.3.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.glassfish.jaxb:jaxb-runtime:4.0.0=nonprodRuntime,runtime
org.glassfish.jaxb:jaxb-runtime:4.0.1=nonprodRuntime,runtime
org.glassfish.jaxb:txw2:2.3.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.glassfish.jaxb:txw2:4.0.0=nonprodRuntime,runtime
org.glassfish.jaxb:txw2:4.0.1=nonprodRuntime,runtime
org.gwtproject:gwt-user:2.10.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.hamcrest:hamcrest-core:1.1=css
org.hamcrest:hamcrest-core:1.3=default,nonprodCompileClasspath,nonprodRuntimeClasspath
@@ -382,8 +381,8 @@ org.hamcrest:hamcrest-library:2.2=testCompileClasspath,testRuntimeClasspath
org.hamcrest:hamcrest:2.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath
org.hamcrest:hamcrest:2.2=testCompileClasspath,testRuntimeClasspath
org.hibernate.common:hibernate-commons-annotations:5.1.2.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.hibernate:hibernate-core:5.6.10.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.hibernate:hibernate-hikaricp:5.6.10.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.hibernate:hibernate-core:5.6.14.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.hibernate:hibernate-hikaricp:5.6.14.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.jacoco:org.jacoco.agent:0.8.6=jacocoAgent,jacocoAnt
org.jacoco:org.jacoco.ant:0.8.6=jacocoAnt
org.jacoco:org.jacoco.core:0.8.6=jacocoAnt
@@ -392,57 +391,58 @@ org.javassist:javassist:3.26.0-GA=checkstyle
org.jboss.logging:jboss-logging:3.4.3.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.jboss:jandex:2.4.2.Final=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.jetbrains.kotlin:kotlin-reflect:1.6.10=annotationProcessor,testAnnotationProcessor
org.jetbrains.kotlin:kotlin-stdlib-common:1.7.0=annotationProcessor,testAnnotationProcessor
org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.7.0=annotationProcessor,testAnnotationProcessor
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.7.0=annotationProcessor,testAnnotationProcessor
org.jetbrains.kotlin:kotlin-stdlib:1.7.0=annotationProcessor,testAnnotationProcessor
org.jetbrains.kotlinx:kotlinx-metadata-jvm:0.4.2=annotationProcessor,testAnnotationProcessor
org.jetbrains.kotlinx:kotlinx-metadata-jvm:0.5.0=annotationProcessor,testAnnotationProcessor
org.jetbrains:annotations:13.0=annotationProcessor,testAnnotationProcessor
org.jetbrains:annotations:17.0.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.joda:joda-money:1.0.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.joda:joda-money:1.0.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.json:json:20160212=soy
org.json:json:20200518=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.jsoup:jsoup:1.15.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.junit-pioneer:junit-pioneer:1.7.1=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-api:5.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-engine:5.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-migrationsupport:5.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-params:5.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-commons:1.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-engine:1.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-launcher:1.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-runner:1.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-suite-api:1.9.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-suite-commons:1.9.0=testRuntimeClasspath
org.junit:junit-bom:5.9.0=testCompileClasspath,testRuntimeClasspath
org.jsoup:jsoup:1.15.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.junit-pioneer:junit-pioneer:1.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-api:5.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-engine:5.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-migrationsupport:5.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-params:5.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-commons:1.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-engine:1.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-launcher:1.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-runner:1.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-suite-api:1.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-suite-commons:1.9.1=testRuntimeClasspath
org.junit:junit-bom:5.9.1=testCompileClasspath,testRuntimeClasspath
org.jvnet.staxex:stax-ex:1.8=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.mockito:mockito-core:1.10.19=css
org.mockito:mockito-core:4.6.1=testCompileClasspath,testRuntimeClasspath
org.mockito:mockito-junit-jupiter:4.6.1=testCompileClasspath,testRuntimeClasspath
org.mockito:mockito-core:4.9.0=testCompileClasspath,testRuntimeClasspath
org.mockito:mockito-junit-jupiter:4.9.0=testCompileClasspath,testRuntimeClasspath
org.mortbay.jetty:jetty-util:6.1.26=testCompileClasspath,testRuntimeClasspath
org.mortbay.jetty:jetty:6.1.26=testCompileClasspath,testRuntimeClasspath
org.objenesis:objenesis:2.1=css
org.objenesis:objenesis:3.2=testRuntimeClasspath
org.objenesis:objenesis:3.3=testRuntimeClasspath
org.opentest4j:opentest4j:1.2.0=testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-analysis:7.0=soy
org.ow2.asm:asm-analysis:8.0.1=jacocoAnt
org.ow2.asm:asm-analysis:9.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-analysis:9.4=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-commons:7.0=soy
org.ow2.asm:asm-commons:8.0.1=jacocoAnt
org.ow2.asm:asm-commons:9.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-tree:7.0=soy
org.ow2.asm:asm-tree:8.0.1=jacocoAnt
org.ow2.asm:asm-tree:9.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-tree:9.4=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-util:7.0=soy
org.ow2.asm:asm-util:9.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-util:9.4=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm:7.0=soy
org.ow2.asm:asm:8.0.1=jacocoAnt
org.ow2.asm:asm:9.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm:9.4=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.pcollections:pcollections:2.1.2=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
org.plumelib:plume-util:1.0.6=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
org.plumelib:reflection-util:0.0.2=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
org.plumelib:require-javadoc:0.1.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
org.postgresql:postgresql:42.4.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntime,nonprodRuntimeClasspath,runtime,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.postgresql:postgresql:42.5.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntime,nonprodRuntimeClasspath,runtime,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.reflections:reflections:0.9.12=checkstyle
org.rnorth.duct-tape:duct-tape:1.0.8=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.seleniumhq.selenium:selenium-api:3.141.59=testCompileClasspath,testRuntimeClasspath
@@ -459,23 +459,23 @@ org.slf4j:jcl-over-slf4j:1.7.30=nonprodRuntime,runtime,testRuntimeClasspath
org.slf4j:jul-to-slf4j:1.7.30=nonprodRuntime,runtime,testRuntimeClasspath
org.slf4j:slf4j-api:1.7.30=nonprodRuntime,runtime
org.slf4j:slf4j-api:1.7.36=compileClasspath,nonprodCompileClasspath,nonprodRuntimeClasspath,testCompileClasspath
org.slf4j:slf4j-api:2.0.0-alpha7=default,deploy_jar,runtimeClasspath,testRuntimeClasspath
org.slf4j:slf4j-jdk14:2.0.0-alpha7=default,deploy_jar,runtimeClasspath,testRuntimeClasspath
org.slf4j:slf4j-api:2.0.5=default,deploy_jar,runtimeClasspath,testRuntimeClasspath
org.slf4j:slf4j-jdk14:2.0.5=default,deploy_jar,runtimeClasspath,testRuntimeClasspath
org.springframework:spring-core:5.3.18=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.springframework:spring-expression:5.3.18=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.springframework:spring-jcl:5.3.18=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:database-commons:1.17.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:jdbc:1.17.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:junit-jupiter:1.17.3=testCompileClasspath,testRuntimeClasspath
org.testcontainers:postgresql:1.17.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:selenium:1.17.3=testCompileClasspath,testRuntimeClasspath
org.testcontainers:testcontainers:1.17.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.threeten:threetenbp:1.6.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:database-commons:1.17.6=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:jdbc:1.17.6=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:junit-jupiter:1.17.6=testCompileClasspath,testRuntimeClasspath
org.testcontainers:postgresql:1.17.6=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:selenium:1.17.6=testCompileClasspath,testRuntimeClasspath
org.testcontainers:testcontainers:1.17.6=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.threeten:threetenbp:1.6.4=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.tukaani:xz:1.5=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.w3c.css:sac:1.3=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.webjars.npm:viz.js-graphviz-java:2.1.3=nonprodRuntime,runtime,testRuntimeClasspath
org.xerial.snappy:snappy-java:1.1.8.4=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.yaml:snakeyaml:1.30=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.yaml:snakeyaml:1.33=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
us.fatehi:schemacrawler-api:16.10.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
us.fatehi:schemacrawler-diagram:16.10.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
us.fatehi:schemacrawler-tools:16.10.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath

View File

@@ -17,25 +17,16 @@ package google.registry.batch;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import com.google.appengine.api.taskqueue.Queue;
import com.google.appengine.api.taskqueue.TaskOptions;
import com.google.appengine.api.taskqueue.TaskOptions.Method;
import com.google.appengine.api.taskqueue.TransientFailureException;
import com.google.common.base.Joiner;
import com.google.common.collect.ArrayListMultimap;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Multimap;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.EppResource;
import google.registry.model.eppcommon.Trid;
import google.registry.model.host.Host;
import google.registry.persistence.VKey;
import google.registry.request.Action.Service;
import google.registry.util.CloudTasksUtils;
import google.registry.util.Retrier;
import javax.inject.Inject;
import javax.inject.Named;
import org.joda.time.DateTime;
import org.joda.time.Duration;
@@ -44,45 +35,25 @@ public final class AsyncTaskEnqueuer {
/** The HTTP parameter names used by async flows. */
public static final String PARAM_RESOURCE_KEY = "resourceKey";
public static final String PARAM_REQUESTING_CLIENT_ID = "requestingClientId";
public static final String PARAM_CLIENT_TRANSACTION_ID = "clientTransactionId";
public static final String PARAM_SERVER_TRANSACTION_ID = "serverTransactionId";
public static final String PARAM_IS_SUPERUSER = "isSuperuser";
public static final String PARAM_HOST_KEY = "hostKey";
public static final String PARAM_REQUESTED_TIME = "requestedTime";
public static final String PARAM_RESAVE_TIMES = "resaveTimes";
/** The task queue names used by async flows. */
public static final String QUEUE_ASYNC_ACTIONS = "async-actions";
public static final String QUEUE_ASYNC_DELETE = "async-delete-pull";
public static final String QUEUE_ASYNC_HOST_RENAME = "async-host-rename-pull";
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final Duration MAX_ASYNC_ETA = Duration.standardDays(30);
private final Duration asyncDeleteDelay;
private final Queue asyncDeletePullQueue;
private final Queue asyncDnsRefreshPullQueue;
private final Retrier retrier;
private CloudTasksUtils cloudTasksUtils;
private final CloudTasksUtils cloudTasksUtils;
@Inject
public AsyncTaskEnqueuer(
@Named(QUEUE_ASYNC_DELETE) Queue asyncDeletePullQueue,
@Named(QUEUE_ASYNC_HOST_RENAME) Queue asyncDnsRefreshPullQueue,
@Config("asyncDeleteDelay") Duration asyncDeleteDelay,
CloudTasksUtils cloudTasksUtils,
Retrier retrier) {
this.asyncDeletePullQueue = asyncDeletePullQueue;
this.asyncDnsRefreshPullQueue = asyncDnsRefreshPullQueue;
this.asyncDeleteDelay = asyncDeleteDelay;
public AsyncTaskEnqueuer(CloudTasksUtils cloudTasksUtils) {
this.cloudTasksUtils = cloudTasksUtils;
this.retrier = retrier;
}
/** Enqueues a task to asynchronously re-save an entity at some point in the future. */
public void enqueueAsyncResave(VKey<?> entityToResave, DateTime now, DateTime whenToResave) {
public void enqueueAsyncResave(
VKey<? extends EppResource> entityToResave, DateTime now, DateTime whenToResave) {
enqueueAsyncResave(entityToResave, now, ImmutableSortedSet.of(whenToResave));
}
@@ -93,7 +64,9 @@ public final class AsyncTaskEnqueuer {
* itself to run at the next time if there are remaining re-saves scheduled.
*/
public void enqueueAsyncResave(
VKey<?> entityKey, DateTime now, ImmutableSortedSet<DateTime> whenToResave) {
VKey<? extends EppResource> entityKey,
DateTime now,
ImmutableSortedSet<DateTime> whenToResave) {
DateTime firstResave = whenToResave.first();
checkArgument(isBeforeOrAt(now, firstResave), "Can't enqueue a resave to run in the past");
Duration etaDuration = new Duration(now, firstResave);
@@ -115,46 +88,4 @@ public final class AsyncTaskEnqueuer {
cloudTasksUtils.createPostTaskWithDelay(
ResaveEntityAction.PATH, Service.BACKEND.toString(), params, etaDuration));
}
/** Enqueues a task to asynchronously delete a contact or host, by key. */
public void enqueueAsyncDelete(
EppResource resourceToDelete,
DateTime now,
String requestingRegistrarId,
Trid trid,
boolean isSuperuser) {
logger.atInfo().log(
"Enqueuing async deletion of %s on behalf of registrar %s.",
resourceToDelete.getRepoId(), requestingRegistrarId);
TaskOptions task =
TaskOptions.Builder.withMethod(Method.PULL)
.countdownMillis(asyncDeleteDelay.getMillis())
.param(PARAM_RESOURCE_KEY, resourceToDelete.createVKey().stringify())
.param(PARAM_REQUESTING_CLIENT_ID, requestingRegistrarId)
.param(PARAM_SERVER_TRANSACTION_ID, trid.getServerTransactionId())
.param(PARAM_IS_SUPERUSER, Boolean.toString(isSuperuser))
.param(PARAM_REQUESTED_TIME, now.toString());
trid.getClientTransactionId()
.ifPresent(clTrid -> task.param(PARAM_CLIENT_TRANSACTION_ID, clTrid));
addTaskToQueueWithRetry(asyncDeletePullQueue, task);
}
/** Enqueues a task to asynchronously refresh DNS for a renamed host. */
public void enqueueAsyncDnsRefresh(Host host, DateTime now) {
VKey<Host> hostKey = host.createVKey();
logger.atInfo().log("Enqueuing async DNS refresh for renamed host %s.", hostKey);
addTaskToQueueWithRetry(
asyncDnsRefreshPullQueue,
TaskOptions.Builder.withMethod(Method.PULL)
.param(PARAM_HOST_KEY, hostKey.stringify())
.param(PARAM_REQUESTED_TIME, now.toString()));
}
/**
* Adds a task to a queue with retrying, to avoid aborting the entire flow over a transient issue
* enqueuing a task.
*/
private void addTaskToQueueWithRetry(final Queue queue, final TaskOptions task) {
retrier.callWithRetry(() -> queue.add(task), TransientFailureException.class);
}
}

View File

@@ -1,165 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch;
import static com.google.appengine.api.taskqueue.QueueConstants.maxLeaseCount;
import static com.google.monitoring.metrics.EventMetric.DEFAULT_FITTER;
import static google.registry.batch.AsyncTaskMetrics.OperationType.CONTACT_AND_HOST_DELETE;
import static google.registry.batch.AsyncTaskMetrics.OperationType.DNS_REFRESH;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import com.google.monitoring.metrics.DistributionFitter;
import com.google.monitoring.metrics.EventMetric;
import com.google.monitoring.metrics.FibonacciFitter;
import com.google.monitoring.metrics.IncrementableMetric;
import com.google.monitoring.metrics.LabelDescriptor;
import com.google.monitoring.metrics.MetricRegistryImpl;
import google.registry.util.Clock;
import javax.inject.Inject;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/**
* Instrumentation for async flows (contact/host deletion and DNS refreshes).
*
* @see AsyncTaskEnqueuer
*/
public class AsyncTaskMetrics {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final Clock clock;
@Inject
public AsyncTaskMetrics(Clock clock) {
this.clock = clock;
}
/**
* A Fibonacci fitter used for bucketing the batch count.
*
* <p>We use a Fibonacci filter because it provides better resolution at the low end than an
* exponential fitter, which is important because most batch sizes are likely to be very low,
* despite going up to 1,000 on the high end. Also, the precision is better, as batch size is
* inherently an integer, whereas an exponential fitter with an exponent base less than 2 would
* have unintuitive boundaries.
*/
private static final DistributionFitter FITTER_BATCH_SIZE =
FibonacciFitter.create(maxLeaseCount());
private static final ImmutableSet<LabelDescriptor> LABEL_DESCRIPTORS =
ImmutableSet.of(
LabelDescriptor.create("operation_type", "The type of async flow operation."),
LabelDescriptor.create("result", "The result of the async flow operation."));
@VisibleForTesting
static final IncrementableMetric asyncFlowOperationCounts =
MetricRegistryImpl.getDefault()
.newIncrementableMetric(
"/async_flows/operations",
"Count of Async Flow Operations",
"count",
LABEL_DESCRIPTORS);
@VisibleForTesting
static final EventMetric asyncFlowOperationProcessingTime =
MetricRegistryImpl.getDefault()
.newEventMetric(
"/async_flows/processing_time",
"Async Flow Processing Time",
"milliseconds",
LABEL_DESCRIPTORS,
DEFAULT_FITTER);
@VisibleForTesting
static final EventMetric asyncFlowBatchSize =
MetricRegistryImpl.getDefault()
.newEventMetric(
"/async_flows/batch_size",
"Async Operation Batch Size",
"batch size",
ImmutableSet.of(
LabelDescriptor.create("operation_type", "The type of async flow operation.")),
FITTER_BATCH_SIZE);
/** The type of asynchronous operation. */
public enum OperationType {
CONTACT_DELETE("contactDelete"),
HOST_DELETE("hostDelete"),
CONTACT_AND_HOST_DELETE("contactAndHostDelete"),
DNS_REFRESH("dnsRefresh");
private final String metricLabelValue;
OperationType(String metricLabelValue) {
this.metricLabelValue = metricLabelValue;
}
String getMetricLabelValue() {
return metricLabelValue;
}
}
/** The result of an asynchronous operation. */
public enum OperationResult {
/** The operation processed correctly and the result was success. */
SUCCESS("success"),
/** The operation processed correctly and the result was failure. */
FAILURE("failure"),
/** The operation did not process correctly due to some unexpected error. */
ERROR("error"),
/** The operation was skipped because the request is now stale. */
STALE("stale");
private final String metricLabelValue;
OperationResult(String metricLabelValue) {
this.metricLabelValue = metricLabelValue;
}
String getMetricLabelValue() {
return metricLabelValue;
}
}
public void recordAsyncFlowResult(
OperationType operationType, OperationResult operationResult, DateTime whenEnqueued) {
asyncFlowOperationCounts.increment(
operationType.getMetricLabelValue(), operationResult.getMetricLabelValue());
long processingMillis = new Duration(whenEnqueued, clock.nowUtc()).getMillis();
asyncFlowOperationProcessingTime.record(
processingMillis,
operationType.getMetricLabelValue(),
operationResult.getMetricLabelValue());
logger.atInfo().log(
"Asynchronous %s operation took %d ms to process, yielding result: %s.",
operationType.getMetricLabelValue(),
processingMillis,
operationResult.getMetricLabelValue());
}
public void recordContactHostDeletionBatchSize(long batchSize) {
asyncFlowBatchSize.record(batchSize, CONTACT_AND_HOST_DELETE.getMetricLabelValue());
}
public void recordDnsRefreshBatchSize(long batchSize) {
asyncFlowBatchSize.record(batchSize, DNS_REFRESH.getMetricLabelValue());
}
}

View File

@@ -14,13 +14,10 @@
package google.registry.batch;
import static com.google.appengine.api.taskqueue.QueueFactory.getQueue;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_REQUESTED_TIME;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_RESAVE_TIMES;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_RESOURCE_KEY;
import static google.registry.batch.AsyncTaskEnqueuer.QUEUE_ASYNC_ACTIONS;
import static google.registry.batch.AsyncTaskEnqueuer.QUEUE_ASYNC_DELETE;
import static google.registry.batch.AsyncTaskEnqueuer.QUEUE_ASYNC_HOST_RENAME;
import static google.registry.batch.CannedScriptExecutionAction.SCRIPT_PARAM;
import static google.registry.request.RequestParameters.extractBooleanParameter;
import static google.registry.request.RequestParameters.extractIntParameter;
import static google.registry.request.RequestParameters.extractLongParameter;
@@ -32,13 +29,11 @@ import static google.registry.request.RequestParameters.extractRequiredDatetimeP
import static google.registry.request.RequestParameters.extractRequiredParameter;
import static google.registry.request.RequestParameters.extractSetOfDatetimeParameters;
import com.google.appengine.api.taskqueue.Queue;
import com.google.common.collect.ImmutableSet;
import dagger.Module;
import dagger.Provides;
import google.registry.request.Parameter;
import java.util.Optional;
import javax.inject.Named;
import javax.servlet.http.HttpServletRequest;
import org.joda.time.DateTime;
@@ -128,21 +123,10 @@ public class BatchModule {
return extractBooleanParameter(req, PARAM_DRY_RUN);
}
// TODO(b/234424397): remove method after credential changes are rolled out.
@Provides
@Named(QUEUE_ASYNC_ACTIONS)
static Queue provideAsyncActionsPushQueue() {
return getQueue(QUEUE_ASYNC_ACTIONS);
}
@Provides
@Named(QUEUE_ASYNC_DELETE)
static Queue provideAsyncDeletePullQueue() {
return getQueue(QUEUE_ASYNC_DELETE);
}
@Provides
@Named(QUEUE_ASYNC_HOST_RENAME)
static Queue provideAsyncHostRenamePullQueue() {
return getQueue(QUEUE_ASYNC_HOST_RENAME);
@Parameter(SCRIPT_PARAM)
static String provideScriptName(HttpServletRequest req) {
return extractRequiredParameter(req, SCRIPT_PARAM);
}
}

View File

@@ -0,0 +1,74 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch;
import static google.registry.request.Action.Method.POST;
import com.google.common.collect.ImmutableMap;
import com.google.common.flogger.FluentLogger;
import google.registry.batch.cannedscript.GroupsApiChecker;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
import javax.inject.Inject;
/**
* Action that executes a canned script specified by the caller.
*
* <p>This class is introduced to help the safe rollout of credential changes. The delegated
* credentials in particular, benefit from this: they require manual configuration of the peer
* system in each environment, and may wait hours or even days after deployment until triggered by
* user activities.
*
* <p>This action can be invoked using the Nomulus CLI command: {@code nomulus -e ${env} curl
* --service BACKEND -X POST -u '/_dr/task/executeCannedScript?script=${script_name}'}
*/
// TODO(b/234424397): remove class after credential changes are rolled out.
@Action(
service = Action.Service.BACKEND,
path = "/_dr/task/executeCannedScript",
method = POST,
automaticallyPrintOk = true,
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
public class CannedScriptExecutionAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
static final String SCRIPT_PARAM = "script";
static final ImmutableMap<String, Runnable> SCRIPTS =
ImmutableMap.of("runGroupsApiChecks", GroupsApiChecker::runGroupsApiChecks);
private final String scriptName;
@Inject
CannedScriptExecutionAction(@Parameter(SCRIPT_PARAM) String scriptName) {
logger.atInfo().log("Received request to run script %s", scriptName);
this.scriptName = scriptName;
}
@Override
public void run() {
if (!SCRIPTS.containsKey(scriptName)) {
throw new IllegalArgumentException("Script not found:" + scriptName);
}
try {
SCRIPTS.get(scriptName).run();
logger.atInfo().log("Finished running %s.", scriptName);
} catch (Throwable t) {
logger.atWarning().withCause(t).log("Error executing %s", scriptName);
throw new RuntimeException("Execution failed.");
}
}
}

View File

@@ -0,0 +1,226 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Iterables;
import com.google.common.flogger.FluentLogger;
import com.google.common.primitives.Ints;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.domain.token.PackagePromotion;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.request.Action;
import google.registry.request.Action.Service;
import google.registry.request.auth.Auth;
import google.registry.ui.server.SendEmailUtils;
import google.registry.util.Clock;
import java.util.Optional;
import javax.inject.Inject;
import org.joda.time.Days;
/**
* An action that checks all {@link PackagePromotion} objects for compliance with their max create
* limit.
*/
@Action(
service = Service.BACKEND,
path = CheckPackagesComplianceAction.PATH,
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
public class CheckPackagesComplianceAction implements Runnable {
public static final String PATH = "/_dr/task/checkPackagesCompliance";
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final SendEmailUtils sendEmailUtils;
private final Clock clock;
private final String packageCreateLimitEmailSubject;
private final String packageDomainLimitWarningEmailSubject;
private final String packageDomainLimitUpgradeEmailSubject;
private final String packageCreateLimitEmailBody;
private final String packageDomainLimitWarningEmailBody;
private final String packageDomainLimitUpgradeEmailBody;
private final String registrySupportEmail;
private static final int THIRTY_DAYS = 30;
private static final int FORTY_DAYS = 40;
@Inject
public CheckPackagesComplianceAction(
SendEmailUtils sendEmailUtils,
Clock clock,
@Config("packageCreateLimitEmailSubject") String packageCreateLimitEmailSubject,
@Config("packageDomainLimitWarningEmailSubject") String packageDomainLimitWarningEmailSubject,
@Config("packageDomainLimitUpgradeEmailSubject") String packageDomainLimitUpgradeEmailSubject,
@Config("packageCreateLimitEmailBody") String packageCreateLimitEmailBody,
@Config("packageDomainLimitWarningEmailBody") String packageDomainLimitWarningEmailBody,
@Config("packageDomainLimitUpgradeEmailBody") String packageDomainLimitUpgradeEmailBody,
@Config("registrySupportEmail") String registrySupportEmail) {
this.sendEmailUtils = sendEmailUtils;
this.clock = clock;
this.packageCreateLimitEmailSubject = packageCreateLimitEmailSubject;
this.packageDomainLimitWarningEmailSubject = packageDomainLimitWarningEmailSubject;
this.packageDomainLimitUpgradeEmailSubject = packageDomainLimitUpgradeEmailSubject;
this.packageCreateLimitEmailBody = packageCreateLimitEmailBody;
this.packageDomainLimitWarningEmailBody = packageDomainLimitWarningEmailBody;
this.packageDomainLimitUpgradeEmailBody = packageDomainLimitUpgradeEmailBody;
this.registrySupportEmail = registrySupportEmail;
}
@Override
public void run() {
tm().transact(this::checkPackages);
}
private void checkPackages() {
ImmutableList<PackagePromotion> packages = tm().loadAllOf(PackagePromotion.class);
ImmutableList.Builder<PackagePromotion> packagesOverCreateLimitBuilder =
new ImmutableList.Builder<>();
ImmutableList.Builder<PackagePromotion> packagesOverActiveDomainsLimitBuilder =
new ImmutableList.Builder<>();
for (PackagePromotion packagePromo : packages) {
Long creates =
(Long)
tm().query(
"SELECT COUNT(*) FROM DomainHistory WHERE current_package_token ="
+ " :token AND modificationTime >= :lastBilling AND type ="
+ " 'DOMAIN_CREATE'")
.setParameter("token", packagePromo.getToken().getKey().toString())
.setParameter("lastBilling", packagePromo.getNextBillingDate().minusYears(1))
.getSingleResult();
if (creates > packagePromo.getMaxCreates()) {
int overage = Ints.saturatedCast(creates) - packagePromo.getMaxCreates();
logger.atInfo().log(
"Package with package token %s has exceeded their max domain creation limit"
+ " by %d name(s).",
packagePromo.getToken().getKey(), overage);
packagesOverCreateLimitBuilder.add(packagePromo);
}
Long activeDomains =
tm().query(
"SELECT COUNT(*) FROM Domain WHERE currentPackageToken = :token"
+ " AND deletionTime = :endOfTime",
Long.class)
.setParameter("token", packagePromo.getToken())
.setParameter("endOfTime", END_OF_TIME)
.getSingleResult();
if (activeDomains > packagePromo.getMaxDomains()) {
int overage = Ints.saturatedCast(activeDomains) - packagePromo.getMaxDomains();
logger.atInfo().log(
"Package with package token %s has exceed their max active domains limit by"
+ " %d name(s).",
packagePromo.getToken().getKey(), overage);
packagesOverActiveDomainsLimitBuilder.add(packagePromo);
}
}
handlePackageCreationOverage(packagesOverCreateLimitBuilder.build());
handleActiveDomainOverage(packagesOverActiveDomainsLimitBuilder.build());
}
private void handlePackageCreationOverage(ImmutableList<PackagePromotion> overageList) {
if (overageList.isEmpty()) {
logger.atInfo().log("Found no packages over their create limit.");
return;
}
logger.atInfo().log("Found %d packages over their create limit.", overageList.size());
for (PackagePromotion packagePromotion : overageList) {
AllocationToken packageToken = tm().loadByKey(packagePromotion.getToken());
Optional<Registrar> registrar =
Registrar.loadByRegistrarIdCached(
Iterables.getOnlyElement(packageToken.getAllowedRegistrarIds()));
if (registrar.isPresent()) {
String body =
String.format(
packageCreateLimitEmailBody,
registrar.get().getRegistrarName(),
packageToken.getToken(),
registrySupportEmail);
sendNotification(packageToken, packageCreateLimitEmailSubject, body, registrar.get());
} else {
throw new IllegalStateException(
String.format("Could not find registrar for package token %s", packageToken));
}
}
}
private void handleActiveDomainOverage(ImmutableList<PackagePromotion> overageList) {
if (overageList.isEmpty()) {
logger.atInfo().log("Found no packages over their active domains limit.");
return;
}
logger.atInfo().log("Found %d packages over their active domains limit.", overageList.size());
for (PackagePromotion packagePromotion : overageList) {
int daysSinceLastNotification =
packagePromotion
.getLastNotificationSent()
.map(sentDate -> Days.daysBetween(sentDate, clock.nowUtc()).getDays())
.orElse(Integer.MAX_VALUE);
if (daysSinceLastNotification < THIRTY_DAYS) {
// Don't send an email if notification was already sent within the last 30
// days
continue;
} else if (daysSinceLastNotification < FORTY_DAYS) {
// Send an upgrade email if last email was between 30 and 40 days ago
sendActiveDomainOverageEmail(/* warning= */ false, packagePromotion);
} else {
// Send a warning email
sendActiveDomainOverageEmail(/* warning= */ true, packagePromotion);
}
}
}
private void sendActiveDomainOverageEmail(boolean warning, PackagePromotion packagePromotion) {
String emailSubject =
warning ? packageDomainLimitWarningEmailSubject : packageDomainLimitUpgradeEmailSubject;
String emailTemplate =
warning ? packageDomainLimitWarningEmailBody : packageDomainLimitUpgradeEmailBody;
AllocationToken packageToken = tm().loadByKey(packagePromotion.getToken());
Optional<Registrar> registrar =
Registrar.loadByRegistrarIdCached(
Iterables.getOnlyElement(packageToken.getAllowedRegistrarIds()));
if (registrar.isPresent()) {
String body =
String.format(
emailTemplate,
registrar.get().getRegistrarName(),
packageToken.getToken(),
registrySupportEmail);
sendNotification(packageToken, emailSubject, body, registrar.get());
tm().put(packagePromotion.asBuilder().setLastNotificationSent(clock.nowUtc()).build());
} else {
throw new IllegalStateException(
String.format("Could not find registrar for package token %s", packageToken));
}
}
private void sendNotification(
AllocationToken packageToken, String subject, String body, Registrar registrar) {
logger.atInfo().log(
String.format(
"Compliance email sent to the %s registrar regarding the package with token" + " %s.",
registrar.getRegistrarName(), packageToken.getToken()));
sendEmailUtils.sendEmail(
subject,
body,
Optional.of(registrySupportEmail),
registrar.getContacts().stream()
.filter(c -> c.getTypes().contains(RegistrarPoc.Type.ADMIN))
.map(RegistrarPoc::getEmailAddress)
.collect(toImmutableList()));
}
}

View File

@@ -44,7 +44,7 @@ import javax.inject.Inject;
/**
* Hard deletes load-test Contacts, Hosts, their subordinate history entries, and the associated
* ForeignKey and EppResourceIndex entities.
* ForeignKey entities.
*
* <p>This only deletes contacts and hosts, NOT domains. To delete domains, use {@link
* DeleteProberDataAction} and pass it the TLD(s) that the load test domains were created on. Note

View File

@@ -21,7 +21,6 @@ import static google.registry.batch.BatchModule.PARAM_DRY_RUN;
import static google.registry.config.RegistryEnvironment.PRODUCTION;
import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_DELETE;
import static google.registry.model.tld.Registries.getTldsOfType;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static google.registry.request.RequestParameters.PARAM_TLDS;
@@ -53,7 +52,7 @@ import org.joda.time.Duration;
/**
* Deletes all prober {@link Domain}s and their subordinate history entries, poll messages, and
* billing events, along with their ForeignKeyDomainIndex and EppResourceIndex entities.
* billing events, along with their ForeignKeyDomainIndex entities.
*/
@Action(
service = Action.Service.BACKEND,
@@ -91,7 +90,7 @@ public class DeleteProberDataAction implements Runnable {
// Note: creationTime must be compared to a Java object (CreateAutoTimestamp) but deletionTime can
// be compared directly to the SQL timestamp (it's a DateTime)
private static final String DOMAIN_QUERY_STRING =
"FROM Domain d WHERE d.tld IN :tlds AND d.fullyQualifiedDomainName NOT LIKE 'nic.%' AND"
"FROM Domain d WHERE d.tld IN :tlds AND d.domainName NOT LIKE 'nic.%' AND"
+ " (d.subordinateHosts IS EMPTY OR d.subordinateHosts IS NULL) AND d.creationTime <"
+ " :creationTimeCutoff AND ((d.creationTime <= :nowAutoTimestamp AND d.deletionTime >"
+ " current_timestamp()) OR d.deletionTime < :nowMinusSoftDeleteDelay) ORDER BY d.repoId";
@@ -101,15 +100,20 @@ public class DeleteProberDataAction implements Runnable {
@Inject DnsQueue dnsQueue;
@Inject @Parameter(PARAM_DRY_RUN) boolean isDryRun;
@Inject
@Parameter(PARAM_DRY_RUN)
boolean isDryRun;
/** List of TLDs to work on. If empty - will work on all TLDs that end with .test. */
@Inject @Parameter(PARAM_TLDS) ImmutableSet<String> tlds;
@Inject
@Parameter(PARAM_TLDS)
ImmutableSet<String> tlds;
@Inject
@Config("registryAdminClientId")
String registryAdminRegistrarId;
@Inject DeleteProberDataAction() {}
@Inject
DeleteProberDataAction() {}
@Override
public void run() {
@@ -135,7 +139,7 @@ public class DeleteProberDataAction implements Runnable {
private void runSqlJob(ImmutableSet<String> deletableTlds) {
AtomicInteger softDeletedDomains = new AtomicInteger();
AtomicInteger hardDeletedDomains = new AtomicInteger();
jpaTm().transact(() -> processDomains(deletableTlds, softDeletedDomains, hardDeletedDomains));
tm().transact(() -> processDomains(deletableTlds, softDeletedDomains, hardDeletedDomains));
logger.atInfo().log(
"%s %d domains.",
isDryRun ? "Would have soft-deleted" : "Soft-deleted", softDeletedDomains.get());
@@ -151,9 +155,8 @@ public class DeleteProberDataAction implements Runnable {
DateTime now = tm().getTransactionTime();
// Scroll through domains, soft-deleting as necessary (very few will be soft-deleted) and
// keeping track of which domains to hard-delete (there can be many, so we batch them up)
ScrollableResults scrollableResult =
jpaTm()
.query(DOMAIN_QUERY_STRING, Domain.class)
try (ScrollableResults scrollableResult =
tm().query(DOMAIN_QUERY_STRING, Domain.class)
.setParameter("tlds", deletableTlds)
.setParameter(
"creationTimeCutoff", CreateAutoTimestamp.create(now.minus(DOMAIN_USED_DURATION)))
@@ -161,28 +164,30 @@ public class DeleteProberDataAction implements Runnable {
.setParameter("nowAutoTimestamp", CreateAutoTimestamp.create(now))
.unwrap(Query.class)
.setCacheMode(CacheMode.IGNORE)
.scroll(ScrollMode.FORWARD_ONLY);
ImmutableList.Builder<String> domainRepoIdsToHardDelete = new ImmutableList.Builder<>();
ImmutableList.Builder<String> hostNamesToHardDelete = new ImmutableList.Builder<>();
for (int i = 1; scrollableResult.next(); i = (i + 1) % BATCH_SIZE) {
Domain domain = (Domain) scrollableResult.get(0);
processDomain(
domain,
domainRepoIdsToHardDelete,
hostNamesToHardDelete,
softDeletedDomains,
hardDeletedDomains);
// Batch the deletion and DB flush + session clearing so we don't OOM
if (i == 0) {
hardDeleteDomainsAndHosts(domainRepoIdsToHardDelete.build(), hostNamesToHardDelete.build());
domainRepoIdsToHardDelete = new ImmutableList.Builder<>();
hostNamesToHardDelete = new ImmutableList.Builder<>();
jpaTm().getEntityManager().flush();
jpaTm().getEntityManager().clear();
.scroll(ScrollMode.FORWARD_ONLY)) {
ImmutableList.Builder<String> domainRepoIdsToHardDelete = new ImmutableList.Builder<>();
ImmutableList.Builder<String> hostNamesToHardDelete = new ImmutableList.Builder<>();
for (int i = 1; scrollableResult.next(); i = (i + 1) % BATCH_SIZE) {
Domain domain = (Domain) scrollableResult.get(0);
processDomain(
domain,
domainRepoIdsToHardDelete,
hostNamesToHardDelete,
softDeletedDomains,
hardDeletedDomains);
// Batch the deletion and DB flush + session clearing, so we don't OOM
if (i == 0) {
hardDeleteDomainsAndHosts(
domainRepoIdsToHardDelete.build(), hostNamesToHardDelete.build());
domainRepoIdsToHardDelete = new ImmutableList.Builder<>();
hostNamesToHardDelete = new ImmutableList.Builder<>();
tm().getEntityManager().flush();
tm().getEntityManager().clear();
}
}
// process the remainder
hardDeleteDomainsAndHosts(domainRepoIdsToHardDelete.build(), hostNamesToHardDelete.build());
}
// process the remainder
hardDeleteDomainsAndHosts(domainRepoIdsToHardDelete.build(), hostNamesToHardDelete.build());
}
private void processDomain(
@@ -219,32 +224,25 @@ public class DeleteProberDataAction implements Runnable {
private void hardDeleteDomainsAndHosts(
ImmutableList<String> domainRepoIds, ImmutableList<String> hostNames) {
jpaTm()
.query("DELETE FROM Host WHERE fullyQualifiedHostName IN :hostNames")
tm().query("DELETE FROM Host WHERE hostName IN :hostNames")
.setParameter("hostNames", hostNames)
.executeUpdate();
jpaTm()
.query("DELETE FROM BillingEvent WHERE domainRepoId IN :repoIds")
tm().query("DELETE FROM BillingEvent WHERE domainRepoId IN :repoIds")
.setParameter("repoIds", domainRepoIds)
.executeUpdate();
jpaTm()
.query("DELETE FROM BillingRecurrence WHERE domainRepoId IN :repoIds")
tm().query("DELETE FROM BillingRecurrence WHERE domainRepoId IN :repoIds")
.setParameter("repoIds", domainRepoIds)
.executeUpdate();
jpaTm()
.query("DELETE FROM BillingCancellation WHERE domainRepoId IN :repoIds")
tm().query("DELETE FROM BillingCancellation WHERE domainRepoId IN :repoIds")
.setParameter("repoIds", domainRepoIds)
.executeUpdate();
jpaTm()
.query("DELETE FROM DomainHistory WHERE domainRepoId IN :repoIds")
tm().query("DELETE FROM DomainHistory WHERE repoId IN :repoIds")
.setParameter("repoIds", domainRepoIds)
.executeUpdate();
jpaTm()
.query("DELETE FROM PollMessage WHERE domainRepoId IN :repoIds")
tm().query("DELETE FROM PollMessage WHERE domainRepoId IN :repoIds")
.setParameter("repoIds", domainRepoIds)
.executeUpdate();
jpaTm()
.query("DELETE FROM Domain WHERE repoId IN :repoIds")
tm().query("DELETE FROM Domain WHERE repoId IN :repoIds")
.setParameter("repoIds", domainRepoIds)
.executeUpdate();
}

View File

@@ -23,7 +23,6 @@ import static google.registry.model.common.Cursor.CursorType.RECURRING_BILLING;
import static google.registry.model.domain.Period.Unit.YEARS;
import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_AUTORENEW;
import static google.registry.persistence.transaction.QueryComposer.Comparator.EQ;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.union;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
@@ -118,25 +117,25 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
do {
final long prevMaxProcessedRecurrenceId = maxProcessedRecurrenceId;
sqlBatchResults =
jpaTm()
.transact(
tm().transact(
() -> {
Set<String> expandedDomains = newHashSet();
int batchBillingEventsSaved = 0;
long maxRecurrenceId = prevMaxProcessedRecurrenceId;
List<Recurring> recurrings =
jpaTm()
.query(
tm().query(
"FROM BillingRecurrence "
+ "WHERE eventTime <= :executeTime "
+ "AND eventTime < recurrenceEndTime "
+ "AND id > :maxProcessedRecurrenceId "
+ "AND recurrenceEndTime > :cursorTime "
+ "AND recurrenceEndTime > :adjustedCursorTime "
+ "ORDER BY id ASC",
Recurring.class)
.setParameter("executeTime", executeTime)
.setParameter("maxProcessedRecurrenceId", prevMaxProcessedRecurrenceId)
.setParameter("cursorTime", cursorTime)
.setParameter(
"adjustedCursorTime",
cursorTime.minus(Registry.DEFAULT_AUTO_RENEW_GRACE_PERIOD))
.setMaxResults(batchSize)
.getResultList();
for (Recurring recurring : recurrings) {
@@ -253,7 +252,7 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
final ImmutableSet<DateTime> billingTimes =
getBillingTimesInScope(eventTimes, cursorTime, executeTime, tld);
VKey<Domain> domainKey = VKey.createSql(Domain.class, recurring.getDomainRepoId());
VKey<Domain> domainKey = VKey.create(Domain.class, recurring.getDomainRepoId());
Iterable<OneTime> oneTimesForDomain;
oneTimesForDomain =
tm().createQueryComposer(OneTime.class)
@@ -279,6 +278,19 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
.setReason("Domain autorenewal by ExpandRecurringBillingEventsAction")
.setRequestedByRegistrar(false)
.setType(DOMAIN_AUTORENEW)
// Note: the following statement seems to not be entirely correct as manual renewal
// during the autorenew grace period also closes out the existing recurrence, but in
// that instance the autorenew history entry should still have the transaction records
// for obvious reasons. It can be argued the history entry should always have the
// transaction record, regardless of what happens afterward. If the domain is deleted
// later during the autorenew grace period, another history entry for the delete would
// record that mutation separately, but the previous autorenew should not have its
// history entry retroactively altered, or in this case have the transaction records
// omitted when its created belatedly (when billing time is in scope). However, since
// we will be rewriting this action and only want to do the absolute minimum change to
// fix it for now, we will leave the current logic in place to avoid any unnecessary
// complications.
//
// Don't write a domain transaction record if the recurrence was
// ended prior to the billing time (i.e. a domain was deleted
// during the autorenew grace period).

View File

@@ -16,7 +16,6 @@ package google.registry.batch;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static google.registry.tools.LockOrUnlockDomainCommand.REGISTRY_LOCK_STATUSES;
@@ -189,7 +188,7 @@ public class RelockDomainAction implements Runnable {
"Domain %s has a pending delete.",
domainName);
checkArgument(
!DateTimeUtils.isAtOrAfter(jpaTm().getTransactionTime(), domain.getDeletionTime()),
!DateTimeUtils.isAtOrAfter(tm().getTransactionTime(), domain.getDeletionTime()),
"Domain %s has been deleted.",
domainName);
checkArgument(

View File

@@ -23,7 +23,6 @@ import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.flogger.FluentLogger;
import google.registry.model.EppResource;
import google.registry.model.ImmutableObject;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Action.Method;
@@ -75,14 +74,11 @@ public class ResaveEntityAction implements Runnable {
"Re-saving entity %s which was enqueued at %s.", resourceKey, requestedTime);
tm().transact(
() -> {
ImmutableObject entity = tm().loadByKey(VKey.create(resourceKey));
tm().put(
(entity instanceof EppResource)
? ((EppResource) entity).cloneProjectedAtTime(tm().getTransactionTime())
: entity);
EppResource entity = tm().loadByKey(VKey.createEppVKeyFromString(resourceKey));
tm().put(entity.cloneProjectedAtTime(tm().getTransactionTime()));
if (!resaveTimes.isEmpty()) {
asyncTaskEnqueuer.enqueueAsyncResave(
VKey.create(resourceKey), requestedTime, resaveTimes);
VKey.createEppVKeyFromString(resourceKey), requestedTime, resaveTimes);
}
});
response.setPayload("Entity re-saved.");

View File

@@ -14,7 +14,7 @@
package google.registry.batch;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static org.apache.http.HttpStatus.SC_INTERNAL_SERVER_ERROR;
import static org.apache.http.HttpStatus.SC_OK;
@@ -77,8 +77,7 @@ public class WipeOutContactHistoryPiiAction implements Runnable {
int numOfWipedEntities = 0;
do {
numOfWipedEntities =
jpaTm()
.transact(
tm().transact(
() ->
wipeOutContactHistoryData(
getNextContactHistoryEntitiesWithPiiBatch(wipeOutTime)));
@@ -113,8 +112,7 @@ public class WipeOutContactHistoryPiiAction implements Runnable {
// email is one of the required fields in EPP, meaning it's initially not null.
// Therefore, checking if it's null is one way to avoid processing contact history entities
// that have been processed previously. Refer to RFC 5733 for more information.
return jpaTm()
.query(
return tm().query(
"FROM ContactHistory WHERE modificationTime < :wipeOutTime " + "AND email IS NOT NULL",
ContactHistory.class)
.setParameter("wipeOutTime", wipeOutTime)
@@ -128,7 +126,7 @@ public class WipeOutContactHistoryPiiAction implements Runnable {
AtomicInteger numOfEntities = new AtomicInteger(0);
contactHistoryEntities.forEach(
contactHistoryEntity -> {
jpaTm().update(contactHistoryEntity.asBuilder().wipeOutPii().build());
tm().update(contactHistoryEntity.asBuilder().wipeOutPii().build());
numOfEntities.incrementAndGet();
});
logger.atInfo().log(

View File

@@ -0,0 +1,122 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch.cannedscript;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.util.RegistrarUtils.normalizeRegistrarId;
import com.google.api.services.admin.directory.Directory;
import com.google.api.services.groupssettings.Groupssettings;
import com.google.common.base.Supplier;
import com.google.common.base.Suppliers;
import com.google.common.base.Throwables;
import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import dagger.Component;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule;
import google.registry.config.CredentialModule.AdcDelegatedCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.groups.DirectoryGroupsConnection;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.util.GoogleCredentialsBundle;
import google.registry.util.UtilsModule;
import java.util.List;
import java.util.Set;
import javax.inject.Singleton;
/**
* Verifies that the credential with the {@link AdcDelegatedCredential} annotation can be used to
* access the Google Workspace Groups API.
*/
// TODO(b/234424397): remove class after credential changes are rolled out.
public class GroupsApiChecker {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final Supplier<GroupsConnectionComponent> COMPONENT_SUPPLIER =
Suppliers.memoize(DaggerGroupsApiChecker_GroupsConnectionComponent::create);
public static void runGroupsApiChecks() {
GroupsConnectionComponent component = COMPONENT_SUPPLIER.get();
DirectoryGroupsConnection groupsConnection = component.groupsConnection();
List<Registrar> registrars =
Streams.stream(Registrar.loadAllCached())
.filter(registrar -> registrar.isLive() && registrar.getType() == Registrar.Type.REAL)
.collect(toImmutableList());
for (Registrar registrar : registrars) {
for (final RegistrarPoc.Type type : RegistrarPoc.Type.values()) {
String groupKey =
String.format(
"%s-%s-contacts@%s",
normalizeRegistrarId(registrar.getRegistrarId()),
type.getDisplayName(),
component.gSuiteDomainName());
try {
Set<String> currentMembers = groupsConnection.getMembersOfGroup(groupKey);
logger.atInfo().log("Found %s members for %s.", currentMembers.size(), groupKey);
} catch (Exception e) {
Throwables.throwIfUnchecked(e);
throw new RuntimeException(e);
}
}
}
}
@Singleton
@Component(
modules = {
ConfigModule.class,
CredentialModule.class,
GroupsApiModule.class,
UtilsModule.class
})
interface GroupsConnectionComponent {
DirectoryGroupsConnection groupsConnection();
@Config("gSuiteDomainName")
String gSuiteDomainName();
}
@Module
static class GroupsApiModule {
@Provides
static Directory provideDirectory(
@AdcDelegatedCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Directory.Builder(
credentialsBundle.getHttpTransport(),
credentialsBundle.getJsonFactory(),
credentialsBundle.getHttpRequestInitializer())
.setApplicationName(projectId)
.build();
}
@Provides
static Groupssettings provideGroupsSettings(
@AdcDelegatedCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Groupssettings.Builder(
credentialsBundle.getHttpTransport(),
credentialsBundle.getJsonFactory(),
credentialsBundle.getHttpRequestInitializer())
.setApplicationName(projectId)
.build();
}
}
}

View File

@@ -18,9 +18,7 @@ import static com.google.common.base.Preconditions.checkArgument;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.io.Resources;
import google.registry.util.Clock;
import google.registry.util.ResourceUtils;
import java.util.regex.Pattern;
import org.apache.avro.generic.GenericRecord;
import org.apache.beam.sdk.io.gcp.bigquery.SchemaAndRecord;
@@ -57,14 +55,6 @@ public class BeamUtils {
}
}
/**
* Returns the {@link String} contents for a file in the {@code sql/} directory relative to a
* class.
*/
public static String getQueryFromFile(Class<?> clazz, String filename) {
return ResourceUtils.readResourceUtf8(Resources.getResource(clazz, "sql/" + filename));
}
/** Creates a beam job name and validates that it conforms to the requirements. */
public static String createJobName(String prefix, Clock clock) {
// Flex template job name must be unique and consists of only characters [-a-z0-9], starting

View File

@@ -15,7 +15,7 @@
package google.registry.beam.common;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.flogger.FluentLogger;
import java.util.List;
@@ -52,7 +52,7 @@ public class DatabaseSnapshot implements AutoCloseable {
}
private DatabaseSnapshot open() {
entityManager = jpaTm().getStandaloneEntityManager();
entityManager = tm().getStandaloneEntityManager();
transaction = entityManager.getTransaction();
transaction.setRollbackOnly();
transaction.begin();

View File

@@ -1,76 +0,0 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.common;
import static com.google.common.base.Verify.verify;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import google.registry.model.contact.Contact;
import google.registry.persistence.transaction.CriteriaQueryBuilder;
import google.registry.persistence.transaction.JpaTransactionManager;
import java.io.Serializable;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.metrics.Counter;
import org.apache.beam.sdk.metrics.Metrics;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.ParDo;
/**
* Toy pipeline that demonstrates how to use {@link JpaTransactionManager} in BEAM pipelines.
*
* <p>This pipeline may also be used as an integration test for {@link RegistryJpaIO.Read} in a
* project with realistic data.
*/
public class JpaDemoPipeline implements Serializable {
public static void main(String[] args) {
RegistryPipelineOptions options =
PipelineOptionsFactory.fromArgs(args).withValidation().as(RegistryPipelineOptions.class);
RegistryPipelineOptions.validateRegistryPipelineOptions(options);
Pipeline pipeline = Pipeline.create(options);
pipeline
.apply(
"Read contacts",
RegistryJpaIO.read(
() -> CriteriaQueryBuilder.create(Contact.class).build(), Contact::getRepoId))
.apply(
"Count Contacts",
ParDo.of(
new DoFn<String, Void>() {
private Counter counter = Metrics.counter("Contacts", "Read");
@ProcessElement
public void processElement() {
// AppEngineEnvironment is needed as long as JPA entity classes still depends
// on Objectify.
int result =
(Integer)
jpaTm()
.transact(
() ->
jpaTm()
.getEntityManager()
.createNativeQuery("select 1;")
.getSingleResult());
verify(result == 1, "Expecting 1, got %s.", result);
counter.inc();
}
}));
pipeline.run();
}
}

View File

@@ -14,25 +14,20 @@
package google.registry.beam.common;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static org.apache.beam.sdk.values.TypeDescriptors.integers;
import com.google.auto.value.AutoValue;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Streams;
import google.registry.beam.common.RegistryQuery.CriteriaQuerySupplier;
import google.registry.model.UpdateAutoTimestamp;
import google.registry.model.UpdateAutoTimestamp.DisableAutoUpdateResource;
import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.persistence.transaction.TransactionManagerFactory;
import java.io.Serializable;
import java.util.Map;
import java.util.Objects;
import java.util.concurrent.ThreadLocalRandom;
import javax.annotation.Nullable;
import javax.persistence.criteria.CriteriaQuery;
import org.apache.beam.sdk.coders.Coder;
import org.apache.beam.sdk.coders.SerializableCoder;
import org.apache.beam.sdk.metrics.Counter;
import org.apache.beam.sdk.metrics.Metrics;
import org.apache.beam.sdk.transforms.Create;
@@ -54,8 +49,8 @@ import org.apache.beam.sdk.values.PCollection;
*
* <p>The {@code JpaTransactionManager} is instantiated once on each pipeline worker VM (through
* {@link RegistryPipelineWorkerInitializer}), made available through the static method {@link
* TransactionManagerFactory#jpaTm()}, and is shared by all threads on the VM. Configuration is
* through {@link RegistryPipelineOptions}.
* TransactionManagerFactory#tm()}, and is shared by all threads on the VM. Configuration is through
* {@link RegistryPipelineOptions}.
*/
public final class RegistryJpaIO {
@@ -135,6 +130,7 @@ public final class RegistryJpaIO {
abstract SerializableFunction<R, T> resultMapper();
@Nullable
abstract Coder<T> coder();
@Nullable
@@ -145,13 +141,16 @@ public final class RegistryJpaIO {
@Override
@SuppressWarnings("deprecation") // Reshuffle still recommended by GCP.
public PCollection<T> expand(PBegin input) {
return input
.apply("Starting " + name(), Create.of((Void) null))
.apply(
"Run query for " + name(),
ParDo.of(new QueryRunner<>(query(), resultMapper(), snapshotId())))
.setCoder(coder())
.apply("Reshuffle", Reshuffle.viaRandomKey());
PCollection<T> output =
input
.apply("Starting " + name(), Create.of((Void) null))
.apply(
"Run query for " + name(),
ParDo.of(new QueryRunner<>(query(), resultMapper(), snapshotId())));
if (coder() != null) {
output = output.setCoder(coder());
}
return output.apply("Reshuffle", Reshuffle.viaRandomKey());
}
public Read<R, T> withName(String name) {
@@ -179,9 +178,7 @@ public final class RegistryJpaIO {
}
static <R, T> Builder<R, T> builder() {
return new AutoValue_RegistryJpaIO_Read.Builder<R, T>()
.name(DEFAULT_NAME)
.coder(SerializableCoder.of(Serializable.class));
return new AutoValue_RegistryJpaIO_Read.Builder<R, T>().name(DEFAULT_NAME);
}
@AutoValue.Builder
@@ -193,7 +190,7 @@ public final class RegistryJpaIO {
abstract Builder<R, T> resultMapper(SerializableFunction<R, T> mapper);
abstract Builder<R, T> coder(Coder coder);
abstract Builder<R, T> coder(Coder<T> coder);
abstract Builder<R, T> snapshotId(@Nullable String sharedSnapshotId);
@@ -233,11 +230,10 @@ public final class RegistryJpaIO {
@ProcessElement
public void processElement(OutputReceiver<T> outputReceiver) {
jpaTm()
.transactNoRetry(
tm().transactNoRetry(
() -> {
if (snapshotId != null) {
jpaTm().setDatabaseSnapshot(snapshotId);
tm().setDatabaseSnapshot(snapshotId);
}
query.stream().map(resultMapper::apply).forEach(outputReceiver::output);
});
@@ -264,46 +260,13 @@ public final class RegistryJpaIO {
public static final int DEFAULT_BATCH_SIZE = 1;
/** The default number of write shard. Please refer to {@link #shards} for more information. */
public static final int DEFAULT_SHARDS = 1;
public abstract String name();
/** Number of elements to be written in one call. */
public abstract int batchSize();
/**
* The number of shards the output should be split into.
*
* <p>This value is a hint to the pipeline runner on the level of parallelism, and should be
* significantly greater than the number of threads working on this transformation (see next
* paragraph for more information). On the other hand, it should not be too large to the point
* that the number of elements per shard is lower than {@link #batchSize()}. As a rule of thumb,
* the following constraint should hold: {@code shards * batchSize * nThreads <=
* inputElementCount}. Although it is not always possible to determine the number of threads
* working on this transform, when the pipeline run is IO-bound, it most likely is close to the
* total number of threads in the pipeline, which is explained below.
*
* <p>With Cloud Dataflow runner, the total number of worker threads in a batch pipeline (which
* includes all existing Registry pipelines) is the number of vCPUs used by the pipeline, and
* can be set by the {@code --maxNumWorkers} and {@code --workerMachineType} parameters. The
* number of worker threads in a streaming pipeline can be set by the {@code --maxNumWorkers}
* and {@code --numberOfWorkerHarnessThreads} parameters.
*
* <p>Note that connections on the database server are a limited resource, therefore the number
* of threads that interact with the database should be set to an appropriate limit. Again, we
* cannot control this number, but can influence it by controlling the total number of threads.
*/
public abstract int shards();
public abstract SerializableFunction<T, Object> jpaConverter();
/**
* Signal to the writer that the {@link UpdateAutoTimestamp} property should be allowed to
* manipulate its value before persistence. The default value is {@code true}.
*/
abstract boolean withUpdateAutoTimestamp();
public Write<T> withName(String name) {
return toBuilder().name(name).build();
}
@@ -312,10 +275,6 @@ public final class RegistryJpaIO {
return toBuilder().batchSize(batchSize).build();
}
public Write<T> withShards(int shards) {
return toBuilder().shards(shards).build();
}
/**
* An optional function that converts the input entities to a form that can be written into the
* database.
@@ -324,19 +283,12 @@ public final class RegistryJpaIO {
return toBuilder().jpaConverter(jpaConverter).build();
}
public Write<T> disableUpdateAutoTimestamp() {
return toBuilder().withUpdateAutoTimestamp(false).build();
}
abstract Builder<T> toBuilder();
@Override
public PCollection<Void> expand(PCollection<T> input) {
return input
.apply(
"Shard data " + name(),
WithKeys.<Integer, T>of(e -> ThreadLocalRandom.current().nextInt(shards()))
.withKeyType(integers()))
.apply("Add key to data " + name(), WithKeys.<Integer, T>of(0).withKeyType(integers()))
// The call to withShardedKey() is performance critical. The resulting transform ensures
// that data is spread evenly across all worker threads.
.apply(
@@ -344,16 +296,14 @@ public final class RegistryJpaIO {
GroupIntoBatches.<Integer, T>ofSize(batchSize()).withShardedKey())
.apply(
"Write in batch for " + name(),
ParDo.of(new SqlBatchWriter<>(name(), jpaConverter(), withUpdateAutoTimestamp())));
ParDo.of(new SqlBatchWriter<>(name(), jpaConverter())));
}
static <T> Builder<T> builder() {
return new AutoValue_RegistryJpaIO_Write.Builder<T>()
.name(DEFAULT_NAME)
.batchSize(DEFAULT_BATCH_SIZE)
.shards(DEFAULT_SHARDS)
.jpaConverter(x -> x)
.withUpdateAutoTimestamp(true);
.jpaConverter(x -> x);
}
@AutoValue.Builder
@@ -363,12 +313,8 @@ public final class RegistryJpaIO {
abstract Builder<T> batchSize(int batchSize);
abstract Builder<T> shards(int jdbcNumConnsHint);
abstract Builder<T> jpaConverter(SerializableFunction<T, Object> jpaConverter);
abstract Builder<T> withUpdateAutoTimestamp(boolean withUpdateAutoTimestamp);
abstract Write<T> build();
}
}
@@ -377,24 +323,15 @@ public final class RegistryJpaIO {
private static class SqlBatchWriter<T> extends DoFn<KV<ShardedKey<Integer>, Iterable<T>>, Void> {
private final Counter counter;
private final SerializableFunction<T, Object> jpaConverter;
private final boolean withAutoTimestamp;
SqlBatchWriter(
String type, SerializableFunction<T, Object> jpaConverter, boolean withAutoTimestamp) {
SqlBatchWriter(String type, SerializableFunction<T, Object> jpaConverter) {
counter = Metrics.counter("SQL_WRITE", type);
this.jpaConverter = jpaConverter;
this.withAutoTimestamp = withAutoTimestamp;
}
@ProcessElement
public void processElement(@Element KV<ShardedKey<Integer>, Iterable<T>> kv) {
if (withAutoTimestamp) {
actuallyProcessElement(kv);
return;
}
try (DisableAutoUpdateResource disable = UpdateAutoTimestamp.disableAutoUpdate()) {
actuallyProcessElement(kv);
}
actuallyProcessElement(kv);
}
private void actuallyProcessElement(@Element KV<ShardedKey<Integer>, Iterable<T>> kv) {
@@ -405,7 +342,11 @@ public final class RegistryJpaIO {
.filter(Objects::nonNull)
.collect(ImmutableList.toImmutableList());
try {
jpaTm().transact(() -> jpaTm().putAll(entities));
tm().transact(
() -> {
// TODO(b/263502442): properly handle creations and blind-writes.
tm().putAll(entities);
});
counter.inc(entities.size());
} catch (RuntimeException e) {
processSingly(entities);
@@ -419,7 +360,11 @@ public final class RegistryJpaIO {
private void processSingly(ImmutableList<Object> entities) {
for (Object entity : entities) {
try {
jpaTm().transact(() -> jpaTm().put(entity));
tm().transact(
() -> {
// TODO(b/263502442): properly handle creations and blind-writes.
tm().put(entity);
});
counter.inc();
} catch (RuntimeException e) {
throw new RuntimeException(toEntityKeyString(entity), e);
@@ -430,14 +375,12 @@ public final class RegistryJpaIO {
/** Returns this entity's primary key field(s) in a string. */
private String toEntityKeyString(Object entity) {
try {
return jpaTm()
.transact(
return tm().transact(
() ->
String.format(
"%s_%s",
entity.getClass().getSimpleName(),
jpaTm()
.getEntityManager()
tm().getEntityManager()
.getEntityManagerFactory()
.getPersistenceUnitUtil()
.getIdentifier(entity)));

View File

@@ -20,9 +20,7 @@ import dagger.Lazy;
import google.registry.config.CredentialModule;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.model.domain.Domain;
import google.registry.persistence.PersistenceModule;
import google.registry.persistence.PersistenceModule.BeamBulkQueryJpaTm;
import google.registry.persistence.PersistenceModule.BeamJpaTm;
import google.registry.persistence.PersistenceModule.BeamReadOnlyReplicaJpaTm;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
@@ -52,14 +50,6 @@ public interface RegistryPipelineComponent {
@BeamJpaTm
Lazy<JpaTransactionManager> getJpaTransactionManager();
/**
* Returns a {@link JpaTransactionManager} optimized for bulk loading multi-level JPA entities
* ({@link Domain} and {@link google.registry.model.domain.DomainHistory}). Please refer to {@link
* google.registry.model.bulkquery.BulkQueryEntities} for more information.
*/
@BeamBulkQueryJpaTm
Lazy<JpaTransactionManager> getBulkQueryJpaTransactionManager();
/**
* A {@link JpaTransactionManager} that uses the Postgres read-only replica if configured (uses
* the standard DB otherwise).

View File

@@ -14,8 +14,8 @@
package google.registry.beam.common;
import google.registry.beam.common.RegistryJpaIO.Write;
import google.registry.config.RegistryEnvironment;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.persistence.PersistenceModule.JpaTransactionManagerType;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import java.util.Objects;
@@ -28,9 +28,9 @@ import org.apache.beam.sdk.options.Description;
* Defines Nomulus-specific pipeline options, e.g. JPA configurations.
*
* <p>When using the Cloud Dataflow runner, users are recommended to set an upper bound on active
* database connections by setting the pipeline worker options including {@code --maxNumWorkers},
* {@code workerMachineType}, and {@code numberOfWorkerHarnessThreads}. Please refer to {@link
* Write#shards()} for more information.
* database connections by setting the max number of pipeline worker threads using {@code
* --maxNumWorkers} and {@code workerMachineType} for batch pipelines, or {@code --maxNumWorkers}
* and {@code --numberOfWorkerHarnessThreads} for streaming pipelines.
*/
public interface RegistryPipelineOptions extends GcpOptions {
@@ -57,13 +57,16 @@ public interface RegistryPipelineOptions extends GcpOptions {
void setSqlWriteBatchSize(int sqlWriteBatchSize);
@DeleteAfterMigration
@Description(
"Number of shards to create out of the data before writing to the SQL database. Please refer "
+ "to the Javadoc of RegistryJpaIO.Write.shards() for how to choose this value.")
@Default.Integer(100)
int getSqlWriteShards();
"Whether to use self allocated primary IDs when building entities. This should only be used"
+ " when the IDs are not significant and the resulting entities are not persisted back to"
+ " the database. Use with caution as self allocated IDs are not unique across workers,"
+ " and persisting entities with these IDs can be dangerous.")
@Default.Boolean(false)
boolean getUseSelfAllocatedId();
void setSqlWriteShards(int maxConcurrentSqlWriters);
void setUseSelfAllocatedId(boolean useSelfAllocatedId);
static RegistryPipelineComponent toRegistryPipelineComponent(RegistryPipelineOptions options) {
return DaggerRegistryPipelineComponent.builder()

View File

@@ -21,7 +21,7 @@ import com.google.common.flogger.FluentLogger;
import dagger.Lazy;
import google.registry.config.RegistryEnvironment;
import google.registry.config.SystemPropertySetter;
import google.registry.model.AppEngineEnvironment;
import google.registry.model.IdService;
import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.persistence.transaction.TransactionManagerFactory;
import org.apache.beam.sdk.harness.JvmInitializer;
@@ -53,9 +53,6 @@ public class RegistryPipelineWorkerInitializer implements JvmInitializer {
toRegistryPipelineComponent(registryOptions);
Lazy<JpaTransactionManager> transactionManagerLazy;
switch (registryOptions.getJpaTransactionManagerType()) {
case BULK_QUERY:
transactionManagerLazy = registryPipelineComponent.getBulkQueryJpaTransactionManager();
break;
case READ_ONLY_REPLICA:
transactionManagerLazy =
registryPipelineComponent.getReadOnlyReplicaJpaTransactionManager();
@@ -65,12 +62,16 @@ public class RegistryPipelineWorkerInitializer implements JvmInitializer {
transactionManagerLazy = registryPipelineComponent.getJpaTransactionManager();
}
TransactionManagerFactory.setJpaTmOnBeamWorker(transactionManagerLazy::get);
// Masquerade all threads as App Engine threads so we can create Ofy keys in the pipeline. Also
// loads all ofy entities.
new AppEngineEnvironment("s~" + registryPipelineComponent.getProjectId())
.setEnvironmentForAllThreads();
// Set the system property so that we can call IdService.allocateId() without access to
// datastore.
SystemPropertySetter.PRODUCTION_IMPL.setProperty(PROPERTY, "true");
// Use self-allocated IDs if requested. Note that this inevitably results in duplicate IDs from
// multiple workers, which can also collide with existing IDs in the database. So they cannot be
// dependent upon for comparison or anything significant. The resulting entities can never be
// persisted back into the database. This is a stop-gap measure that should only be used when
// you need to create Buildables in Beam, but do not have control over how the IDs are
// allocated, and you don't care about the generated IDs as long
// as you can build the entities.
if (registryOptions.getUseSelfAllocatedId()) {
IdService.setForceUseSelfAllocatedId();
}
}
}

View File

@@ -14,7 +14,7 @@
package google.registry.beam.common;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import google.registry.persistence.transaction.JpaTransactionManager;
import java.io.Serializable;
@@ -53,7 +53,7 @@ public interface RegistryQuery<T> extends Serializable {
static <T> RegistryQuery<T> createQuery(
String sql, @Nullable Map<String, Object> parameters, boolean nativeQuery) {
return () -> {
EntityManager entityManager = jpaTm().getEntityManager();
EntityManager entityManager = tm().getEntityManager();
Query query =
nativeQuery ? entityManager.createNativeQuery(sql) : entityManager.createQuery(sql);
if (parameters != null) {
@@ -76,7 +76,7 @@ public interface RegistryQuery<T> extends Serializable {
String jpql, @Nullable Map<String, Object> parameters, Class<T> clazz) {
return () -> {
// TODO(b/193662898): switch to jpaTm().query() when it can properly detach loaded entities.
EntityManager entityManager = jpaTm().getEntityManager();
EntityManager entityManager = tm().getEntityManager();
TypedQuery<T> query = entityManager.createQuery(jpql, clazz);
if (parameters != null) {
parameters.forEach(query::setParameter);
@@ -98,7 +98,7 @@ public interface RegistryQuery<T> extends Serializable {
static <T> RegistryQuery<T> createQuery(CriteriaQuerySupplier<T> criteriaQuery) {
return () -> {
// TODO(b/193662898): switch to jpaTm().query() when it can properly detach loaded entities.
EntityManager entityManager = jpaTm().getEntityManager();
EntityManager entityManager = tm().getEntityManager();
TypedQuery<T> query = entityManager.createQuery(criteriaQuery.get());
JpaTransactionManager.setQueryFetchSize(query, QUERY_FETCH_SIZE);
return query.getResultStream().map(e -> detach(entityManager, e));

View File

@@ -64,7 +64,7 @@ public abstract class BillingEvent implements Serializable {
"amount",
"flags");
/** Returns the unique Objectify ID for the {@code OneTime} associated with this event. */
/** Returns the unique ID for the {@code OneTime} associated with this event. */
abstract long id();
/** Returns the UTC DateTime this event becomes billable. */

View File

@@ -15,7 +15,6 @@
package google.registry.beam.invoicing;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.beam.BeamUtils.getQueryFromFile;
import static org.apache.beam.sdk.values.TypeDescriptors.strings;
import com.google.common.flogger.FluentLogger;
@@ -29,6 +28,7 @@ import google.registry.model.registrar.Registrar;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.reporting.billing.BillingModule;
import google.registry.util.DomainNameUtils;
import google.registry.util.ResourceUtils;
import google.registry.util.SqlTemplate;
import java.io.Serializable;
import java.time.YearMonth;
@@ -37,6 +37,7 @@ import java.util.Optional;
import java.util.regex.Pattern;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.PipelineResult;
import org.apache.beam.sdk.coders.SerializableCoder;
import org.apache.beam.sdk.coders.StringUtf8Coder;
import org.apache.beam.sdk.io.FileIO;
import org.apache.beam.sdk.io.TextIO;
@@ -93,8 +94,9 @@ public class InvoicingPipeline implements Serializable {
static PCollection<BillingEvent> readFromCloudSql(
InvoicingPipelineOptions options, Pipeline pipeline) {
Read<Object[], BillingEvent> read =
RegistryJpaIO.read(
makeCloudSqlQuery(options.getYearMonth()), false, row -> parseRow(row).orElse(null));
RegistryJpaIO.<Object[], BillingEvent>read(
makeCloudSqlQuery(options.getYearMonth()), false, row -> parseRow(row).orElse(null))
.withCoder(SerializableCoder.of(BillingEvent.class));
PCollection<BillingEvent> billingEventsWithNulls =
pipeline.apply("Read BillingEvents from Cloud SQL", read);
@@ -207,7 +209,8 @@ public class InvoicingPipeline implements Serializable {
YearMonth endMonth = YearMonth.parse(yearMonth).plusMonths(1);
String queryWithComments =
SqlTemplate.create(
getQueryFromFile(InvoicingPipeline.class, "cloud_sql_billing_events.sql"))
ResourceUtils.readResourceUtf8(
InvoicingPipeline.class, "sql/cloud_sql_billing_events.sql"))
.put("FIRST_TIMESTAMP_OF_MONTH", yearMonth + "-01")
.put(
"LAST_TIMESTAMP_OF_MONTH",

View File

@@ -26,7 +26,7 @@ import static google.registry.beam.rde.RdePipeline.TupleTags.REFERENCED_HOSTS;
import static google.registry.beam.rde.RdePipeline.TupleTags.REVISION_ID;
import static google.registry.beam.rde.RdePipeline.TupleTags.SUPERORDINATE_DOMAINS;
import static google.registry.model.reporting.HistoryEntryDao.RESOURCE_TYPES_TO_HISTORY_TYPES;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static org.apache.beam.sdk.values.TypeDescriptors.kvs;
import com.google.common.collect.ImmutableList;
@@ -55,7 +55,7 @@ import google.registry.model.rde.RdeMode;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.Registrar.Type;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.HistoryEntryDao;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.persistence.VKey;
import google.registry.rde.DepositFragment;
@@ -71,11 +71,9 @@ import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;
import java.lang.reflect.InvocationTargetException;
import java.util.HashSet;
import javax.inject.Inject;
import javax.inject.Singleton;
import javax.persistence.IdClass;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.PipelineResult;
import org.apache.beam.sdk.coders.KvCoder;
@@ -128,7 +126,7 @@ import org.joda.time.DateTime;
* <h2>{@link EppResource}</h2>
*
* All EPP resources are loaded from the corresponding {@link HistoryEntry}, which has the resource
* embedded. In general we find most recent history entry before watermark and filter out the ones
* embedded. In general, we find most recent history entry before watermark and filter out the ones
* that are soft-deleted by watermark. The history is emitted as pairs of (resource repo ID: history
* revision ID) from the SQL query.
*
@@ -164,7 +162,7 @@ import org.joda.time.DateTime;
*
* The (pending deposit: deposit fragment) pairs from different resources are combined and grouped
* by pending deposit. For each pending deposit, all the relevant deposit fragments are written into
* a encrypted file stored on GCS. The filename is uniquely determined by the Beam job ID so there
* an encrypted file stored on GCS. The filename is uniquely determined by the Beam job ID so there
* is no need to lock the GCS write operation to prevent stomping. The cursor for staging the
* pending deposit is then rolled forward, and the next action is enqueued. The latter two
* operations are performed in a transaction so the cursor is rolled back if enqueueing failed.
@@ -172,6 +170,7 @@ import org.joda.time.DateTime;
* @see <a href="https://cloud.google.com/dataflow/docs/guides/templates/using-flex-templates">Using
* Flex Templates</a>
*/
@SuppressWarnings("ALL")
@Singleton
public class RdePipeline implements Serializable {
@@ -191,16 +190,6 @@ public class RdePipeline implements Serializable {
private static final ImmutableSet<Type> IGNORED_REGISTRAR_TYPES =
Sets.immutableEnumSet(Registrar.Type.MONITORING, Registrar.Type.TEST);
// The field name of the EPP resource embedded in its corresponding history entry.
private static final ImmutableMap<Class<? extends HistoryEntry>, String> EPP_RESOURCE_FIELD_NAME =
ImmutableMap.of(
DomainHistory.class,
"domainBase",
ContactHistory.class,
"contactBase",
HostHistory.class,
"hostBase");
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
@Inject
@@ -301,10 +290,11 @@ public class RdePipeline implements Serializable {
.apply(
"Read all production Registrars",
RegistryJpaIO.read(
"SELECT clientIdentifier FROM Registrar WHERE type NOT IN (:types)",
ImmutableMap.of("types", IGNORED_REGISTRAR_TYPES),
String.class,
id -> VKey.createSql(Registrar.class, id)))
"SELECT registrarId FROM Registrar WHERE type NOT IN (:types)",
ImmutableMap.of("types", IGNORED_REGISTRAR_TYPES),
String.class,
x -> x)
.withCoder(StringUtf8Coder.of()))
.apply(
"Marshall Registrar into DepositFragment",
FlatMapElements.into(
@@ -312,9 +302,10 @@ public class RdePipeline implements Serializable {
TypeDescriptor.of(PendingDeposit.class),
TypeDescriptor.of(DepositFragment.class)))
.via(
(VKey<Registrar> key) -> {
(String registrarRepoId) -> {
VKey<Registrar> key = VKey.create(Registrar.class, registrarRepoId);
includedRegistrarCounter.inc();
Registrar registrar = jpaTm().transact(() -> jpaTm().loadByKey(key));
Registrar registrar = tm().transact(() -> tm().loadByKey(key));
DepositFragment fragment = marshaller.marshalRegistrar(registrar);
ImmutableSet<KV<PendingDeposit, DepositFragment>> fragments =
pendingDeposits.stream()
@@ -335,31 +326,24 @@ public class RdePipeline implements Serializable {
*/
private <T extends HistoryEntry> PCollection<KV<String, Long>> getMostRecentHistoryEntries(
Pipeline pipeline, Class<T> historyClass) {
String repoIdFieldName = HistoryEntryDao.REPO_ID_FIELD_NAMES.get(historyClass);
String resourceFieldName = EPP_RESOURCE_FIELD_NAME.get(historyClass);
return pipeline
.apply(
String.format("Load most recent %s", historyClass.getSimpleName()),
RegistryJpaIO.read(
("SELECT %repoIdField%, id FROM %entity% WHERE (%repoIdField%, modificationTime)"
+ " IN (SELECT %repoIdField%, MAX(modificationTime) FROM %entity% WHERE"
+ " modificationTime <= :watermark GROUP BY %repoIdField%) AND"
+ " %resourceField%.deletionTime > :watermark AND"
+ " COALESCE(%resourceField%.creationClientId, '') NOT LIKE 'prober-%' AND"
+ " COALESCE(%resourceField%.currentSponsorClientId, '') NOT LIKE 'prober-%'"
+ " AND COALESCE(%resourceField%.lastEppUpdateClientId, '') NOT LIKE"
return pipeline.apply(
String.format("Load most recent %s", historyClass.getSimpleName()),
RegistryJpaIO.read(
("SELECT repoId, revisionId FROM %entity% WHERE (repoId, modificationTime) IN"
+ " (SELECT repoId, MAX(modificationTime) FROM %entity% WHERE"
+ " modificationTime <= :watermark GROUP BY repoId) AND resource.deletionTime"
+ " > :watermark AND COALESCE(resource.creationRegistrarId, '') NOT LIKE"
+ " 'prober-%' AND COALESCE(resource.currentSponsorRegistrarId, '') NOT LIKE"
+ " 'prober-%' AND COALESCE(resource.lastEppUpdateRegistrarId, '') NOT LIKE"
+ " 'prober-%' "
+ (historyClass == DomainHistory.class
? "AND %resourceField%.tld IN "
+ "(SELECT id FROM Tld WHERE tldType = 'REAL')"
? "AND resource.tld IN " + "(SELECT id FROM Tld WHERE tldType = 'REAL')"
: ""))
.replace("%entity%", historyClass.getSimpleName())
.replace("%repoIdField%", repoIdFieldName)
.replace("%resourceField%", resourceFieldName),
.replace("%entity%", historyClass.getSimpleName()),
ImmutableMap.of("watermark", watermark),
Object[].class,
row -> KV.of((String) row[0], (long) row[1])))
.setCoder(KvCoder.of(StringUtf8Coder.of(), VarLongCoder.of()));
row -> KV.of((String) row[0], (long) row[1]))
.withCoder(KvCoder.of(StringUtf8Coder.of(), VarLongCoder.of())));
}
private <T extends HistoryEntry> EppResource loadResourceByHistoryEntryId(
@@ -379,38 +363,26 @@ public class RdePipeline implements Serializable {
checkState(
dedupedIds.size() == 1,
"Multiple unique revision IDs detected for %s repo ID %s: %s",
EPP_RESOURCE_FIELD_NAME.get(historyEntryClazz),
historyEntryClazz.getSimpleName(),
repoId,
ids);
logger.atSevere().log(
"Duplicate revision IDs detected for %s repo ID %s: %s",
EPP_RESOURCE_FIELD_NAME.get(historyEntryClazz), repoId, ids);
historyEntryClazz.getSimpleName(), repoId, ids);
}
return loadResourceByHistoryEntryId(historyEntryClazz, repoId, ids.get(0));
}
private <T extends HistoryEntry> EppResource loadResourceByHistoryEntryId(
Class<T> historyEntryClazz, String repoId, long revisionId) {
try {
Class<?> idClazz = historyEntryClazz.getAnnotation(IdClass.class).value();
Serializable idObject =
(Serializable)
idClazz.getConstructor(String.class, long.class).newInstance(repoId, revisionId);
return jpaTm()
.transact(() -> jpaTm().loadByKey(VKey.createSql(historyEntryClazz, idObject)))
.getResourceAtPointInTime()
.map(resource -> resource.cloneProjectedAtTime(watermark))
.get();
} catch (NoSuchMethodException
| InvocationTargetException
| InstantiationException
| IllegalAccessException e) {
throw new RuntimeException(
String.format(
"Cannot load resource from %s with repoId %s and revisionId %s",
historyEntryClazz.getSimpleName(), repoId, revisionId),
e);
}
return tm().transact(
() ->
tm().loadByKey(
VKey.create(historyEntryClazz, new HistoryEntryId(repoId, revisionId))))
.getResourceAtPointInTime()
.map(resource -> resource.cloneProjectedAtTime(watermark))
.get();
}
/**
@@ -495,12 +467,12 @@ public class RdePipeline implements Serializable {
// Contacts and hosts are only deposited in RDE, not BRDA.
if (pendingDeposit.mode() == RdeMode.FULL) {
HashSet<Serializable> contacts = new HashSet<>();
contacts.add(domain.getAdminContact().getSqlKey());
contacts.add(domain.getTechContact().getSqlKey());
contacts.add(domain.getRegistrant().getSqlKey());
contacts.add(domain.getAdminContact().getKey());
contacts.add(domain.getTechContact().getKey());
contacts.add(domain.getRegistrant().getKey());
// Billing contact is not mandatory.
if (domain.getBillingContact() != null) {
contacts.add(domain.getBillingContact().getSqlKey());
contacts.add(domain.getBillingContact().getKey());
}
referencedContactCounter.inc(contacts.size());
contacts.forEach(
@@ -518,7 +490,7 @@ public class RdePipeline implements Serializable {
.get(REFERENCED_HOSTS)
.output(
KV.of(
(String) hostKey.getSqlKey(),
(String) hostKey.getKey(),
pendingDeposit)));
}
}
@@ -591,7 +563,7 @@ public class RdePipeline implements Serializable {
// The output are pairs of
// (superordinateDomainRepoId,
// (subordinateHostRepoId, (pendingDeposit, revisionId))).
KV.of((String) host.getSuperordinateDomain().getSqlKey(), kv));
KV.of((String) host.getSuperordinateDomain().getKey(), kv));
} else {
externalHostCounter.inc();
DepositFragment fragment = marshaller.marshalExternalHost(host);
@@ -698,8 +670,8 @@ public class RdePipeline implements Serializable {
}
/**
* Encodes the pending deposit set in an URL safe string that is sent to the pipeline worker by
* the pipeline launcher as a pipeline option.
* Encodes the pending deposit set in a URL safe string that is sent to the pipeline worker by the
* pipeline launcher as a pipeline option.
*/
public static String encodePendingDeposits(ImmutableSet<PendingDeposit> pendingDeposits)
throws IOException {
@@ -715,6 +687,14 @@ public class RdePipeline implements Serializable {
PipelineOptionsFactory.register(RdePipelineOptions.class);
RdePipelineOptions options =
PipelineOptionsFactory.fromArgs(args).withValidation().as(RdePipelineOptions.class);
// We need to self allocate the IDs because the pipeline creates EPP resources from history
// entries and projects them to watermark. These buildable entities would otherwise request an
// ID from datastore, which Beam does not have access to. The IDs are not included in the
// deposits or are these entities persisted back to the database, so it is OK to use a self
// allocated ID to get around the limitations of beam.
options.setUseSelfAllocatedId(true);
RegistryPipelineOptions.validateRegistryPipelineOptions(options);
options.setIsolationOverride(TransactionIsolationLevel.TRANSACTION_READ_COMMITTED);
DaggerRdePipeline_RdePipelineComponent.builder().options(options).build().rdePipeline().run();

View File

@@ -14,11 +14,14 @@
package google.registry.beam.resave;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static org.apache.beam.sdk.values.TypeDescriptors.integers;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Streams;
import google.registry.beam.common.RegistryJpaIO;
import google.registry.beam.common.RegistryJpaIO.Read;
import google.registry.model.EppResource;
@@ -27,12 +30,12 @@ import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainBase;
import google.registry.model.host.Host;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.persistence.transaction.CriteriaQueryBuilder;
import google.registry.persistence.VKey;
import google.registry.util.DateTimeUtils;
import java.io.Serializable;
import java.util.concurrent.ThreadLocalRandom;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.PipelineResult;
import org.apache.beam.sdk.coders.StringUtf8Coder;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.GroupIntoBatches;
@@ -69,7 +72,7 @@ public class ResaveAllEppResourcesPipeline implements Serializable {
* multiple times, and to avoid projecting and resaving the same domain multiple times.
*/
private static final String DOMAINS_TO_PROJECT_QUERY =
"FROM Domain d WHERE (d.transferData.transferStatus = 'PENDING' AND"
"SELECT repoId FROM Domain d WHERE (d.transferData.transferStatus = 'PENDING' AND"
+ " d.transferData.pendingTransferExpirationTime < current_timestamp()) OR"
+ " (d.registrationExpirationTime < current_timestamp() AND d.deletionTime ="
+ " (:END_OF_TIME)) OR (EXISTS (SELECT 1 FROM GracePeriod gp WHERE gp.domainRepoId ="
@@ -88,7 +91,6 @@ public class ResaveAllEppResourcesPipeline implements Serializable {
}
void setupPipeline(Pipeline pipeline) {
options.setIsolationOverride(TransactionIsolationLevel.TRANSACTION_READ_COMMITTED);
if (options.getFast()) {
fastResaveContacts(pipeline);
fastResaveDomains(pipeline);
@@ -99,13 +101,14 @@ public class ResaveAllEppResourcesPipeline implements Serializable {
/** Projects to the current time and saves any contacts with expired transfers. */
private void fastResaveContacts(Pipeline pipeline) {
Read<Contact, Contact> read =
Read<String, String> repoIdRead =
RegistryJpaIO.read(
"FROM Contact WHERE transferData.transferStatus = 'PENDING' AND"
+ " transferData.pendingTransferExpirationTime < current_timestamp()",
Contact.class,
c -> c);
projectAndResaveResources(pipeline, Contact.class, read);
"SELECT repoId FROM Contact WHERE transferData.transferStatus = 'PENDING' AND"
+ " transferData.pendingTransferExpirationTime < current_timestamp()",
String.class,
r -> r)
.withCoder(StringUtf8Coder.of());
projectAndResaveResources(pipeline, Contact.class, repoIdRead);
}
/**
@@ -116,61 +119,71 @@ public class ResaveAllEppResourcesPipeline implements Serializable {
* DomainBase#cloneProjectedAtTime(DateTime)}.
*/
private void fastResaveDomains(Pipeline pipeline) {
Read<Domain, Domain> read =
Read<String, String> repoIdRead =
RegistryJpaIO.read(
DOMAINS_TO_PROJECT_QUERY,
ImmutableMap.of("END_OF_TIME", DateTimeUtils.END_OF_TIME),
Domain.class,
d -> d);
projectAndResaveResources(pipeline, Domain.class, read);
DOMAINS_TO_PROJECT_QUERY,
ImmutableMap.of("END_OF_TIME", DateTimeUtils.END_OF_TIME),
String.class,
r -> r)
.withCoder(StringUtf8Coder.of());
projectAndResaveResources(pipeline, Domain.class, repoIdRead);
}
/** Projects all resources to the current time and saves them. */
private <T extends EppResource> void forceResaveAllResources(Pipeline pipeline, Class<T> clazz) {
Read<T, T> read = RegistryJpaIO.read(() -> CriteriaQueryBuilder.create(clazz).build());
projectAndResaveResources(pipeline, clazz, read);
Read<String, String> repoIdRead =
RegistryJpaIO.read(
// Note: cannot use SQL parameters for the table name
String.format("SELECT repoId FROM %s", clazz.getSimpleName()), String.class, r -> r)
.withCoder(StringUtf8Coder.of());
projectAndResaveResources(pipeline, clazz, repoIdRead);
}
/** Projects and re-saves the result of the provided {@link Read}. */
/** Projects and re-saves all resources with repo IDs provided by the {@link Read}. */
private <T extends EppResource> void projectAndResaveResources(
Pipeline pipeline, Class<T> clazz, Read<?, T> read) {
int numShards = options.getSqlWriteShards();
Pipeline pipeline, Class<T> clazz, Read<?, String> repoIdRead) {
int batchSize = options.getSqlWriteBatchSize();
String className = clazz.getSimpleName();
pipeline
.apply("Read " + className, read)
.apply("Read " + className, repoIdRead)
.apply(
"Shard data for class" + className,
WithKeys.<Integer, T>of(e -> ThreadLocalRandom.current().nextInt(numShards))
.withKeyType(integers()))
WithKeys.<Integer, String>of(0).withKeyType(integers()))
.apply(
"Group into batches for class" + className,
GroupIntoBatches.<Integer, T>ofSize(batchSize).withShardedKey())
.apply("Map " + className + " to now", ParDo.of(new BatchedProjectionFunction<>()))
GroupIntoBatches.<Integer, String>ofSize(batchSize).withShardedKey())
.apply(
"Write transformed " + className,
RegistryJpaIO.<EppResource>write()
.withName("Write transformed " + className)
.withBatchSize(batchSize)
.withShards(numShards));
"Load, map, and save " + className,
ParDo.of(new BatchedLoadProjectAndSaveFunction(clazz)));
}
private static class BatchedProjectionFunction<T extends EppResource>
extends DoFn<KV<ShardedKey<Integer>, Iterable<T>>, EppResource> {
/** Function that loads, projects, and saves resources all in the same transaction. */
private static class BatchedLoadProjectAndSaveFunction
extends DoFn<KV<ShardedKey<Integer>, Iterable<String>>, Void> {
private final Class<? extends EppResource> clazz;
private BatchedLoadProjectAndSaveFunction(Class<? extends EppResource> clazz) {
this.clazz = clazz;
}
@ProcessElement
public void processElement(
@Element KV<ShardedKey<Integer>, Iterable<T>> element,
OutputReceiver<EppResource> outputReceiver) {
jpaTm()
.transact(
() ->
element
.getValue()
.forEach(
resource ->
outputReceiver.output(
resource.cloneProjectedAtTime(jpaTm().getTransactionTime()))));
@Element KV<ShardedKey<Integer>, Iterable<String>> element,
OutputReceiver<Void> outputReceiver) {
tm().transact(
() -> {
DateTime now = tm().getTransactionTime();
ImmutableList<VKey<? extends EppResource>> keys =
Streams.stream(element.getValue())
.map(repoId -> VKey.create(clazz, repoId))
.collect(toImmutableList());
ImmutableList<EppResource> mappedResources =
tm().loadByKeys(keys).values().stream()
.map(r -> r.cloneProjectedAtTime(now))
.collect(toImmutableList());
tm().putAll(mappedResources);
});
}
}
@@ -180,6 +193,7 @@ public class ResaveAllEppResourcesPipeline implements Serializable {
PipelineOptionsFactory.fromArgs(args)
.withValidation()
.as(ResaveAllEppResourcesPipelineOptions.class);
options.setIsolationOverride(TransactionIsolationLevel.TRANSACTION_REPEATABLE_READ);
new ResaveAllEppResourcesPipeline(options).run();
}
}

View File

@@ -75,8 +75,8 @@ public class SafeBrowsingTransforms {
private final String apiKey;
/**
* Maps a domain name's {@code fullyQualifiedDomainName} to its corresponding {@link
* DomainNameInfo} to facilitate batching SafeBrowsing API requests.
* Maps a domain name's {@code domainName} to its corresponding {@link DomainNameInfo} to
* facilitate batching SafeBrowsing API requests.
*/
private final Map<String, DomainNameInfo> domainNameInfoBuffer =
new LinkedHashMap<>(BATCH_SIZE);
@@ -186,8 +186,8 @@ public class SafeBrowsingTransforms {
private JSONObject createRequestBody() throws JSONException {
// Accumulate all domain names to evaluate.
JSONArray threatArray = new JSONArray();
for (String fullyQualifiedDomainName : domainNameInfoBuffer.keySet()) {
threatArray.put(new JSONObject().put("url", fullyQualifiedDomainName));
for (String domainName : domainNameInfoBuffer.keySet()) {
threatArray.put(new JSONObject().put("url", domainName));
}
// Construct the JSON request body
return new JSONObject()

View File

@@ -15,7 +15,7 @@
package google.registry.beam.spec11;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.auto.value.AutoValue;
import com.google.common.collect.ImmutableSet;
@@ -26,6 +26,7 @@ import google.registry.beam.common.RegistryJpaIO;
import google.registry.beam.common.RegistryJpaIO.Read;
import google.registry.beam.spec11.SafeBrowsingTransforms.EvaluateSafeBrowsingFn;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.model.IdService;
import google.registry.model.domain.Domain;
import google.registry.model.reporting.Spec11ThreatMatch;
import google.registry.model.reporting.Spec11ThreatMatch.ThreatType;
@@ -37,12 +38,15 @@ import java.io.Serializable;
import javax.inject.Singleton;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.PipelineResult;
import org.apache.beam.sdk.coders.KvCoder;
import org.apache.beam.sdk.coders.StringUtf8Coder;
import org.apache.beam.sdk.io.TextIO;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.GroupByKey;
import org.apache.beam.sdk.transforms.MapElements;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.transforms.Reshuffle;
import org.apache.beam.sdk.values.KV;
import org.apache.beam.sdk.values.PCollection;
import org.apache.beam.sdk.values.TypeDescriptor;
@@ -112,11 +116,12 @@ public class Spec11Pipeline implements Serializable {
static PCollection<DomainNameInfo> readFromCloudSql(Pipeline pipeline) {
Read<Object[], KV<String, String>> read =
RegistryJpaIO.read(
"select d.repoId, r.emailAddress from Domain d join Registrar r on"
+ " d.currentSponsorClientId = r.clientIdentifier where r.type = 'REAL' and"
+ " d.deletionTime > now()",
false,
Spec11Pipeline::parseRow);
"select d.repoId, r.emailAddress from Domain d join Registrar r on"
+ " d.currentSponsorRegistrarId = r.registrarId where r.type = 'REAL' and"
+ " d.deletionTime > now()",
false,
Spec11Pipeline::parseRow)
.withCoder(KvCoder.of(StringUtf8Coder.of(), StringUtf8Coder.of()));
return pipeline
.apply("Read active domains from Cloud SQL", read)
@@ -128,11 +133,8 @@ public class Spec11Pipeline implements Serializable {
public void processElement(
@Element KV<String, String> input, OutputReceiver<DomainNameInfo> output) {
Domain domain =
jpaTm()
.transact(
() ->
jpaTm()
.loadByKey(VKey.createSql(Domain.class, input.getKey())));
tm().transact(
() -> tm().loadByKey(VKey.create(Domain.class, input.getKey())));
String emailAddress = input.getValue();
if (emailAddress == null) {
emailAddress = "";
@@ -154,26 +156,36 @@ public class Spec11Pipeline implements Serializable {
static void saveToSql(
PCollection<KV<DomainNameInfo, ThreatMatch>> threatMatches, Spec11PipelineOptions options) {
String transformId = "Spec11 Threat Matches";
LocalDate date = LocalDate.parse(options.getDate(), ISODateTimeFormat.date());
threatMatches.apply(
"Write to Sql: " + transformId,
RegistryJpaIO.<KV<DomainNameInfo, ThreatMatch>>write()
.withName(transformId)
.withBatchSize(options.getSqlWriteBatchSize())
.withShards(options.getSqlWriteShards())
.withJpaConverter(
(kv) -> {
DomainNameInfo domainNameInfo = kv.getKey();
return new Spec11ThreatMatch.Builder()
.setThreatTypes(
ImmutableSet.of(ThreatType.valueOf(kv.getValue().threatType())))
.setCheckDate(date)
.setDomainName(domainNameInfo.domainName())
.setDomainRepoId(domainNameInfo.domainRepoId())
.setRegistrarId(domainNameInfo.registrarId())
.build();
}));
String transformId = "Spec11 Threat Matches";
threatMatches
.apply(
"Construct objects",
ParDo.of(
new DoFn<KV<DomainNameInfo, ThreatMatch>, Spec11ThreatMatch>() {
@ProcessElement
public void processElement(
@Element KV<DomainNameInfo, ThreatMatch> input,
OutputReceiver<Spec11ThreatMatch> output) {
Spec11ThreatMatch spec11ThreatMatch =
new Spec11ThreatMatch.Builder()
.setThreatTypes(
ImmutableSet.of(ThreatType.valueOf(input.getValue().threatType())))
.setCheckDate(date)
.setDomainName(input.getKey().domainName())
.setDomainRepoId(input.getKey().domainRepoId())
.setRegistrarId(input.getKey().registrarId())
.setId(IdService.allocateId())
.build();
output.output(spec11ThreatMatch);
}
}))
.apply("Prevent Fusing", Reshuffle.viaRandomKey())
.apply(
"Write to Sql: " + transformId,
RegistryJpaIO.<Spec11ThreatMatch>write()
.withName(transformId)
.withBatchSize(options.getSqlWriteBatchSize()));
}
static void saveToGcs(
@@ -214,8 +226,7 @@ public class Spec11Pipeline implements Serializable {
return output.toString();
} catch (JSONException e) {
throw new RuntimeException(
String.format(
"Encountered an error constructing the JSON for %s", kv.toString()),
String.format("Encountered an error constructing the JSON for %s", kv),
e);
}
}))

View File

@@ -25,28 +25,34 @@ import org.json.JSONObject;
public abstract class ThreatMatch implements Serializable {
private static final String THREAT_TYPE_FIELD = "threatType";
private static final String DOMAIN_NAME_FIELD = "fullyQualifiedDomainName";
private static final String DOMAIN_NAME_FIELD = "domainName";
private static final String OUTDATED_NAME_FIELD = "fullyQualifiedDomainName";
/** Returns what kind of threat it is (malware, phishing etc.) */
public abstract String threatType();
/** Returns the fully qualified domain name [SLD].[TLD] of the matched threat. */
public abstract String fullyQualifiedDomainName();
public abstract String domainName();
@VisibleForTesting
static ThreatMatch create(String threatType, String fullyQualifiedDomainName) {
return new AutoValue_ThreatMatch(threatType, fullyQualifiedDomainName);
static ThreatMatch create(String threatType, String domainName) {
return new AutoValue_ThreatMatch(threatType, domainName);
}
/** Returns a {@link JSONObject} representing a subset of this object's data. */
JSONObject toJSON() throws JSONException {
return new JSONObject()
.put(THREAT_TYPE_FIELD, threatType())
.put(DOMAIN_NAME_FIELD, fullyQualifiedDomainName());
.put(DOMAIN_NAME_FIELD, domainName());
}
/** Parses a {@link JSONObject} and returns an equivalent {@link ThreatMatch}. */
public static ThreatMatch fromJSON(JSONObject threatMatch) throws JSONException {
// TODO: delete OUTDATED_NAME_FIELD once we no longer process reports saved with
// fullyQualifiedDomainName in them, likely 2023
return new AutoValue_ThreatMatch(
threatMatch.getString(THREAT_TYPE_FIELD), threatMatch.getString(DOMAIN_NAME_FIELD));
threatMatch.getString(THREAT_TYPE_FIELD),
threatMatch.has(OUTDATED_NAME_FIELD)
? threatMatch.getString(OUTDATED_NAME_FIELD)
: threatMatch.getString(DOMAIN_NAME_FIELD));
}
}

View File

@@ -14,22 +14,21 @@
package google.registry.config;
import static java.nio.charset.StandardCharsets.UTF_8;
import static com.google.common.base.Preconditions.checkArgument;
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.auth.ServiceAccountSigner;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.common.collect.ImmutableList;
import dagger.Module;
import dagger.Provides;
import google.registry.config.RegistryConfig.Config;
import google.registry.keyring.api.KeyModule.Key;
import google.registry.util.Clock;
import google.registry.util.GoogleCredentialsBundle;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.lang.annotation.Documented;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.time.Duration;
import javax.inject.Qualifier;
import javax.inject.Singleton;
@@ -37,6 +36,36 @@ import javax.inject.Singleton;
@Module
public abstract class CredentialModule {
/**
* Provides a {@link GoogleCredentialsBundle} backed by the application default credential from
* the Google Cloud Runtime. This credential may be used to access GCP APIs that are NOT part of
* the Google Workspace.
*
* <p>The credential returned by the Cloud Runtime depends on the runtime environment:
*
* <ul>
* <li>On App Engine, returns a scope-less {@code ComputeEngineCredentials} for
* PROJECT_ID@appspot.gserviceaccount.com
* <li>On Compute Engine, returns a scope-less {@code ComputeEngineCredentials} for
* PROJECT_NUMBER-compute@developer.gserviceaccount.com
* <li>On end user host, this returns the credential downloaded by gcloud. Please refer to <a
* href="https://cloud.google.com/sdk/gcloud/reference/auth/application-default/login">Cloud
* SDK documentation</a> for details.
* </ul>
*/
@ApplicationDefaultCredential
@Provides
@Singleton
public static GoogleCredentialsBundle provideApplicationDefaultCredential() {
GoogleCredentials credential;
try {
credential = GoogleCredentials.getApplicationDefault();
} catch (IOException e) {
throw new RuntimeException(e);
}
return GoogleCredentialsBundle.create(credential);
}
/**
* Provides the default {@link GoogleCredentialsBundle} from the Google Cloud runtime.
*
@@ -70,102 +99,90 @@ public abstract class CredentialModule {
}
/**
* Provides the default {@link GoogleCredential} from the Google Cloud runtime for G Suite
* Drive API.
* TODO(b/138195359): Deprecate this credential once we figure out how to use
* {@link GoogleCredentials} for G Suite Drive API.
* Provides a {@link GoogleCredentialsBundle} for accessing Google Workspace APIs, such as Drive
* and Sheets.
*/
@GSuiteDriveCredential
@GoogleWorkspaceCredential
@Provides
@Singleton
public static GoogleCredential provideGSuiteDriveCredential(
public static GoogleCredentialsBundle provideGSuiteDriveCredential(
@ApplicationDefaultCredential GoogleCredentialsBundle applicationDefaultCredential,
@Config("defaultCredentialOauthScopes") ImmutableList<String> requiredScopes) {
GoogleCredential credential;
try {
credential = GoogleCredential.getApplicationDefault();
} catch (IOException e) {
throw new RuntimeException(e);
}
if (credential.createScopedRequired()) {
credential = credential.createScoped(requiredScopes);
}
return credential;
}
/**
* Provides a {@link GoogleCredentialsBundle} from the service account's JSON key file.
*
* <p>On App Engine, a thread created using Java's built-in API needs this credential when it
* calls App Engine API. The Google Sheets API also needs this credential.
*/
@JsonCredential
@Provides
@Singleton
public static GoogleCredentialsBundle provideJsonCredential(
@Config("defaultCredentialOauthScopes") ImmutableList<String> requiredScopes,
@Key("jsonCredential") String jsonCredential) {
GoogleCredentials credential;
try {
credential =
GoogleCredentials.fromStream(new ByteArrayInputStream(jsonCredential.getBytes(UTF_8)));
} catch (IOException e) {
throw new UncheckedIOException(e);
}
if (credential.createScopedRequired()) {
credential = credential.createScoped(requiredScopes);
}
GoogleCredentials credential = applicationDefaultCredential.getGoogleCredentials();
// Although credential is scope-less, its `createScopedRequired` method still returns false.
credential = credential.createScoped(requiredScopes);
return GoogleCredentialsBundle.create(credential);
}
/**
* Provides a {@link GoogleCredentialsBundle} with delegated admin access for a G Suite domain.
* Provides a {@link GoogleCredentialsBundle} with delegated access to Google Workspace APIs for
* the application default credential user.
*
* <p>The G Suite domain must grant delegated admin access to the registry service account with
* all scopes in {@code requiredScopes}, including ones not related to G Suite.
* <p>The Workspace domain must grant delegated admin access to the default service account user
* (project-id@appspot.gserviceaccount.com on AppEngine) with all scopes in {@code defaultScopes}
* and {@code delegationScopes}.
*/
@DelegatedCredential
@AdcDelegatedCredential
@Provides
@Singleton
public static GoogleCredentialsBundle provideDelegatedCredential(
@Config("delegatedCredentialOauthScopes") ImmutableList<String> requiredScopes,
@JsonCredential GoogleCredentialsBundle credentialsBundle,
@Config("gSuiteAdminAccountEmailAddress") String gSuiteAdminAccountEmailAddress) {
return GoogleCredentialsBundle.create(credentialsBundle
.getGoogleCredentials()
.createDelegated(gSuiteAdminAccountEmailAddress)
.createScoped(requiredScopes));
public static GoogleCredentialsBundle provideSelfSignedDelegatedCredential(
@Config("defaultCredentialOauthScopes") ImmutableList<String> defaultScopes,
@Config("delegatedCredentialOauthScopes") ImmutableList<String> delegationScopes,
@ApplicationDefaultCredential GoogleCredentialsBundle credentialsBundle,
@Config("gSuiteAdminAccountEmailAddress") String gSuiteAdminAccountEmailAddress,
@Config("tokenRefreshDelay") Duration tokenRefreshDelay,
Clock clock) {
GoogleCredentials signer = credentialsBundle.getGoogleCredentials();
checkArgument(
signer instanceof ServiceAccountSigner,
"Expecting a ServiceAccountSigner, found %s.",
signer.getClass().getSimpleName());
try {
// Refreshing as sanity check on the ADC.
signer.refresh();
} catch (IOException e) {
throw new RuntimeException("Cannot refresh the ApplicationDefaultCredential", e);
}
DelegatedCredentials credential =
DelegatedCredentials.createSelfSignedDelegatedCredential(
(ServiceAccountSigner) signer,
ImmutableList.<String>builder().addAll(defaultScopes).addAll(delegationScopes).build(),
gSuiteAdminAccountEmailAddress,
clock,
tokenRefreshDelay);
return GoogleCredentialsBundle.create(credential);
}
/** Dagger qualifier for the scope-less Application Default Credential. */
@Qualifier
@Documented
@Retention(RetentionPolicy.RUNTIME)
public @interface ApplicationDefaultCredential {}
/** Dagger qualifier for the Application Default Credential. */
@Qualifier
@Documented
@Retention(RetentionPolicy.RUNTIME)
@Deprecated // Switching to @ApplicationDefaultCredential
public @interface DefaultCredential {}
/** Dagger qualifier for the credential for G Suite Drive API. */
/** Dagger qualifier for the credential for Google Workspace APIs. */
@Qualifier
@Documented
@Retention(RetentionPolicy.RUNTIME)
public @interface GSuiteDriveCredential {}
public @interface GoogleWorkspaceCredential {}
/**
* Dagger qualifier for a credential from a service account's JSON key, to be used in non-request
* threads.
* Dagger qualifier for a credential with delegated admin access for a dasher domain (for Google
* Workspace) backed by the application default credential (ADC).
*/
@Qualifier
@Documented
@Retention(RetentionPolicy.RUNTIME)
public @interface JsonCredential {}
/**
* Dagger qualifier for a credential with delegated admin access for a dasher domain (for G
* Suite).
*/
@Qualifier
@Documented
@Retention(RetentionPolicy.RUNTIME)
public @interface DelegatedCredential {}
public @interface AdcDelegatedCredential {}
/** Dagger qualifier for the local credential used in the nomulus tool. */
@Qualifier

View File

@@ -0,0 +1,268 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.config;
import static com.google.common.base.Preconditions.checkArgument;
import com.google.api.client.http.GenericUrl;
import com.google.api.client.http.HttpBackOffIOExceptionHandler;
import com.google.api.client.http.HttpBackOffUnsuccessfulResponseHandler;
import com.google.api.client.http.HttpRequest;
import com.google.api.client.http.HttpRequestFactory;
import com.google.api.client.http.HttpResponse;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.http.UrlEncodedContent;
import com.google.api.client.http.javanet.NetHttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.JsonObjectParser;
import com.google.api.client.json.gson.GsonFactory;
import com.google.api.client.json.webtoken.JsonWebSignature;
import com.google.api.client.json.webtoken.JsonWebToken;
import com.google.api.client.util.ExponentialBackOff;
import com.google.api.client.util.GenericData;
import com.google.api.client.util.StringUtils;
import com.google.auth.ServiceAccountSigner;
import com.google.auth.http.HttpTransportFactory;
import com.google.auth.oauth2.AccessToken;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Iterables;
import google.registry.util.Clock;
import java.io.IOException;
import java.math.BigDecimal;
import java.time.Duration;
import java.util.Collection;
import java.util.Date;
import java.util.Map;
import java.util.ServiceLoader;
import org.apache.commons.codec.binary.Base64;
/**
* OAuth2 credentials for accessing Google Workspace APIs with domain-wide delegation. It fetches
* access tokens using JSON Web Tokens (JWT) signed by a user-provided {@link ServiceAccountSigner}.
*
* <p>This class accepts the application-default-credential as {@code ServiceAccountSigner},
* avoiding the need for exported private keys. In this case, the default credential user itself
* (project-id@appspot.gserviceaccount.com on AppEngine) must have domain-wide delegation to the
* Workspace APIs. The default credential user also must have the Token Creator role to itself.
*
* <p>If the user provides a credential {@code S} that carries its own private key, such as {@link
* com.google.auth.oauth2.ServiceAccountCredentials}, this class can use {@code S} to impersonate
* another service account {@code D} and gain delegated access as {@code D}, as long as S has the
* Token Creator role to {@code D}. This usage is documented here for future reference.
*
* <p>As of October 2022, the functionalities described above are not implemented in the GCP Java
* Auth library, although they are available in the Python library. We have filed a <a
* href="https://github.com/googleapis/google-auth-library-java/issues/1064">feature request</a>.
* This class is a stop-gap implementation.
*
* <p>The main body of this class is adapted from {@link
* com.google.auth.oauth2.ServiceAccountCredentials} with cosmetic changes. The important changes
* include the removal of all uses of the private key and the signing of the JWT (in {@link
* #signAssertion}). We choose not to extend {@code ServiceAccountCredentials} because it would add
* dependency to the non-public details of that class.
*/
public class DelegatedCredentials extends GoogleCredentials {
private static final long serialVersionUID = 617127523756785546L;
private static final String DEFAULT_TOKEN_URI = "https://accounts.google.com/o/oauth2/token";
private static final String GRANT_TYPE = "urn:ietf:params:oauth:grant-type:jwt-bearer";
private static final JsonFactory JSON_FACTORY = GsonFactory.getDefaultInstance();
private static final HttpTransport HTTP_TRANSPORT = new NetHttpTransport();
private static final String VALUE_NOT_FOUND_MESSAGE = "%sExpected value %s not found.";
private static final String VALUE_WRONG_TYPE_MESSAGE = "%sExpected %s value %s of wrong type.";
private static final String PARSE_ERROR_PREFIX = "Error parsing token refresh response. ";
private static final Duration MAX_TOKEN_REFRESH_DELAY = Duration.ofHours(1);
private final ServiceAccountSigner signer;
private final String delegatedServiceAccountUser;
private final ImmutableList<String> scopes;
private final String delegatingUserEmail;
private final Clock clock;
private final Duration tokenRefreshDelay;
private final HttpTransportFactory transportFactory;
/**
* Creates a {@link DelegatedCredentials} instance that is self-signed by the signer, which must
* have delegated access to the Workspace APIs.
*
* @param signer Signs for the generated JWT tokens. This may be the application default
* credential
* @param scopes The scopes to use when generating JWT tokens
* @param delegatingUserEmail The Workspace user whose permissions are delegated to the signer
* @param clock Used for setting token expiration times.
* @param tokenRefreshDelay The lifetime of each token. Should not exceed one hour according to
* GCP recommendations.
* @return
*/
static DelegatedCredentials createSelfSignedDelegatedCredential(
ServiceAccountSigner signer,
Collection<String> scopes,
String delegatingUserEmail,
Clock clock,
Duration tokenRefreshDelay) {
return new DelegatedCredentials(
signer, signer.getAccount(), scopes, delegatingUserEmail, clock, tokenRefreshDelay);
}
private DelegatedCredentials(
ServiceAccountSigner signer,
String delegatedServiceAccountUser,
Collection<String> scopes,
String delegatingUserEmail,
Clock clock,
Duration tokenRefreshDelay) {
checkArgument(
tokenRefreshDelay.getSeconds() <= MAX_TOKEN_REFRESH_DELAY.getSeconds(),
"Max refresh delay must not exceed %s.",
MAX_TOKEN_REFRESH_DELAY);
this.signer = signer;
this.delegatedServiceAccountUser = delegatedServiceAccountUser;
this.scopes = ImmutableList.copyOf(scopes);
this.delegatingUserEmail = delegatingUserEmail;
this.clock = clock;
this.tokenRefreshDelay = tokenRefreshDelay;
this.transportFactory =
getFromServiceLoader(
HttpTransportFactory.class, DelegatedCredentials::provideHttpTransport);
}
/**
* Refreshes the OAuth2 access token by getting a new access token using a JSON Web Token (JWT).
*/
@Override
public AccessToken refreshAccessToken() throws IOException {
JsonFactory jsonFactory = JSON_FACTORY;
long currentTime = clock.nowUtc().getMillis();
String assertion = createAssertion(jsonFactory, currentTime);
GenericData tokenRequest = new GenericData();
tokenRequest.set("grant_type", GRANT_TYPE);
tokenRequest.set("assertion", assertion);
UrlEncodedContent content = new UrlEncodedContent(tokenRequest);
HttpRequestFactory requestFactory = transportFactory.create().createRequestFactory();
HttpRequest request =
requestFactory.buildPostRequest(new GenericUrl(DEFAULT_TOKEN_URI), content);
request.setParser(new JsonObjectParser(jsonFactory));
request.setIOExceptionHandler(new HttpBackOffIOExceptionHandler(new ExponentialBackOff()));
request.setUnsuccessfulResponseHandler(
new HttpBackOffUnsuccessfulResponseHandler(new ExponentialBackOff())
.setBackOffRequired(
response -> {
int code = response.getStatusCode();
return (
// Server error --- includes timeout errors, which use 500 instead of 408
code / 100 == 5
// Forbidden error --- for historical reasons, used for rate_limit_exceeded
// errors instead of 429, but there currently seems no robust automatic way
// to
// distinguish these cases: see
// https://github.com/google/google-api-java-client/issues/662
|| code == 403);
}));
HttpResponse response;
try {
response = request.execute();
} catch (IOException e) {
throw new IOException(
String.format("Error getting access token for service account: %s", e.getMessage()), e);
}
GenericData responseData = response.parseAs(GenericData.class);
String accessToken = validateString(responseData, "access_token", PARSE_ERROR_PREFIX);
int expiresInSeconds = validateInt32(responseData, "expires_in", PARSE_ERROR_PREFIX);
long expiresAtMilliseconds = clock.nowUtc().getMillis() + expiresInSeconds * 1000L;
return new AccessToken(accessToken, new Date(expiresAtMilliseconds));
}
String createAssertion(JsonFactory jsonFactory, long currentTime) throws IOException {
JsonWebSignature.Header header = new JsonWebSignature.Header();
header.setAlgorithm("RS256");
header.setType("JWT");
JsonWebToken.Payload payload = new JsonWebToken.Payload();
payload.setIssuer(this.delegatedServiceAccountUser);
payload.setIssuedAtTimeSeconds(currentTime / 1000);
payload.setExpirationTimeSeconds(currentTime / 1000 + tokenRefreshDelay.getSeconds());
payload.setSubject(delegatingUserEmail);
payload.put("scope", Joiner.on(' ').join(scopes));
payload.setAudience(DEFAULT_TOKEN_URI);
return signAssertion(jsonFactory, header, payload);
}
String signAssertion(
JsonFactory jsonFactory, JsonWebSignature.Header header, JsonWebToken.Payload payload)
throws IOException {
String content =
Base64.encodeBase64URLSafeString(jsonFactory.toByteArray(header))
+ "."
+ Base64.encodeBase64URLSafeString(jsonFactory.toByteArray(payload));
byte[] contentBytes = StringUtils.getBytesUtf8(content);
byte[] signature = signer.sign(contentBytes); // Changed from ServiceAccountCredentials.
return content + "." + Base64.encodeBase64URLSafeString(signature);
}
static HttpTransport provideHttpTransport() {
return HTTP_TRANSPORT;
}
protected static <T> T getFromServiceLoader(Class<? extends T> clazz, T defaultInstance) {
return Iterables.getFirst(ServiceLoader.load(clazz), defaultInstance);
}
/** Return the specified string from JSON or throw a helpful error message. */
static String validateString(Map<String, Object> map, String key, String errorPrefix)
throws IOException {
Object value = map.get(key);
if (value == null) {
throw new IOException(String.format(VALUE_NOT_FOUND_MESSAGE, errorPrefix, key));
}
if (!(value instanceof String)) {
throw new IOException(String.format(VALUE_WRONG_TYPE_MESSAGE, errorPrefix, "string", key));
}
return (String) value;
}
/** Return the specified integer from JSON or throw a helpful error message. */
static int validateInt32(Map<String, Object> map, String key, String errorPrefix)
throws IOException {
Object value = map.get(key);
if (value == null) {
throw new IOException(String.format(VALUE_NOT_FOUND_MESSAGE, errorPrefix, key));
}
if (value instanceof BigDecimal) {
BigDecimal bigDecimalValue = (BigDecimal) value;
return bigDecimalValue.intValueExact();
}
if (!(value instanceof Integer)) {
throw new IOException(String.format(VALUE_WRONG_TYPE_MESSAGE, errorPrefix, "integer", key));
}
return (Integer) value;
}
}

View File

@@ -103,13 +103,19 @@ public final class RegistryConfig {
@Provides
@Config("projectId")
public static String provideProjectId(RegistryConfigSettings config) {
return config.appEngine.projectId;
return config.gcpProject.projectId;
}
@Provides
@Config("projectIdNumber")
public static long provideProjectIdNumber(RegistryConfigSettings config) {
return config.gcpProject.projectIdNumber;
}
@Provides
@Config("locationId")
public static String provideLocationId(RegistryConfigSettings config) {
return config.appEngine.locationId;
return config.gcpProject.locationId;
}
/**
@@ -1027,38 +1033,6 @@ public final class RegistryConfig {
return 50;
}
/**
* Returns the delay before executing async delete flow mapreduces.
*
* <p>This delay should be sufficiently longer than a transaction, to solve the following
* problem:
*
* <ul>
* <li>a domain mutation flow starts a transaction
* <li>the domain flow non-transactionally reads a resource and sees that it's not in
* PENDING_DELETE
* <li>the domain flow creates a new reference to this resource
* <li>a contact/host delete flow runs and marks the resource PENDING_DELETE and commits
* <li>the domain flow commits
* </ul>
*
* <p>Although we try not to add references to a PENDING_DELETE resource, strictly speaking that
* is ok as long as the mapreduce eventually sees the new reference (and therefore
* asynchronously fails the delete). Without this delay, the mapreduce might have started before
* the domain flow committed, and could potentially miss the reference.
*
* <p>If you are using EPP resource caching (eppResourceCachingEnabled in YAML), then this
* duration should also be longer than that cache duration (eppResourceCachingSeconds).
*
* @see google.registry.config.RegistryConfigSettings.Caching
* @see google.registry.batch.AsyncTaskEnqueuer
*/
@Provides
@Config("asyncDeleteDelay")
public static Duration provideAsyncDeleteDelay(RegistryConfigSettings config) {
return Duration.standardSeconds(config.misc.asyncDeleteDelaySeconds);
}
/**
* The server ID used in the 'svID' element of an EPP 'greeting'.
*
@@ -1076,24 +1050,6 @@ public final class RegistryConfig {
return config.keyring.activeKeyring;
}
/**
* The name to use for the Cloud KMS KeyRing containing encryption keys for Nomulus secrets.
*
* @see <a
* href="https://cloud.google.com/kms/docs/reference/rest/v1/projects.locations.keyRings#KeyRing">projects.locations.keyRings</a>
*/
@Provides
@Config("cloudKmsKeyRing")
public static String provideCloudKmsKeyRing(RegistryConfigSettings config) {
return config.keyring.kms.keyringName;
}
@Provides
@Config("cloudKmsProjectId")
public static String provideCloudKmsProjectId(RegistryConfigSettings config) {
return config.keyring.kms.projectId;
}
@Provides
@Config("customLogicFactoryClass")
public static String provideCustomLogicFactoryClass(RegistryConfigSettings config) {
@@ -1223,6 +1179,12 @@ public final class RegistryConfig {
return ImmutableList.copyOf(config.credentialOAuth.localCredentialOauthScopes);
}
@Provides
@Config("tokenRefreshDelay")
public static java.time.Duration provideTokenRefreshDelay(RegistryConfigSettings config) {
return java.time.Duration.ofSeconds(config.credentialOAuth.tokenRefreshDelaySeconds);
}
/** OAuth client ID used by the nomulus tool. */
@Provides
@Config("toolsClientId")
@@ -1353,11 +1315,49 @@ public final class RegistryConfig {
public static int provideHibernateJdbcBatchSize(RegistryConfigSettings config) {
return config.hibernate.jdbcBatchSize;
}
@Provides
@Config("packageCreateLimitEmailSubject")
public static String providePackageCreateLimitEmailSubject(RegistryConfigSettings config) {
return config.packageMonitoring.packageCreateLimitEmailSubject;
}
@Provides
@Config("packageCreateLimitEmailBody")
public static String providePackageCreateLimitEmailBody(RegistryConfigSettings config) {
return config.packageMonitoring.packageCreateLimitEmailBody;
}
@Provides
@Config("packageDomainLimitWarningEmailSubject")
public static String providePackageDomainLimitWarningEmailSubject(
RegistryConfigSettings config) {
return config.packageMonitoring.packageDomainLimitWarningEmailSubject;
}
@Provides
@Config("packageDomainLimitWarningEmailBody")
public static String providePackageDomainLimitWarningEmailBody(RegistryConfigSettings config) {
return config.packageMonitoring.packageDomainLimitWarningEmailBody;
}
@Provides
@Config("packageDomainLimitUpgradeEmailSubject")
public static String providePackageDomainLimitUpgradeEmailSubject(
RegistryConfigSettings config) {
return config.packageMonitoring.packageDomainLimitUpgradeEmailSubject;
}
@Provides
@Config("packageDomainLimitUpgradeEmailBody")
public static String providePackageDomainLimitUpgradeEmailBody(RegistryConfigSettings config) {
return config.packageMonitoring.packageDomainLimitUpgradeEmailBody;
}
}
/** Returns the App Engine project ID, which is based off the environment name. */
public static String getProjectId() {
return CONFIG_SETTINGS.get().appEngine.projectId;
return CONFIG_SETTINGS.get().gcpProject.projectId;
}
/**
@@ -1370,7 +1370,7 @@ public final class RegistryConfig {
}
public static boolean areServersLocal() {
return CONFIG_SETTINGS.get().appEngine.isLocal;
return CONFIG_SETTINGS.get().gcpProject.isLocal;
}
/**
@@ -1379,7 +1379,7 @@ public final class RegistryConfig {
* <p>This is used by the {@code nomulus} tool to connect to the App Engine remote API.
*/
public static URL getDefaultServer() {
return makeUrl(CONFIG_SETTINGS.get().appEngine.defaultServiceUrl);
return makeUrl(CONFIG_SETTINGS.get().gcpProject.defaultServiceUrl);
}
/**
@@ -1388,7 +1388,7 @@ public final class RegistryConfig {
* <p>This is used by the {@code nomulus} tool to connect to the App Engine remote API.
*/
public static URL getBackendServer() {
return makeUrl(CONFIG_SETTINGS.get().appEngine.backendServiceUrl);
return makeUrl(CONFIG_SETTINGS.get().gcpProject.backendServiceUrl);
}
/**
@@ -1397,7 +1397,7 @@ public final class RegistryConfig {
* <p>This is used by the {@code nomulus} tool to connect to the App Engine remote API.
*/
public static URL getToolsServer() {
return makeUrl(CONFIG_SETTINGS.get().appEngine.toolsServiceUrl);
return makeUrl(CONFIG_SETTINGS.get().gcpProject.toolsServiceUrl);
}
/**
@@ -1406,7 +1406,7 @@ public final class RegistryConfig {
* <p>This is used by the {@code nomulus} tool to connect to the App Engine remote API.
*/
public static URL getPubapiServer() {
return makeUrl(CONFIG_SETTINGS.get().appEngine.pubapiServiceUrl);
return makeUrl(CONFIG_SETTINGS.get().gcpProject.pubapiServiceUrl);
}
/** Returns the amount of time a singleton should be cached, before expiring. */
@@ -1482,16 +1482,6 @@ public final class RegistryConfig {
return CONFIG_SETTINGS.get().registryPolicy.defaultRegistrarWhoisServer;
}
/** Returns the number of {@code EppResourceIndex} buckets to be used. */
public static int getEppResourceIndexBucketCount() {
return CONFIG_SETTINGS.get().datastore.eppResourceIndexBucketsNum;
}
/** Returns the base retry duration that gets doubled after each failure within {@code Ofy}. */
public static Duration getBaseOfyRetryDuration() {
return Duration.millis(CONFIG_SETTINGS.get().datastore.baseOfyRetryMillis);
}
/** Returns the default database transaction isolation. */
public static String getHibernateConnectionIsolation() {
return CONFIG_SETTINGS.get().hibernate.connectionIsolation;

View File

@@ -21,12 +21,11 @@ import java.util.Set;
/** The POJO that YAML config files are deserialized into. */
public class RegistryConfigSettings {
public AppEngine appEngine;
public GcpProject gcpProject;
public GSuite gSuite;
public OAuth oAuth;
public CredentialOAuth credentialOAuth;
public RegistryPolicy registryPolicy;
public Datastore datastore;
public Hibernate hibernate;
public CloudSql cloudSql;
public CloudDns cloudDns;
@@ -43,10 +42,12 @@ public class RegistryConfigSettings {
public SslCertificateValidation sslCertificateValidation;
public ContactHistory contactHistory;
public DnsUpdate dnsUpdate;
public PackageMonitoring packageMonitoring;
/** Configuration options that apply to the entire App Engine project. */
public static class AppEngine {
/** Configuration options that apply to the entire GCP project. */
public static class GcpProject {
public String projectId;
public long projectIdNumber;
public String locationId;
public boolean isLocal;
public String defaultServiceUrl;
@@ -67,6 +68,7 @@ public class RegistryConfigSettings {
public List<String> defaultCredentialOauthScopes;
public List<String> delegatedCredentialOauthScopes;
public List<String> localCredentialOauthScopes;
public int tokenRefreshDelaySeconds;
}
/** Configuration options for the G Suite account used by Nomulus. */
@@ -106,12 +108,6 @@ public class RegistryConfigSettings {
public boolean requireSslCertificates;
}
/** Configuration for Cloud Datastore. */
public static class Datastore {
public int eppResourceIndexBucketsNum;
public int baseOfyRetryMillis;
}
/** Configuration for Hibernate. */
public static class Hibernate {
public String connectionIsolation;
@@ -207,13 +203,13 @@ public class RegistryConfigSettings {
public String alertRecipientEmailAddress;
public String spec11OutgoingEmailAddress;
public List<String> spec11BccEmailAddresses;
public int asyncDeleteDelaySeconds;
public int transientFailureRetries;
}
/** Configuration for keyrings (used to store secrets outside of source). */
public static class Keyring {
public String activeKeyring;
// TODO(b/257276342): Remove after config files in nomulus-internal are updated.
public Kms kms;
}
@@ -256,4 +252,14 @@ public class RegistryConfigSettings {
public String registrySupportEmail;
public String registryCcEmail;
}
/** Configuration for package compliance monitoring. */
public static class PackageMonitoring {
public String packageCreateLimitEmailSubject;
public String packageCreateLimitEmailBody;
public String packageDomainLimitWarningEmailSubject;
public String packageDomainLimitWarningEmailBody;
public String packageDomainLimitUpgradeEmailSubject;
public String packageDomainLimitUpgradeEmailBody;
}
}

View File

@@ -5,12 +5,14 @@
# to override some of these values to configure and enable some services used in
# production environments.
appEngine:
# Globally unique App Engine project ID
gcpProject:
# Globally unique GCP project ID
projectId: registry-project-id
# Location of the App engine project, note that us-central1 and europe-west1 are special in that
# they are used without the trailing number in App Engine commands and Google Cloud Console.
# See: https://cloud.google.com/appengine/docs/locations
# Corresponding project ID number
projectIdNumber: 123456789012
# Location of the GCP project, note that us-central1 and europe-west1 are special in that
# they are used without the trailing number in GCP commands and Google Cloud Console.
# See: https://cloud.google.com/appengine/docs/locations as an example
locationId: registry-location-id
# whether to use local/test credentials when connecting to the servers
@@ -182,15 +184,6 @@ registryPolicy:
# should generally be true for production environments, for added security.
requireSslCertificates: true
datastore:
# Number of EPP resource index buckets in Datastore. Dont change after
# initial install.
eppResourceIndexBucketsNum: 997
# Milliseconds that Objectify waits to retry a Datastore transaction (this
# doubles after each failure).
baseOfyRetryMillis: 100
hibernate:
# Make 'SERIALIZABLE' the default isolation level to ensure correctness.
#
@@ -344,6 +337,9 @@ credentialOAuth:
- https://www.googleapis.com/auth/userinfo.email
# View and manage your applications deployed on Google App Engine
- https://www.googleapis.com/auth/appengine.admin
# The lifetime of an access token generated by our custom credentials classes
# Must be shorter than one hour.
tokenRefreshDelaySeconds: 1800
icannReporting:
# URL we PUT monthly ICANN transactions reports to.
@@ -422,11 +418,6 @@ misc:
spec11BccEmailAddresses:
- abuse@example.com
# How long to delay processing of asynchronous deletions. This should always
# be longer than eppResourceCachingSeconds, to prevent deleted contacts or
# hosts from being used on domains.
asyncDeleteDelaySeconds: 90
# Number of times to retry a GAE operation when a transient exception is thrown.
# The number of milliseconds it'll sleep before giving up is (2^n - 2) * 100.
transientFailureRetries: 12
@@ -450,7 +441,8 @@ beam:
stagingBucketUrl: gcs-bucket-with-staged-templates
keyring:
# The name of the active keyring, either "KMS" or "Dummy".
# The name of the active keyring, either "Dummy" or "CSM". The latter stands
# for Cloud SecretManager.
activeKeyring: Dummy
# Configuration options specific to Google Cloud KMS.
@@ -540,3 +532,55 @@ sslCertificateValidation:
allowedEcdsaCurves:
- secp256r1
- secp384r1
# Configuration options for the package compliance monitoring
packageMonitoring:
# Email subject text to notify partners their package has exceeded the limit for domain creates
packageCreateLimitEmailSubject: "NOTICE: Your package is being upgraded"
# Email body text template notify partners their package has exceeded the limit for domain creates
packageCreateLimitEmailBody: >
Dear %1$s,
We are contacting you to inform you that your package with the package token
%2$s has exceeded its limit for annual domain creations.
Your package will now be upgraded to the next tier.
If you have any questions or require additional support, please contact us
at %3$s.
Regards,
Example Registry
# Email subject text to notify partners their package has exceeded the limit for current active domains and warn them their package will be upgraded in 30 days
packageDomainLimitWarningEmailSubject: "WARNING: Your package has exceeded the domain limit"
# Email body text template to warn partners their package has exceeded the limit for active domains and will be upgraded in 30 days
packageDomainLimitWarningEmailBody: >
Dear %1$s,
We are contacting you to inform you that your package with the package token
%2$s has exceeded its limit for active domains.
Your package will be upgraded to the next tier in 30 days if the number of active domains does not return below the limit.
If you have any questions or require additional support, please contact us
at %3$s.
Regards,
Example Registry
# Email subject text to notify partners their package has exceeded the limit
# for current active domains for more than 30 days and will be upgraded
packageDomainLimitUpgradeEmailSubject: "NOTICE: Your package is being upgraded"
# Email body text template to warn partners their package has exceeded the
# limit for active domains for more than 30 days and will be upgraded
packageDomainLimitUpgradeEmailBody: >
Dear %1$s,
We are contacting you to inform you that your package with the package token
%2$s has exceeded its limit for active domains.
Your package will now be upgraded to the next tier.
If you have any questions or require additional support, please contact us
at %3$s.
Regards,
Example Registry

View File

@@ -2,7 +2,7 @@
# This is the same as what Google Registry runs in production, except with
# placeholders for Google-specific settings.
appEngine:
gcpProject:
projectId: placeholder
# Set to true if running against local servers (localhost)
isLocal: false

View File

@@ -10,10 +10,6 @@ registryPolicy:
Disclaimer line 1.
Line 2 is this 1.
datastore:
eppResourceIndexBucketsNum: 3
baseOfyRetryMillis: 0
caching:
singletonCacheRefreshSeconds: 0
domainLabelCachingSeconds: 0

View File

@@ -16,6 +16,7 @@ package google.registry.dns;
import static google.registry.dns.DnsConstants.DNS_PUBLISH_PUSH_QUEUE_NAME;
import static google.registry.dns.DnsConstants.DNS_PULL_QUEUE_NAME;
import static google.registry.dns.RefreshDnsOnHostRenameAction.PARAM_HOST_KEY;
import static google.registry.request.RequestParameters.extractEnumParameter;
import static google.registry.request.RequestParameters.extractIntParameter;
import static google.registry.request.RequestParameters.extractRequiredParameter;
@@ -61,7 +62,7 @@ public abstract class DnsModule {
*/
@Provides
static HashFunction provideHashFunction() {
return Hashing.murmur3_32();
return Hashing.murmur3_32_fixed();
}
@Provides
@@ -118,6 +119,12 @@ public abstract class DnsModule {
return extractSetOfParameters(req, PARAM_HOSTS);
}
@Provides
@Parameter(PARAM_HOST_KEY)
static String provideResourceKey(HttpServletRequest req) {
return extractRequiredParameter(req, PARAM_HOST_KEY);
}
@Provides
@Parameter("domainOrHostName")
static String provideName(HttpServletRequest req) {

View File

@@ -81,7 +81,7 @@ public final class PublishDnsUpdatesAction implements Runnable, Callable<Void> {
// tasks.
public static final String APP_ENGINE_RETRY_HEADER = "X-AppEngine-TaskRetryCount";
public static final String CLOUD_TASKS_RETRY_HEADER = "X-CloudTasks-TaskRetryCount";
public static final int RETRIES_BEFORE_PERMANENT_FAILURE = 10;
public static final int RETRIES_BEFORE_PERMANENT_FAILURE = 20;
private static final FluentLogger logger = FluentLogger.forEnclosingClass();

View File

@@ -0,0 +1,90 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.dns;
import static google.registry.dns.RefreshDnsOnHostRenameAction.PATH;
import static google.registry.model.EppResourceUtils.getLinkedDomainKeys;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static javax.servlet.http.HttpServletResponse.SC_NO_CONTENT;
import com.google.common.net.MediaType;
import google.registry.model.EppResourceUtils;
import google.registry.model.domain.Domain;
import google.registry.model.host.Host;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Action.Service;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import javax.inject.Inject;
import org.joda.time.DateTime;
@Action(
service = Service.BACKEND,
path = PATH,
method = Action.Method.POST,
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
public class RefreshDnsOnHostRenameAction implements Runnable {
public static final String QUEUE_HOST_RENAME = "async-host-rename";
public static final String PARAM_HOST_KEY = "hostKey";
public static final String PATH = "/_dr/task/refreshDnsOnHostRename";
private final VKey<Host> hostKey;
private final Response response;
private final DnsQueue dnsQueue;
@Inject
RefreshDnsOnHostRenameAction(
@Parameter(PARAM_HOST_KEY) String hostKey, Response response, DnsQueue dnsQueue) {
this.hostKey = VKey.createEppVKeyFromString(hostKey);
this.response = response;
this.dnsQueue = dnsQueue;
}
@Override
public void run() {
tm().transact(
() -> {
DateTime now = tm().getTransactionTime();
Host host = tm().loadByKeyIfPresent(hostKey).orElse(null);
boolean hostValid = true;
String failureMessage = null;
if (host == null) {
hostValid = false;
failureMessage = String.format("Host to refresh does not exist: %s", hostKey);
} else if (EppResourceUtils.isDeleted(host, now)) {
hostValid = false;
failureMessage =
String.format("Host to refresh is already deleted: %s", host.getHostName());
} else {
getLinkedDomainKeys(
host.createVKey(), host.getUpdateTimestamp().getTimestamp(), null)
.stream()
.map(domainKey -> tm().loadByKey(domainKey))
.filter(Domain::shouldPublishToDns)
.forEach(domain -> dnsQueue.addDomainRefreshTask(domain.getDomainName()));
}
if (!hostValid) {
// Set the response status code to be 204 so to not retry.
response.setContentType(MediaType.PLAIN_TEXT_UTF_8);
response.setStatus(SC_NO_CONTENT);
response.setPayload(failureMessage);
}
});
}
}

View File

@@ -37,7 +37,7 @@ import google.registry.dns.writer.BaseDnsWriter;
import google.registry.dns.writer.DnsWriter;
import google.registry.dns.writer.DnsWriterZone;
import google.registry.model.domain.Domain;
import google.registry.model.domain.secdns.DelegationSignerData;
import google.registry.model.domain.secdns.DomainDsData;
import google.registry.model.host.Host;
import google.registry.model.tld.Registries;
import google.registry.util.Clock;
@@ -134,10 +134,10 @@ public class CloudDnsWriter extends BaseDnsWriter {
ImmutableSet.Builder<ResourceRecordSet> domainRecords = new ImmutableSet.Builder<>();
// Construct DS records (if any).
Set<DelegationSignerData> dsData = domain.get().getDsData();
Set<DomainDsData> dsData = domain.get().getDsData();
if (!dsData.isEmpty()) {
HashSet<String> dsRrData = new HashSet<>();
for (DelegationSignerData ds : dsData) {
for (DomainDsData ds : dsData) {
dsRrData.add(ds.toRrData());
}

View File

@@ -28,7 +28,7 @@ import google.registry.config.RegistryConfig.Config;
import google.registry.dns.writer.BaseDnsWriter;
import google.registry.dns.writer.DnsWriterZone;
import google.registry.model.domain.Domain;
import google.registry.model.domain.secdns.DelegationSignerData;
import google.registry.model.domain.secdns.DomainDsData;
import google.registry.model.host.Host;
import google.registry.model.tld.Registries;
import google.registry.util.Clock;
@@ -185,7 +185,7 @@ public class DnsUpdateWriter extends BaseDnsWriter {
private RRset makeDelegationSignerSet(Domain domain) {
RRset signerSet = new RRset();
for (DelegationSignerData signerData : domain.getDsData()) {
for (DomainDsData signerData : domain.getDsData()) {
DSRecord dsRecord =
new DSRecord(
toAbsoluteName(domain.getDomainName()),

View File

@@ -175,16 +175,6 @@
<url-pattern>/_dr/cron/fanout</url-pattern>
</servlet-mapping>
<!-- Pipeline GUI servlets. -->
<servlet>
<servlet-name>pipeline</servlet-name>
<servlet-class>com.google.appengine.tools.pipeline.impl.servlets.PipelineServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>pipeline</servlet-name>
<url-pattern>/_ah/pipeline/*</url-pattern>
</servlet-mapping>
<!-- Syncs registrars to the registrar spreadsheet. -->
<servlet-mapping>
<servlet-name>backend-servlet</servlet-name>
@@ -244,12 +234,6 @@
<url-pattern>/_dr/task/resaveEntity</url-pattern>
</servlet-mapping>
<!-- Enqueues DNS update tasks following a host rename. -->
<servlet-mapping>
<servlet-name>backend-servlet</servlet-name>
<url-pattern>/_dr/task/dnsRefreshForHostRename</url-pattern>
</servlet-mapping>
<!-- Enqueues DNS update tasks following a host rename. -->
<servlet-mapping>
<servlet-name>backend-servlet</servlet-name>
@@ -293,6 +277,12 @@ have been in the database for a certain period of time. -->
<url-pattern>/_dr/task/wipeOutCloudSql</url-pattern>
</servlet-mapping>
<!-- Action to execute canned scripts -->
<servlet-mapping>
<servlet-name>backend-servlet</servlet-name>
<url-pattern>/_dr/task/executeCannedScript</url-pattern>
</servlet-mapping>
<!-- Security config -->
<security-constraint>
<web-resource-collection>
@@ -341,24 +331,4 @@ have been in the database for a certain period of time. -->
<transport-guarantee>CONFIDENTIAL</transport-guarantee>
</user-data-constraint>
</security-constraint>
<!-- See: https://code.google.com/p/objectify-appengine/wiki/Setup -->
<filter>
<filter-name>ObjectifyFilter</filter-name>
<filter-class>com.googlecode.objectify.ObjectifyFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>ObjectifyFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
<!-- Register types with Objectify. -->
<filter>
<filter-name>OfyFilter</filter-name>
<filter-class>google.registry.model.ofy.OfyFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>OfyFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
</web-app>

View File

@@ -18,86 +18,6 @@
</retry-parameters>
</queue>
<queue>
<name>async-delete-pull</name>
<mode>pull</mode>
</queue>
<queue>
<name>async-host-rename-pull</name>
<mode>pull</mode>
</queue>
<queue>
<name>export-commits</name>
<rate>10/s</rate>
<bucket-size>100</bucket-size>
<retry-parameters>
<!-- Retry aggressively since a single delayed export increases our time window of
unrecoverable data loss in the event of a Datastore failure. -->
<min-backoff-seconds>1</min-backoff-seconds>
<max-backoff-seconds>60</max-backoff-seconds>
<!-- No age limit; a failed export should be retried as long as possible to avoid
having data missing from our exported commit log record. -->
</retry-parameters>
</queue>
<!-- Queue for polling export BigQuery jobs for completion. -->
<queue>
<name>export-bigquery-poll</name>
<!-- Limit queue to 5 concurrent tasks and 5 per second to avoid hitting BigQuery quotas. -->
<rate>5/s</rate>
<bucket-size>5</bucket-size>
<max-concurrent-requests>5</max-concurrent-requests>
<!-- Check every 20s and increase interval to every 5 minutes. -->
<retry-parameters>
<min-backoff-seconds>20</min-backoff-seconds>
<max-backoff-seconds>300</max-backoff-seconds>
<max-doublings>2</max-doublings>
</retry-parameters>
</queue>
<!-- Queue for launching new snapshots and for triggering the initial BigQuery load jobs. -->
<queue>
<name>export-snapshot</name>
<rate>1/s</rate>
<retry-parameters>
<!-- Should be less than the exportSnapshot cron interval; see cron.xml. -->
<task-age-limit>22h</task-age-limit>
<!-- Retry starting at a 5m interval and increasing up to a 30m interval. -->
<min-backoff-seconds>300</min-backoff-seconds>
<max-backoff-seconds>1800</max-backoff-seconds>
<task-retry-limit>10</task-retry-limit>
</retry-parameters>
</queue>
<!-- Queue for polling managed backup snapshots for completion. -->
<queue>
<name>export-snapshot-poll</name>
<rate>5/m</rate>
<retry-parameters>
<!-- Should be less than the exportSnapshot cron interval; see cron.xml. -->
<task-age-limit>22h</task-age-limit>
<!-- Retry starting at a 1m interval and increasing up to a 5m interval. -->
<min-backoff-seconds>60</min-backoff-seconds>
<max-backoff-seconds>300</max-backoff-seconds>
</retry-parameters>
</queue>
<!-- Queue for updating BigQuery views after a snapshot kind's load job completes. -->
<queue>
<name>export-snapshot-update-view</name>
<rate>1/s</rate>
<retry-parameters>
<!-- Should be less than the exportSnapshot cron interval; see cron.xml. -->
<task-age-limit>22h</task-age-limit>
<!-- Retry starting at a 10s interval and increasing up to a 1m interval. -->
<min-backoff-seconds>10</min-backoff-seconds>
<max-backoff-seconds>60</max-backoff-seconds>
<task-retry-limit>10</task-retry-limit>
</retry-parameters>
</queue>
<queue>
<name>rde-upload</name>
<rate>10/m</rate>
@@ -126,6 +46,12 @@
</retry-parameters>
</queue>
<!-- Queue for tasks that trigger domain DNS update upon host rename. -->
<queue>
<name>async-host-rename</name>
<rate>1/s</rate>
</queue>
<!-- Queue for tasks that wait for a Beam pipeline to complete (i.e. Spec11 and invoicing). -->
<queue>
<name>beam-reporting</name>
@@ -149,7 +75,7 @@
</retry-parameters>
</queue>
<!-- Queue for tasks to produce LORDN CSV reports, either by by the query or queue method. -->
<!-- Queue for tasks to produce LORDN CSV reports, either by the query or queue method. -->
<queue>
<name>nordn</name>
<rate>1/s</rate>
@@ -171,17 +97,6 @@
<mode>pull</mode>
</queue>
<!-- Queue used by the MapReduce library for running tasks.
Do not re-use this queue for tasks that our code creates (e.g. tasks to launch MapReduces
that aren't themselves part of a running MapReduce).-->
<queue>
<name>mapreduce</name>
<!-- Warning: DO NOT SET A <target> parameter for this queue. See b/24782801 for why. -->
<rate>500/s</rate>
<bucket-size>100</bucket-size>
</queue>
<!-- Queue for tasks that sync data to Google Spreadsheets. -->
<queue>
<name>sheet</name>
@@ -208,71 +123,4 @@
<max-concurrent-requests>5</max-concurrent-requests>
</queue>
<!-- Queue for replaying commit logs to SQL during the transition from Datastore -> SQL. -->
<queue>
<name>replay-commit-logs-to-sql</name>
<rate>1/s</rate>
</queue>
<!-- The load[0-9] queues are used for load-testing, and can be safely deleted
in any environment that doesn't require load-testing. -->
<queue>
<name>load0</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load1</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load2</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load3</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load4</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load5</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load6</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load7</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load8</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
<queue>
<name>load9</name>
<rate>500/s</rate>
<bucket-size>500</bucket-size>
</queue>
</queue-entries>

View File

@@ -139,24 +139,4 @@
<transport-guarantee>CONFIDENTIAL</transport-guarantee>
</user-data-constraint>
</security-constraint>
<!-- See: https://code.google.com/p/objectify-appengine/wiki/Setup -->
<filter>
<filter-name>ObjectifyFilter</filter-name>
<filter-class>com.googlecode.objectify.ObjectifyFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>ObjectifyFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
<!-- Register types with Objectify. -->
<filter>
<filter-name>OfyFilter</filter-name>
<filter-class>google.registry.model.ofy.OfyFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>OfyFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
</web-app>

View File

@@ -105,24 +105,4 @@
<transport-guarantee>CONFIDENTIAL</transport-guarantee>
</user-data-constraint>
</security-constraint>
<!-- See: https://code.google.com/p/objectify-appengine/wiki/Setup -->
<filter>
<filter-name>ObjectifyFilter</filter-name>
<filter-class>com.googlecode.objectify.ObjectifyFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>ObjectifyFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
<!-- Register types with Objectify. -->
<filter>
<filter-name>OfyFilter</filter-name>
<filter-class>google.registry.model.ofy.OfyFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>OfyFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
</web-app>

View File

@@ -48,34 +48,12 @@
<url-pattern>/_dr/loadtest</url-pattern>
</servlet-mapping>
<!-- The nomulus command line tool uses this endpoint to write to Datastore. -->
<servlet>
<display-name>Remote API Servlet</display-name>
<servlet-name>RemoteApiServlet</servlet-name>
<servlet-class>com.google.apphosting.utils.remoteapi.RemoteApiServlet</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>RemoteApiServlet</servlet-name>
<url-pattern>/remote_api</url-pattern>
</servlet-mapping>
<!-- ExecuteEppCommand uses this to execute remotely. -->
<servlet-mapping>
<servlet-name>tools-servlet</servlet-name>
<url-pattern>/_dr/epptool</url-pattern>
</servlet-mapping>
<!-- Pipeline GUI servlets. -->
<servlet>
<servlet-name>pipeline</servlet-name>
<servlet-class>com.google.appengine.tools.pipeline.impl.servlets.PipelineServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>pipeline</servlet-name>
<url-pattern>/_ah/pipeline/*</url-pattern>
</servlet-mapping>
<!-- Refreshes all active domains in DNS -->
<servlet-mapping>
<servlet-name>tools-servlet</servlet-name>
@@ -135,24 +113,4 @@
<transport-guarantee>CONFIDENTIAL</transport-guarantee>
</user-data-constraint>
</security-constraint>
<!-- See: https://code.google.com/p/objectify-appengine/wiki/Setup -->
<filter>
<filter-name>ObjectifyFilter</filter-name>
<filter-class>com.googlecode.objectify.ObjectifyFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>ObjectifyFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
<!-- Register types with Objectify. -->
<filter>
<filter-name>OfyFilter</filter-name>
<filter-class>google.registry.model.ofy.OfyFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>OfyFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
</web-app>

View File

@@ -102,6 +102,7 @@
<target>backend</target>
</cron>
<!-- TODO(b/249863289): disable until it is safe to run this pipeline
<cron>
<url><![CDATA[/_dr/task/resaveAllEppResourcesPipeline?fast=true]]></url>
<description>
@@ -110,6 +111,7 @@
<schedule>1st monday of month 09:00</schedule>
<target>backend</target>
</cron>
-->
<cron>
<url><![CDATA[/_dr/task/updateRegistrarRdapBaseUrls]]></url>
@@ -267,7 +269,7 @@
about 2 hours to complete, so we give 11 hours to be safe. Normally, we give 24+ hours (see
icannReportingStaging), but the invoicing team prefers receiving the e-mail on the first of
each month. -->
<schedule>1 of month 17:00</schedule>
<schedule>1 of month 19:00</schedule>
<target>backend</target>
</cron>

View File

@@ -86,6 +86,7 @@
<target>backend</target>
</cron>
<!-- TODO(b/249863289): disable until it is safe to run this pipeline
<cron>
<url><![CDATA[/_dr/task/resaveAllEppResourcesPipeline?fast=true]]></url>
<description>
@@ -94,6 +95,7 @@
<schedule>1st monday of month 09:00</schedule>
<target>backend</target>
</cron>
-->
<cron>
<url><![CDATA[/_dr/cron/fanout?queue=retryable-cron-tasks&endpoint=/_dr/task/exportDomainLists&runInEmpty]]></url>

View File

@@ -14,16 +14,16 @@
package google.registry.export;
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.services.drive.Drive;
import dagger.Component;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule;
import google.registry.config.CredentialModule.GSuiteDriveCredential;
import google.registry.config.CredentialModule.GoogleWorkspaceCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.storage.drive.DriveConnection;
import google.registry.util.GoogleCredentialsBundle;
import javax.inject.Singleton;
/** Dagger module for Google {@link Drive} service connection objects. */
@@ -32,13 +32,13 @@ public final class DriveModule {
@Provides
static Drive provideDrive(
@GSuiteDriveCredential GoogleCredential googleCredential,
@GoogleWorkspaceCredential GoogleCredentialsBundle googleCredential,
@Config("projectId") String projectId) {
return new Drive.Builder(
googleCredential.getTransport(),
googleCredential.getHttpTransport(),
googleCredential.getJsonFactory(),
googleCredential)
googleCredential.getHttpRequestInitializer())
.setApplicationName(projectId)
.build();
}

View File

@@ -16,7 +16,6 @@ package google.registry.export;
import static com.google.common.base.Verify.verifyNotNull;
import static google.registry.model.tld.Registries.getTldsOfType;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static java.nio.charset.StandardCharsets.UTF_8;
@@ -89,12 +88,11 @@ public class ExportDomainListsAction implements Runnable {
// DateTime are persisted as timestamp_z in SQL. It is only the
// validation that compares the Java types, and only with the first
// field that compares with the substituted value.
jpaTm()
.query(
"SELECT fullyQualifiedDomainName FROM Domain "
tm().query(
"SELECT domainName FROM Domain "
+ "WHERE tld = :tld "
+ "AND deletionTime > :now "
+ "ORDER by fullyQualifiedDomainName ASC",
+ "ORDER by domainName ASC",
String.class)
.setParameter("tld", tld)
.setParameter("now", clock.nowUtc())

View File

@@ -163,7 +163,7 @@ public final class SyncGroupMembersAction implements Runnable {
registrarsToSave.add(result.getKey().asBuilder().setContactsRequireSyncing(false).build());
}
}
tm().transactNew(() -> tm().updateAll(registrarsToSave.build()));
tm().transact(() -> tm().updateAll(registrarsToSave.build()));
return errors;
}

View File

@@ -17,7 +17,7 @@ package google.registry.export.sheet;
import com.google.api.services.sheets.v4.Sheets;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule.JsonCredential;
import google.registry.config.CredentialModule.GoogleWorkspaceCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.util.GoogleCredentialsBundle;
@@ -27,7 +27,7 @@ public final class SheetsServiceModule {
@Provides
static Sheets provideSheets(
@JsonCredential GoogleCredentialsBundle credentialsBundle,
@GoogleWorkspaceCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Sheets.Builder(
credentialsBundle.getHttpTransport(),

View File

@@ -114,7 +114,7 @@ class SyncRegistrarsSheet {
// and you'll need to remove deleted columns probably like a week after
// deployment.
//
builder.put("clientIdentifier", convert(registrar.getRegistrarId()));
builder.put("registrarId", convert(registrar.getRegistrarId()));
builder.put("registrarName", convert(registrar.getRegistrarName()));
builder.put("state", convert(registrar.getState()));
builder.put("ianaIdentifier", convert(registrar.getIanaIdentifier()));

View File

@@ -44,8 +44,8 @@ import dagger.Module;
import dagger.Provides;
import google.registry.flows.domain.DomainFlowUtils.BadCommandForRegistryPhaseException;
import google.registry.flows.domain.DomainFlowUtils.InvalidIdnDomainLabelException;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.domain.Domain;
import google.registry.model.index.ForeignKeyIndex;
import google.registry.model.tld.Registry;
import google.registry.model.tld.label.ReservationType;
import google.registry.monitoring.whitebox.CheckApiMetric;
@@ -156,7 +156,7 @@ public class CheckApiAction implements Runnable {
}
private boolean checkExists(String domainString, DateTime now) {
return !ForeignKeyIndex.loadCached(Domain.class, ImmutableList.of(domainString), now).isEmpty();
return !ForeignKeyUtils.loadCached(Domain.class, ImmutableList.of(domainString), now).isEmpty();
}
private Optional<String> checkReserved(InternetDomainName domainName) {

View File

@@ -23,19 +23,18 @@ import static java.nio.charset.StandardCharsets.UTF_8;
import com.google.common.base.Throwables;
import com.google.common.flogger.FluentLogger;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException.CommandUseErrorException;
import google.registry.flows.EppException.ParameterValueRangeErrorException;
import google.registry.flows.EppException.SyntaxErrorException;
import google.registry.flows.EppException.UnimplementedProtocolVersionException;
import google.registry.flows.custom.EntityChanges;
import google.registry.model.EppResource;
import google.registry.model.adapters.CurrencyUnitAdapter.UnknownCurrencyException;
import google.registry.model.eppcommon.EppXmlTransformer;
import google.registry.model.eppinput.EppInput.WrongProtocolVersionException;
import google.registry.model.eppoutput.EppOutput;
import google.registry.model.host.InetAddressAdapter.IpVersionMismatchException;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.translators.CurrencyUnitAdapter.UnknownCurrencyException;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.xml.XmlException;
import java.util.List;
@@ -103,9 +102,8 @@ public final class FlowUtils {
}
}
public static <H extends HistoryEntry> Key<H> createHistoryKey(
EppResource parent, Class<H> clazz) {
return Key.create(Key.create(parent), clazz, allocateId());
public static HistoryEntryId createHistoryEntryId(EppResource parent) {
return new HistoryEntryId(parent.getRepoId(), allocateId());
}
/** Registrar is not logged in. */
@@ -118,7 +116,7 @@ public final class FlowUtils {
/** IP address version mismatch. */
public static class IpAddressVersionMismatchException extends ParameterValueRangeErrorException {
public IpAddressVersionMismatchException() {
super("IP adddress version mismatch");
super("IP address version mismatch");
}
}

View File

@@ -17,7 +17,6 @@ package google.registry.flows;
import static com.google.common.collect.Sets.intersection;
import static google.registry.model.EppResourceUtils.isLinked;
import static google.registry.model.EppResourceUtils.loadByForeignKey;
import static google.registry.model.index.ForeignKeyIndex.loadAndGetKey;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
@@ -38,6 +37,7 @@ import google.registry.flows.exceptions.TooManyResourceChecksException;
import google.registry.model.EppResource;
import google.registry.model.EppResource.ForeignKeyedEppResource;
import google.registry.model.EppResource.ResourceWithTransferData;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.contact.Contact;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainBase;
@@ -45,7 +45,6 @@ import google.registry.model.domain.Period;
import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.eppcommon.AuthInfo;
import google.registry.model.eppcommon.StatusValue;
import google.registry.model.index.ForeignKeyIndex;
import google.registry.model.transfer.TransferStatus;
import google.registry.persistence.VKey;
import java.util.List;
@@ -70,22 +69,17 @@ public final class ResourceFlowUtils {
/**
* Check whether if there are domains linked to the resource to be deleted. Throws an exception if
* so.
*
* <p>Note that in datastore this is a smoke test as the query for linked domains is eventually
* consistent, so we only check a few domains to fail fast.
*/
public static <R extends EppResource> void checkLinkedDomains(
final String targetId, final DateTime now, final Class<R> resourceClass) throws EppException {
EppException failfastException =
tm().transact(
() -> {
final ForeignKeyIndex<R> fki = ForeignKeyIndex.load(resourceClass, targetId, now);
if (fki == null) {
VKey<R> key = ForeignKeyUtils.load(resourceClass, targetId, now);
if (key == null) {
return new ResourceDoesNotExistException(resourceClass, targetId);
}
return isLinked(fki.getResourceKey(), now)
? new ResourceToDeleteIsReferencedException()
: null;
return isLinked(key, now) ? new ResourceToDeleteIsReferencedException() : null;
});
if (failfastException != null) {
throw failfastException;
@@ -118,7 +112,7 @@ public final class ResourceFlowUtils {
public static <R extends EppResource> void verifyResourceDoesNotExist(
Class<R> clazz, String targetId, DateTime now, String registrarId) throws EppException {
VKey<R> key = loadAndGetKey(clazz, targetId, now);
VKey<R> key = ForeignKeyUtils.load(clazz, targetId, now);
if (key != null) {
R resource = tm().loadByKey(key);
// These are similar exceptions, but we can track them internally as log-based metrics.
@@ -169,7 +163,7 @@ public final class ResourceFlowUtils {
// The roid should match one of the contacts.
Optional<VKey<Contact>> foundContact =
domain.getReferencedContacts().stream()
.filter(key -> key.getSqlKey().equals(authRepoId))
.filter(key -> key.getKey().equals(authRepoId))
.findFirst();
if (!foundContact.isPresent()) {
throw new BadAuthInfoForResourceException();

View File

@@ -23,7 +23,6 @@ import static google.registry.model.IdService.allocateId;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.config.RegistryConfig.Config;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
@@ -40,7 +39,6 @@ import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppinput.ResourceCommand;
import google.registry.model.eppoutput.CreateData.ContactCreateData;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.index.EppResourceIndex;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import javax.inject.Inject;
@@ -95,11 +93,7 @@ public final class ContactCreateFlow implements TransactionalFlow {
.setType(HistoryEntry.Type.CONTACT_CREATE)
.setXmlBytes(null) // We don't want to store contact details in the history entry.
.setContact(newContact);
tm().insertAll(
ImmutableSet.of(
newContact,
historyBuilder.build(),
EppResourceIndex.create(Key.create(newContact))));
tm().insertAll(ImmutableSet.of(newContact, historyBuilder.build()));
return responseBuilder
.setResData(ContactCreateData.create(newContact.getContactId(), now))
.build();

View File

@@ -27,7 +27,6 @@ import static google.registry.model.transfer.TransferStatus.SERVER_CANCELLED;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import google.registry.batch.AsyncTaskEnqueuer;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -79,7 +78,6 @@ public final class ContactDeleteFlow implements TransactionalFlow {
@Inject @Superuser boolean isSuperuser;
@Inject Optional<AuthInfo> authInfo;
@Inject ContactHistory.Builder historyBuilder;
@Inject AsyncTaskEnqueuer asyncTaskEnqueuer;
@Inject EppResponse.Builder responseBuilder;
@Inject

View File

@@ -20,17 +20,15 @@ import com.google.common.base.CharMatcher;
import com.google.common.base.Preconditions;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Sets;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.EppException.ParameterValuePolicyErrorException;
import google.registry.flows.EppException.ParameterValueSyntaxErrorException;
import google.registry.model.contact.Contact;
import google.registry.model.contact.ContactAddress;
import google.registry.model.contact.ContactHistory;
import google.registry.model.contact.ContactHistory.ContactHistoryId;
import google.registry.model.contact.PostalInfo;
import google.registry.model.poll.PendingActionNotificationResponse.ContactPendingActionNotificationResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.transfer.TransferData;
import google.registry.model.transfer.TransferResponse.ContactTransferResponse;
import java.util.Set;
@@ -69,10 +67,7 @@ public class ContactFlowUtils {
/** Create a poll message for the gaining client in a transfer. */
static PollMessage createGainingTransferPollMessage(
String targetId,
TransferData transferData,
DateTime now,
Key<ContactHistory> contactHistoryKey) {
String targetId, TransferData transferData, DateTime now, HistoryEntryId contactHistoryId) {
return new PollMessage.OneTime.Builder()
.setRegistrarId(transferData.getGainingRegistrarId())
.setEventTime(transferData.getPendingTransferExpirationTime())
@@ -85,23 +80,19 @@ public class ContactFlowUtils {
transferData.getTransferStatus().isApproved(),
transferData.getTransferRequestTrid(),
now)))
.setContactHistoryId(
new ContactHistoryId(
contactHistoryKey.getParent().getName(), contactHistoryKey.getId()))
.setContactHistoryId(contactHistoryId)
.build();
}
/** Create a poll message for the losing client in a transfer. */
static PollMessage createLosingTransferPollMessage(
String targetId, TransferData transferData, Key<ContactHistory> contactHistoryKey) {
String targetId, TransferData transferData, HistoryEntryId contactHistoryId) {
return new PollMessage.OneTime.Builder()
.setRegistrarId(transferData.getLosingRegistrarId())
.setEventTime(transferData.getPendingTransferExpirationTime())
.setMsg(transferData.getTransferStatus().getMessage())
.setResponseData(ImmutableList.of(createTransferResponse(targetId, transferData)))
.setContactHistoryId(
new ContactHistoryId(
contactHistoryKey.getParent().getName(), contactHistoryKey.getId()))
.setContactHistoryId(contactHistoryId)
.build();
}

View File

@@ -90,10 +90,10 @@ public final class ContactInfoFlow implements Flow {
.setVoiceNumber(contact.getVoiceNumber())
.setFaxNumber(contact.getFaxNumber())
.setEmailAddress(contact.getEmailAddress())
.setCurrentSponsorClientId(contact.getCurrentSponsorRegistrarId())
.setCreationClientId(contact.getCreationRegistrarId())
.setCurrentSponsorRegistrarId(contact.getCurrentSponsorRegistrarId())
.setCreationRegistrarId(contact.getCreationRegistrarId())
.setCreationTime(contact.getCreationTime())
.setLastEppUpdateClientId(contact.getLastEppUpdateRegistrarId())
.setLastEppUpdateRegistrarId(contact.getLastEppUpdateRegistrarId())
.setLastEppUpdateTime(contact.getLastEppUpdateTime())
.setLastTransferTime(contact.getLastTransferTime())
.setAuthInfo(includeAuthInfo ? contact.getAuthInfo() : null)

View File

@@ -26,7 +26,6 @@ import static google.registry.model.reporting.HistoryEntry.Type.CONTACT_TRANSFER
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -93,7 +92,7 @@ public final class ContactTransferApproveFlow implements TransactionalFlow {
// Create a poll message for the gaining client.
PollMessage gainingPollMessage =
createGainingTransferPollMessage(
targetId, newContact.getTransferData(), now, Key.create(contactHistory));
targetId, newContact.getTransferData(), now, contactHistory.getHistoryEntryId());
tm().insertAll(ImmutableSet.of(contactHistory, gainingPollMessage));
tm().update(newContact);
// Delete the billing event and poll messages that were written in case the transfer would have

View File

@@ -26,7 +26,6 @@ import static google.registry.model.reporting.HistoryEntry.Type.CONTACT_TRANSFER
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -73,7 +72,7 @@ public final class ContactTransferCancelFlow implements TransactionalFlow {
@Inject ContactTransferCancelFlow() {}
@Override
public final EppResponse run() throws EppException {
public EppResponse run() throws EppException {
extensionManager.register(MetadataExtension.class);
validateRegistrarIsLoggedIn(registrarId);
extensionManager.validate();
@@ -89,7 +88,7 @@ public final class ContactTransferCancelFlow implements TransactionalFlow {
// Create a poll message for the losing client.
PollMessage losingPollMessage =
createLosingTransferPollMessage(
targetId, newContact.getTransferData(), Key.create(contactHistory));
targetId, newContact.getTransferData(), contactHistory.getHistoryEntryId());
tm().insertAll(ImmutableSet.of(contactHistory, losingPollMessage));
tm().update(newContact);
// Delete the billing event and poll messages that were written in case the transfer would have

View File

@@ -26,7 +26,6 @@ import static google.registry.model.reporting.HistoryEntry.Type.CONTACT_TRANSFER
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -86,7 +85,7 @@ public final class ContactTransferRejectFlow implements TransactionalFlow {
historyBuilder.setType(CONTACT_TRANSFER_REJECT).setContact(newContact).build();
PollMessage gainingPollMessage =
createGainingTransferPollMessage(
targetId, newContact.getTransferData(), now, Key.create(contactHistory));
targetId, newContact.getTransferData(), now, contactHistory.getHistoryEntryId());
tm().insertAll(ImmutableSet.of(contactHistory, gainingPollMessage));
tm().update(newContact);
// Delete the billing event and poll messages that were written in case the transfer would have

View File

@@ -14,7 +14,7 @@
package google.registry.flows.contact;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
import static google.registry.flows.ResourceFlowUtils.verifyAuthInfo;
@@ -28,7 +28,6 @@ import static google.registry.model.reporting.HistoryEntry.Type.CONTACT_TRANSFER
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.config.RegistryConfig.Config;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
@@ -46,6 +45,7 @@ import google.registry.model.eppcommon.StatusValue;
import google.registry.model.eppcommon.Trid;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.transfer.ContactTransferData;
import google.registry.model.transfer.TransferStatus;
@@ -74,21 +74,27 @@ import org.joda.time.Duration;
@ReportingSpec(ActivityReportField.CONTACT_TRANSFER_REQUEST)
public final class ContactTransferRequestFlow implements TransactionalFlow {
private static final ImmutableSet<StatusValue> DISALLOWED_STATUSES = ImmutableSet.of(
StatusValue.CLIENT_TRANSFER_PROHIBITED,
StatusValue.PENDING_DELETE,
StatusValue.SERVER_TRANSFER_PROHIBITED);
private static final ImmutableSet<StatusValue> DISALLOWED_STATUSES =
ImmutableSet.of(
StatusValue.CLIENT_TRANSFER_PROHIBITED,
StatusValue.PENDING_DELETE,
StatusValue.SERVER_TRANSFER_PROHIBITED);
@Inject ExtensionManager extensionManager;
@Inject Optional<AuthInfo> authInfo;
@Inject @RegistrarId String gainingClientId;
@Inject @TargetId String targetId;
@Inject @Config("contactAutomaticTransferLength") Duration automaticTransferLength;
@Inject
@Config("contactAutomaticTransferLength")
Duration automaticTransferLength;
@Inject ContactHistory.Builder historyBuilder;
@Inject Trid trid;
@Inject EppResponse.Builder responseBuilder;
@Inject ContactTransferRequestFlow() {}
@Inject
ContactTransferRequestFlow() {}
@Override
public EppResponse run() throws EppException {
@@ -120,29 +126,31 @@ public final class ContactTransferRequestFlow implements TransactionalFlow {
.setPendingTransferExpirationTime(transferExpirationTime)
.setTransferStatus(TransferStatus.SERVER_APPROVED)
.build();
Key<ContactHistory> contactHistoryKey = createHistoryKey(existingContact, ContactHistory.class);
historyBuilder.setId(contactHistoryKey.getId()).setType(CONTACT_TRANSFER_REQUEST);
HistoryEntryId contactHistoryId = createHistoryEntryId(existingContact);
historyBuilder
.setRevisionId(contactHistoryId.getRevisionId())
.setType(CONTACT_TRANSFER_REQUEST);
// If the transfer is server approved, this message will be sent to the losing registrar. */
PollMessage serverApproveLosingPollMessage =
createLosingTransferPollMessage(targetId, serverApproveTransferData, contactHistoryKey);
createLosingTransferPollMessage(targetId, serverApproveTransferData, contactHistoryId);
// If the transfer is server approved, this message will be sent to the gaining registrar. */
PollMessage serverApproveGainingPollMessage =
createGainingTransferPollMessage(
targetId, serverApproveTransferData, now, contactHistoryKey);
targetId, serverApproveTransferData, now, contactHistoryId);
ContactTransferData pendingTransferData =
serverApproveTransferData
.asBuilder()
.setTransferStatus(TransferStatus.PENDING)
.setServerApproveEntities(
serverApproveGainingPollMessage.getContactRepoId(),
contactHistoryKey.getId(),
contactHistoryId.getRevisionId(),
ImmutableSet.of(
serverApproveGainingPollMessage.createVKey(),
serverApproveLosingPollMessage.createVKey()))
.build();
// When a transfer is requested, a poll message is created to notify the losing registrar.
PollMessage requestPollMessage =
createLosingTransferPollMessage(targetId, pendingTransferData, contactHistoryKey)
createLosingTransferPollMessage(targetId, pendingTransferData, contactHistoryId)
.asBuilder()
.setEventTime(now) // Unlike the serverApprove messages, this applies immediately.
.build();
@@ -165,4 +173,3 @@ public final class ContactTransferRequestFlow implements TransactionalFlow {
.build();
}
}

View File

@@ -53,6 +53,7 @@ import google.registry.flows.custom.DomainCheckFlowCustomLogic.BeforeResponseRet
import google.registry.flows.domain.token.AllocationTokenDomainCheckResults;
import google.registry.flows.domain.token.AllocationTokenFlowUtils;
import google.registry.model.EppResource;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.billing.BillingEvent;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Check;
@@ -70,7 +71,6 @@ import google.registry.model.eppoutput.CheckData.DomainCheck;
import google.registry.model.eppoutput.CheckData.DomainCheckData;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.eppoutput.EppResponse.ResponseExtension;
import google.registry.model.index.ForeignKeyIndex;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.tld.Registry.TldState;
@@ -169,8 +169,8 @@ public final class DomainCheckFlow implements Flow {
// TODO: Use as of date from fee extension v0.12 instead of now, if specified.
.setAsOfDate(now)
.build());
ImmutableMap<String, ForeignKeyIndex<Domain>> existingDomains =
ForeignKeyIndex.load(Domain.class, domainNames, now);
ImmutableMap<String, VKey<Domain>> existingDomains =
ForeignKeyUtils.load(Domain.class, domainNames, now);
Optional<AllocationTokenExtension> allocationTokenExtension =
eppInput.getSingleExtension(AllocationTokenExtension.class);
Optional<AllocationTokenDomainCheckResults> tokenDomainCheckResults =
@@ -227,7 +227,7 @@ public final class DomainCheckFlow implements Flow {
private Optional<String> getMessageForCheck(
InternetDomainName domainName,
ImmutableMap<String, ForeignKeyIndex<Domain>> existingDomains,
ImmutableMap<String, VKey<Domain>> existingDomains,
ImmutableMap<InternetDomainName, String> tokenCheckResults,
ImmutableMap<String, TldState> tldStates,
Optional<AllocationToken> allocationToken) {
@@ -251,7 +251,7 @@ public final class DomainCheckFlow implements Flow {
/** Handle the fee check extension. */
private ImmutableList<? extends ResponseExtension> getResponseExtensions(
ImmutableMap<String, InternetDomainName> domainNames,
ImmutableMap<String, ForeignKeyIndex<Domain>> existingDomains,
ImmutableMap<String, VKey<Domain>> existingDomains,
ImmutableSet<String> availableDomains,
DateTime now,
Optional<AllocationToken> allocationToken)
@@ -297,14 +297,14 @@ public final class DomainCheckFlow implements Flow {
* renewal is part of the cost of a restore.
*
* <p>This may be resource-intensive for large checks of many restore fees, but those are
* comparatively rare, and we are at least using an in-memory cache. Also this will get a lot
* comparatively rare, and we are at least using an in-memory cache. Also, this will get a lot
* nicer in Cloud SQL when we can SELECT just the fields we want rather than having to load the
* entire entity.
*/
private ImmutableMap<String, Domain> loadDomainsForRestoreChecks(
FeeCheckCommandExtension<?, ?> feeCheck,
ImmutableMap<String, InternetDomainName> domainNames,
ImmutableMap<String, ForeignKeyIndex<Domain>> existingDomains) {
ImmutableMap<String, VKey<Domain>> existingDomains) {
ImmutableList<String> restoreCheckDomains;
if (feeCheck instanceof FeeCheckCommandExtensionV06) {
// The V06 fee extension supports specifying the command fees to check on a per-domain basis.
@@ -329,7 +329,7 @@ public final class DomainCheckFlow implements Flow {
ImmutableMap<String, VKey<Domain>> existingDomainsToLoad =
restoreCheckDomains.stream()
.filter(existingDomains::containsKey)
.collect(toImmutableMap(d -> d, d -> existingDomains.get(d).getResourceKey()));
.collect(toImmutableMap(d -> d, existingDomains::get));
ImmutableMap<VKey<? extends EppResource>, EppResource> loadedDomains =
EppResource.loadCached(ImmutableList.copyOf(existingDomainsToLoad.values()));
return ImmutableMap.copyOf(

View File

@@ -59,7 +59,6 @@ import com.google.auto.value.AutoValue;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.net.InternetDomainName;
import com.googlecode.objectify.Key;
import google.registry.dns.DnsQueue;
import google.registry.flows.EppException;
import google.registry.flows.EppException.CommandUseErrorException;
@@ -87,7 +86,6 @@ import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand;
import google.registry.model.domain.DomainCommand.Create;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.Period;
import google.registry.model.domain.fee.FeeCreateCommandExtension;
@@ -105,13 +103,13 @@ import google.registry.model.eppinput.EppInput;
import google.registry.model.eppinput.ResourceCommand;
import google.registry.model.eppoutput.CreateData.DomainCreateData;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.index.EppResourceIndex;
import google.registry.model.poll.PendingActionNotificationResponse.DomainPendingActionNotificationResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.poll.PollMessage.Autorenew;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.tld.Registry.TldState;
@@ -153,6 +151,7 @@ import org.joda.time.Duration;
* @error {@link DomainCreateFlow.MustHaveSignedMarksInCurrentPhaseException}
* @error {@link DomainCreateFlow.NoGeneralRegistrationsInCurrentPhaseException}
* @error {@link DomainCreateFlow.NoTrademarkedRegistrationsBeforeSunriseException}
* @error {@link DomainCreateFlow.PackageDomainRegisteredForTooManyYearsException}
* @error {@link DomainCreateFlow.SignedMarksOnlyDuringSunriseException}
* @error {@link DomainFlowTmchUtils.NoMarksFoundMatchingDomainException}
* @error {@link DomainFlowTmchUtils.FoundMarkNotYetValidException}
@@ -249,7 +248,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
validateRegistrationPeriod(years);
verifyResourceDoesNotExist(Domain.class, targetId, now, registrarId);
// Validate that this is actually a legal domain name on a TLD that the registrar has access to.
InternetDomainName domainName = validateDomainName(command.getFullyQualifiedDomainName());
InternetDomainName domainName = validateDomainName(command.getDomainName());
String domainLabel = domainName.parts().get(0);
Registry registry = Registry.get(domainName.parent().toString());
validateCreateCommandContactsAndNameservers(command, registry, domainName);
@@ -328,14 +327,14 @@ public final class DomainCreateFlow implements TransactionalFlow {
FeesAndCredits feesAndCredits =
pricingLogic.getCreatePrice(
registry, targetId, now, years, isAnchorTenant, allocationToken);
validateFeeChallenge(targetId, now, feeCreate, feesAndCredits);
validateFeeChallenge(feeCreate, feesAndCredits);
Optional<SecDnsCreateExtension> secDnsCreate =
validateSecDnsExtension(eppInput.getSingleExtension(SecDnsCreateExtension.class));
DateTime registrationExpirationTime = leapSafeAddYears(now, years);
String repoId = createDomainRepoId(allocateId(), registry.getTldStr());
long historyRevisionId = allocateId();
DomainHistoryId domainHistoryId = new DomainHistoryId(repoId, historyRevisionId);
historyBuilder.setId(historyRevisionId);
HistoryEntryId domainHistoryId = new HistoryEntryId(repoId, historyRevisionId);
historyBuilder.setRevisionId(historyRevisionId);
// Bill for the create.
BillingEvent.OneTime createBillingEvent =
createOneTimeBillingEvent(
@@ -391,6 +390,9 @@ public final class DomainCreateFlow implements TransactionalFlow {
.build();
if (allocationToken.isPresent()
&& allocationToken.get().getTokenType().equals(TokenType.PACKAGE)) {
if (years > 1) {
throw new PackageDomainRegisteredForTooManyYearsException(allocationToken.get().getToken());
}
domain =
domain.asBuilder().setCurrentPackageToken(allocationToken.get().createVKey()).build();
}
@@ -400,14 +402,12 @@ public final class DomainCreateFlow implements TransactionalFlow {
entitiesToSave.add(
createNameCollisionOneTimePollMessage(targetId, domainHistory, registrarId, now));
}
entitiesToSave.add(
domain,
domainHistory,
EppResourceIndex.create(Key.create(domain)));
entitiesToSave.add(domain, domainHistory);
if (allocationToken.isPresent()
&& TokenType.SINGLE_USE.equals(allocationToken.get().getTokenType())) {
entitiesToSave.add(
allocationTokenFlowUtils.redeemToken(allocationToken.get(), domainHistory.createVKey()));
allocationTokenFlowUtils.redeemToken(
allocationToken.get(), domainHistory.getHistoryEntryId()));
}
enqueueTasks(domain, hasSignedMarks, hasClaimsNotice);
@@ -563,7 +563,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
boolean isReserved,
int years,
FeesAndCredits feesAndCredits,
DomainHistoryId domainHistoryId,
HistoryEntryId domainHistoryId,
Optional<AllocationToken> allocationToken,
DateTime now) {
ImmutableSet.Builder<Flag> flagsBuilder = new ImmutableSet.Builder<>();
@@ -597,7 +597,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
}
private Recurring createAutorenewBillingEvent(
DomainHistoryId domainHistoryId,
HistoryEntryId domainHistoryId,
DateTime registrationExpirationTime,
RenewalPriceInfo renewalpriceInfo) {
return new BillingEvent.Recurring.Builder()
@@ -614,7 +614,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
}
private Autorenew createAutorenewPollMessage(
DomainHistoryId domainHistoryId, DateTime registrationExpirationTime) {
HistoryEntryId domainHistoryId, DateTime registrationExpirationTime) {
return new PollMessage.Autorenew.Builder()
.setTargetId(targetId)
.setRegistrarId(registrarId)
@@ -635,15 +635,12 @@ public final class DomainCreateFlow implements TransactionalFlow {
.setEventTime(createBillingEvent.getEventTime())
.setBillingTime(createBillingEvent.getBillingTime())
.setFlags(createBillingEvent.getFlags())
.setDomainHistoryId(createBillingEvent.getDomainHistoryId())
.setDomainHistoryId(createBillingEvent.getHistoryEntryId())
.build();
}
private static PollMessage.OneTime createNameCollisionOneTimePollMessage(
String fullyQualifiedDomainName,
HistoryEntry historyEntry,
String registrarId,
DateTime now) {
String domainName, HistoryEntry historyEntry, String registrarId, DateTime now) {
return new PollMessage.OneTime.Builder()
.setRegistrarId(registrarId)
.setEventTime(now)
@@ -651,7 +648,7 @@ public final class DomainCreateFlow implements TransactionalFlow {
.setResponseData(
ImmutableList.of(
DomainPendingActionNotificationResponse.create(
fullyQualifiedDomainName, true, historyEntry.getTrid(), now)))
domainName, true, historyEntry.getTrid(), now)))
.setHistoryEntry(historyEntry)
.build();
}
@@ -678,11 +675,11 @@ public final class DomainCreateFlow implements TransactionalFlow {
Optional<AllocationToken> allocationToken,
FeesAndCredits feesAndCredits) {
if (isAnchorTenant) {
if (allocationToken.isPresent()) {
checkArgument(
allocationToken.get().getRenewalPriceBehavior() != RenewalPriceBehavior.SPECIFIED,
"Renewal price behavior cannot be SPECIFIED for anchor tenant");
}
allocationToken.ifPresent(
token ->
checkArgument(
token.getRenewalPriceBehavior() != RenewalPriceBehavior.SPECIFIED,
"Renewal price behavior cannot be SPECIFIED for anchor tenant"));
return RenewalPriceInfo.create(RenewalPriceBehavior.NONPREMIUM, null);
} else if (allocationToken.isPresent()
&& allocationToken.get().getRenewalPriceBehavior() == RenewalPriceBehavior.SPECIFIED) {
@@ -709,9 +706,12 @@ public final class DomainCreateFlow implements TransactionalFlow {
private static ImmutableList<FeeTransformResponseExtension> createResponseExtensions(
Optional<FeeCreateCommandExtension> feeCreate, FeesAndCredits feesAndCredits) {
return feeCreate.isPresent()
? ImmutableList.of(createFeeCreateResponse(feeCreate.get(), feesAndCredits))
: ImmutableList.of();
return feeCreate
.map(
feeCreateCommandExtension ->
ImmutableList.of(
createFeeCreateResponse(feeCreateCommandExtension, feesAndCredits)))
.orElseGet(ImmutableList::of);
}
/** Signed marks are only allowed during sunrise. */
@@ -755,4 +755,14 @@ public final class DomainCreateFlow implements TransactionalFlow {
ANCHOR_TENANT_CREATE_VALID_YEARS, invalidYears));
}
}
/** Package domain registered for too many years. */
static class PackageDomainRegisteredForTooManyYearsException extends CommandUseErrorException {
public PackageDomainRegisteredForTooManyYearsException(String token) {
super(
String.format(
"The package token %s cannot be used to register names for longer than 1 year.",
token));
}
}
}

View File

@@ -16,7 +16,7 @@ package google.registry.flows.domain;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Strings.isNullOrEmpty;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.persistEntityChanges;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -43,7 +43,6 @@ import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Sets;
import com.googlecode.objectify.Key;
import google.registry.batch.AsyncTaskEnqueuer;
import google.registry.dns.DnsQueue;
import google.registry.flows.EppException;
@@ -66,7 +65,6 @@ import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.fee.BaseFee.FeeType;
import google.registry.model.domain.fee.Credit;
@@ -88,6 +86,7 @@ import google.registry.model.poll.PendingActionNotificationResponse.DomainPendin
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.tld.Registry.TldType;
@@ -181,8 +180,8 @@ public final class DomainDeleteFlow implements TransactionalFlow {
? Duration.ZERO
// By default, this should be 30 days of grace, and 5 days of pending delete.
: redemptionGracePeriodLength.plus(pendingDeleteLength);
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
historyBuilder.setId(domainHistoryKey.getId());
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder.setRevisionId(domainHistoryId.getRevisionId());
DateTime deletionTime = now.plus(durationUntilDelete);
if (durationUntilDelete.equals(Duration.ZERO)) {
builder.setDeletionTime(now).setStatusValues(null);
@@ -214,7 +213,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
// it is synchronous).
if (durationUntilDelete.isLongerThan(Duration.ZERO) || isSuperuser) {
PollMessage.OneTime deletePollMessage =
createDeletePollMessage(existingDomain, domainHistoryKey, deletionTime);
createDeletePollMessage(existingDomain, domainHistoryId, deletionTime);
entitiesToSave.add(deletePollMessage);
builder.setDeletePollMessage(deletePollMessage.createVKey());
}
@@ -224,7 +223,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
if (durationUntilDelete.isLongerThan(Duration.ZERO)
&& !registrarId.equals(existingDomain.getPersistedCurrentSponsorRegistrarId())) {
entitiesToSave.add(
createImmediateDeletePollMessage(existingDomain, domainHistoryKey, now, deletionTime));
createImmediateDeletePollMessage(existingDomain, domainHistoryId, now, deletionTime));
}
// Cancel any grace periods that were still active, and set the expiration time accordingly.
@@ -233,14 +232,9 @@ public final class DomainDeleteFlow implements TransactionalFlow {
// No cancellation is written if the grace period was not for a billable event.
if (gracePeriod.hasBillingEvent()) {
entitiesToSave.add(
BillingEvent.Cancellation.forGracePeriod(
gracePeriod,
now,
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()),
targetId));
BillingEvent.Cancellation.forGracePeriod(gracePeriod, now, domainHistoryId, targetId));
if (gracePeriod.getOneTimeBillingEvent() != null) {
// Take the amount of amount of registration time being refunded off the expiration time.
// Take the amount of registration time being refunded off the expiration time.
// This can be either add grace periods or renew grace periods.
BillingEvent.OneTime oneTime = tm().loadByKey(gracePeriod.getOneTimeBillingEvent());
newExpirationTime = newExpirationTime.minusYears(oneTime.getPeriodYears());
@@ -262,7 +256,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
BillingEvent.Recurring recurringBillingEvent =
updateAutorenewRecurrenceEndTime(
existingDomain, existingRecurring, now, domainHistory.getDomainHistoryId());
existingDomain, existingRecurring, now, domainHistory.getHistoryEntryId());
// If there's a pending transfer, the gaining client's autorenew billing
// event and poll message will already have been deleted in
// ResourceDeleteFlow since it's listed in serverApproveEntities.
@@ -343,7 +337,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
}
private PollMessage.OneTime createDeletePollMessage(
Domain existingDomain, Key<DomainHistory> domainHistoryKey, DateTime deletionTime) {
Domain existingDomain, HistoryEntryId domainHistoryId, DateTime deletionTime) {
Optional<MetadataExtension> metadataExtension =
eppInput.getSingleExtension(MetadataExtension.class);
boolean hasMetadataMessage =
@@ -362,21 +356,16 @@ public final class DomainDeleteFlow implements TransactionalFlow {
ImmutableList.of(
DomainPendingActionNotificationResponse.create(
existingDomain.getDomainName(), true, trid, deletionTime)))
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
}
private PollMessage.OneTime createImmediateDeletePollMessage(
Domain existingDomain,
Key<DomainHistory> domainHistoryKey,
DateTime now,
DateTime deletionTime) {
Domain existingDomain, HistoryEntryId domainHistoryId, DateTime now, DateTime deletionTime) {
return new PollMessage.OneTime.Builder()
.setRegistrarId(existingDomain.getPersistedCurrentSponsorRegistrarId())
.setEventTime(now)
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.setMsg(
String.format(
"Domain %s was deleted by registry administrator with final deletion effective: %s",

View File

@@ -37,7 +37,6 @@ import static google.registry.model.tld.label.ReservationType.FULLY_BLOCKED;
import static google.registry.model.tld.label.ReservationType.NAME_COLLISION;
import static google.registry.model.tld.label.ReservationType.RESERVED_FOR_ANCHOR_TENANT;
import static google.registry.model.tld.label.ReservationType.RESERVED_FOR_SPECIFIC_USE;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.pricing.PricingEngineProxy.isDomainPremium;
import static google.registry.util.CollectionUtils.nullToEmpty;
@@ -88,7 +87,6 @@ import google.registry.model.domain.DomainCommand.CreateOrUpdate;
import google.registry.model.domain.DomainCommand.InvalidReferencesException;
import google.registry.model.domain.DomainCommand.Update;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.ForeignKeyedDesignatedContact;
import google.registry.model.domain.Period;
import google.registry.model.domain.fee.BaseFee;
@@ -105,7 +103,7 @@ import google.registry.model.domain.launch.LaunchNotice;
import google.registry.model.domain.launch.LaunchNotice.InvalidChecksumException;
import google.registry.model.domain.launch.LaunchPhase;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.domain.secdns.DelegationSignerData;
import google.registry.model.domain.secdns.DomainDsData;
import google.registry.model.domain.secdns.SecDnsCreateExtension;
import google.registry.model.domain.secdns.SecDnsInfoExtension;
import google.registry.model.domain.secdns.SecDnsUpdateExtension;
@@ -121,7 +119,7 @@ import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.Registrar.State;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.tld.Registry;
import google.registry.model.tld.Registry.TldState;
import google.registry.model.tld.Registry.TldType;
@@ -316,14 +314,14 @@ public class DomainFlowUtils {
}
/** Check that the DS data that will be set on a domain is valid. */
static void validateDsData(Set<DelegationSignerData> dsData) throws EppException {
static void validateDsData(Set<DomainDsData> dsData) throws EppException {
if (dsData != null) {
if (dsData.size() > MAX_DS_RECORDS_PER_DOMAIN) {
throw new TooManyDsRecordsException(
String.format(
"A maximum of %s DS records are allowed per domain.", MAX_DS_RECORDS_PER_DOMAIN));
}
ImmutableList<DelegationSignerData> invalidAlgorithms =
ImmutableList<DomainDsData> invalidAlgorithms =
dsData.stream()
.filter(ds -> !validateAlgorithm(ds.getAlgorithm()))
.collect(toImmutableList());
@@ -333,7 +331,7 @@ public class DomainFlowUtils {
"Domain contains DS record(s) with an invalid algorithm wire value: %s",
invalidAlgorithms));
}
ImmutableList<DelegationSignerData> invalidDigestTypes =
ImmutableList<DomainDsData> invalidDigestTypes =
dsData.stream()
.filter(ds -> !DigestType.fromWireValue(ds.getDigestType()).isPresent())
.collect(toImmutableList());
@@ -343,7 +341,7 @@ public class DomainFlowUtils {
"Domain contains DS record(s) with an invalid digest type: %s",
invalidDigestTypes));
}
ImmutableList<DelegationSignerData> digestsWithInvalidDigestLength =
ImmutableList<DomainDsData> digestsWithInvalidDigestLength =
dsData.stream()
.filter(
ds ->
@@ -429,7 +427,7 @@ public class DomainFlowUtils {
contacts.stream()
.collect(
toImmutableSetMultimap(
DesignatedContact::getType, contact -> contact.getContactKey()));
DesignatedContact::getType, DesignatedContact::getContactKey));
// If any contact type has multiple contacts:
if (contactsByType.asMap().values().stream().anyMatch(v -> v.size() > 1)) {
@@ -590,7 +588,7 @@ public class DomainFlowUtils {
Domain domain,
Recurring existingRecurring,
DateTime newEndTime,
@Nullable DomainHistoryId historyId) {
@Nullable HistoryEntryId historyId) {
Optional<PollMessage.Autorenew> autorenewPollMessage =
tm().loadByKeyIfPresent(domain.getAutorenewPollMessage());
@@ -609,7 +607,7 @@ public class DomainFlowUtils {
historyId, "Cannot create a new autorenew poll message without a domain history id");
updatedAutorenewPollMessage =
newAutorenewPollMessage(domain)
.setId((Long) domain.getAutorenewPollMessage().getSqlKey())
.setId((Long) domain.getAutorenewPollMessage().getKey())
.setAutorenewEndTime(newEndTime)
.setDomainHistoryId(historyId)
.build();
@@ -683,7 +681,13 @@ public class DomainFlowUtils {
builder.setAvailIfSupported(true);
fees =
pricingLogic
.getCreatePrice(registry, domainNameString, now, years, false, allocationToken)
.getCreatePrice(
registry,
domainNameString,
now,
years,
isAnchorTenant(domainName, allocationToken, Optional.empty()),
allocationToken)
.getFees();
}
break;
@@ -773,8 +777,6 @@ public class DomainFlowUtils {
* domain names.
*/
public static void validateFeeChallenge(
String domainName,
DateTime priceTime,
final Optional<? extends FeeTransformCommandExtension> feeCommand,
FeesAndCredits feesAndCredits)
throws EppException {
@@ -920,16 +922,15 @@ public class DomainFlowUtils {
* and we are going to ignore it; clients who don't care about secDNS can just ignore it.
*/
static void addSecDnsExtensionIfPresent(
ImmutableList.Builder<ResponseExtension> extensions,
ImmutableSet<DelegationSignerData> dsData) {
ImmutableList.Builder<ResponseExtension> extensions, ImmutableSet<DomainDsData> dsData) {
if (!dsData.isEmpty()) {
extensions.add(SecDnsInfoExtension.create(dsData));
}
}
/** Update {@link DelegationSignerData} based on an update extension command. */
static ImmutableSet<DelegationSignerData> updateDsData(
ImmutableSet<DelegationSignerData> oldDsData, SecDnsUpdateExtension secDnsUpdate)
/** Update {@link DomainDsData} based on an update extension command. */
static ImmutableSet<DomainDsData> updateDsData(
ImmutableSet<DomainDsData> oldDsData, SecDnsUpdateExtension secDnsUpdate)
throws EppException {
// We don't support 'urgent' because we do everything as fast as we can anyways.
if (Boolean.TRUE.equals(secDnsUpdate.getUrgent())) { // We allow both false and null.
@@ -948,8 +949,8 @@ public class DomainFlowUtils {
if (remove != null && Boolean.FALSE.equals(remove.getAll())) {
throw new SecDnsAllUsageException(); // Explicit all=false is meaningless.
}
Set<DelegationSignerData> toAdd = (add == null) ? ImmutableSet.of() : add.getDsData();
Set<DelegationSignerData> toRemove =
Set<DomainDsData> toAdd = (add == null) ? ImmutableSet.of() : add.getDsData();
Set<DomainDsData> toRemove =
(remove == null)
? ImmutableSet.of()
: (remove.getAll() == null) ? remove.getDsData() : oldDsData;
@@ -1001,9 +1002,9 @@ public class DomainFlowUtils {
validateRegistrantAllowedOnTld(tld, command.getRegistrantContactId());
validateNoDuplicateContacts(command.getContacts());
validateRequiredContactsPresent(command.getRegistrant(), command.getContacts());
ImmutableSet<String> fullyQualifiedHostNames = command.getNameserverFullyQualifiedHostNames();
validateNameserversCountForTld(tld, domainName, fullyQualifiedHostNames.size());
validateNameserversAllowedOnTld(tld, fullyQualifiedHostNames);
ImmutableSet<String> hostNames = command.getNameserverHostNames();
validateNameserversCountForTld(tld, domainName, hostNames.size());
validateNameserversAllowedOnTld(tld, hostNames);
}
/** Validate the secDNS extension, if present. */
@@ -1136,9 +1137,9 @@ public class DomainFlowUtils {
Duration maxSearchPeriod,
final ImmutableSet<TransactionReportField> cancelableFields) {
List<? extends HistoryEntry> recentHistoryEntries =
List<DomainHistory> recentHistoryEntries =
findRecentHistoryEntries(domain, now, maxSearchPeriod);
Optional<? extends HistoryEntry> entryToCancel =
Optional<DomainHistory> entryToCancel =
Streams.findLast(
recentHistoryEntries.stream()
.filter(
@@ -1175,11 +1176,10 @@ public class DomainFlowUtils {
return recordsBuilder.build();
}
private static List<? extends HistoryEntry> findRecentHistoryEntries(
private static List<DomainHistory> findRecentHistoryEntries(
Domain domain, DateTime now, Duration maxSearchPeriod) {
return jpaTm()
.query(
"FROM DomainHistory WHERE modificationTime >= :beginning AND domainRepoId = "
return tm().query(
"FROM DomainHistory WHERE modificationTime >= :beginning AND repoId = "
+ ":repoId ORDER BY modificationTime ASC",
DomainHistory.class)
.setParameter("beginning", now.minus(maxSearchPeriod))
@@ -1542,11 +1542,11 @@ public class DomainFlowUtils {
/** Nameservers are not allow-listed for this TLD. */
public static class NameserversNotAllowedForTldException
extends StatusProhibitsOperationException {
public NameserversNotAllowedForTldException(Set<String> fullyQualifiedHostNames) {
public NameserversNotAllowedForTldException(Set<String> hostNames) {
super(
String.format(
"Nameservers '%s' are not allow-listed for this TLD",
Joiner.on(',').join(fullyQualifiedHostNames)));
Joiner.on(',').join(hostNames)));
}
}

View File

@@ -105,32 +105,33 @@ public final class DomainInfoFlow implements Flow {
verifyOptionalAuthInfo(authInfo, domain);
flowCustomLogic.afterValidation(
AfterValidationParameters.newBuilder().setDomain(domain).build());
HostsRequest hostsRequest = ((Info) resourceCommand).getHostsRequest();
// Registrars can only see a few fields on unauthorized domains.
// This is a policy decision that is left up to us by the rfcs.
DomainInfoData.Builder infoBuilder =
DomainInfoData.newBuilder()
.setFullyQualifiedDomainName(domain.getDomainName())
.setDomainName(domain.getDomainName())
.setRepoId(domain.getRepoId())
.setCurrentSponsorClientId(domain.getCurrentSponsorRegistrarId())
.setCurrentSponsorRegistrarId(domain.getCurrentSponsorRegistrarId())
.setStatusValues(domain.getStatusValues())
.setNameservers(
hostsRequest.requestDelegated() ? domain.loadNameserverHostNames() : null)
.setCreationTime(domain.getCreationTime())
.setLastEppUpdateTime(domain.getLastEppUpdateTime())
.setRegistrationExpirationTime(domain.getRegistrationExpirationTime())
.setLastTransferTime(domain.getLastTransferTime())
.setRegistrant(
tm().transact(() -> tm().loadByKey(domain.getRegistrant())).getContactId());
// If authInfo is non-null, then the caller is authorized to see the full information since we
// will have already verified the authInfo is valid.
if (registrarId.equals(domain.getCurrentSponsorRegistrarId()) || authInfo.isPresent()) {
HostsRequest hostsRequest = ((Info) resourceCommand).getHostsRequest();
infoBuilder
.setStatusValues(domain.getStatusValues())
.setContacts(
tm().transact(() -> loadForeignKeyedDesignatedContacts(domain.getContacts())))
.setNameservers(hostsRequest.requestDelegated() ? domain.loadNameserverHostNames() : null)
.setSubordinateHosts(
hostsRequest.requestSubordinate() ? domain.getSubordinateHosts() : null)
.setCreationClientId(domain.getCreationRegistrarId())
.setCreationTime(domain.getCreationTime())
.setLastEppUpdateClientId(domain.getLastEppUpdateRegistrarId())
.setLastEppUpdateTime(domain.getLastEppUpdateTime())
.setRegistrationExpirationTime(domain.getRegistrationExpirationTime())
.setLastTransferTime(domain.getLastTransferTime())
.setCreationRegistrarId(domain.getCreationRegistrarId())
.setLastEppUpdateRegistrarId(domain.getLastEppUpdateRegistrarId())
.setAuthInfo(domain.getAuthInfo());
}
BeforeResponseReturnData responseData =

View File

@@ -14,7 +14,7 @@
package google.registry.flows.domain;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.persistEntityChanges;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -38,7 +38,6 @@ import static google.registry.util.DateTimeUtils.leapSafeAddYears;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.EppException.ParameterValueRangeErrorException;
import google.registry.flows.ExtensionManager;
@@ -62,7 +61,6 @@ import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Renew;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.DomainRenewData;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.Period;
@@ -83,6 +81,7 @@ import google.registry.model.eppoutput.EppResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import java.util.Optional;
@@ -100,7 +99,7 @@ import org.joda.time.Duration;
*
* <p>ICANN prohibits any registration from being longer than ten years so if the request would
* result in a registration greater than ten years long it will fail. In practice this means it's
* impossible to request a ten year renewal, since that will always cause the new registration to be
* impossible to request a ten-year renewal, since that will always cause the new registration to be
* longer than 10 years unless it comes in at the exact millisecond that the domain would have
* expired.
*
@@ -200,44 +199,36 @@ public final class DomainRenewFlow implements TransactionalFlow {
now,
years,
existingRecurringBillingEvent);
validateFeeChallenge(targetId, now, feeRenew, feesAndCredits);
validateFeeChallenge(feeRenew, feesAndCredits);
flowCustomLogic.afterValidation(
AfterValidationParameters.newBuilder()
.setExistingDomain(existingDomain)
.setNow(now)
.setYears(years)
.build());
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
historyBuilder.setId(domainHistoryKey.getId());
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder.setRevisionId(domainHistoryId.getRevisionId());
String tld = existingDomain.getTld();
// Bill for this explicit renew itself.
BillingEvent.OneTime explicitRenewEvent =
createRenewBillingEvent(
tld, feesAndCredits.getTotalCost(), years, domainHistoryKey, allocationToken, now);
tld, feesAndCredits.getTotalCost(), years, domainHistoryId, allocationToken, now);
// Create a new autorenew billing event and poll message starting at the new expiration time.
BillingEvent.Recurring newAutorenewEvent =
newAutorenewBillingEvent(existingDomain)
.setEventTime(newExpirationTime)
.setRenewalPrice(existingRecurringBillingEvent.getRenewalPrice().orElse(null))
.setRenewalPriceBehavior(existingRecurringBillingEvent.getRenewalPriceBehavior())
.setDomainHistoryId(
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
PollMessage.Autorenew newAutorenewPollMessage =
newAutorenewPollMessage(existingDomain)
.setEventTime(newExpirationTime)
.setDomainHistoryId(
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
// End the old autorenew billing event and poll message now. This may delete the poll message.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
updateAutorenewRecurrenceEndTime(
existingDomain,
existingRecurring,
now,
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()));
updateAutorenewRecurrenceEndTime(existingDomain, existingRecurring, now, domainHistoryId);
Domain newDomain =
existingDomain
.asBuilder()
@@ -260,7 +251,8 @@ public final class DomainRenewFlow implements TransactionalFlow {
if (allocationToken.isPresent()
&& TokenType.SINGLE_USE.equals(allocationToken.get().getTokenType())) {
entitiesToSave.add(
allocationTokenFlowUtils.redeemToken(allocationToken.get(), domainHistory.createVKey()));
allocationTokenFlowUtils.redeemToken(
allocationToken.get(), domainHistory.getHistoryEntryId()));
}
EntityChanges entityChanges =
flowCustomLogic.beforeSave(
@@ -339,7 +331,7 @@ public final class DomainRenewFlow implements TransactionalFlow {
String tld,
Money renewCost,
int years,
Key<DomainHistory> domainHistoryKey,
HistoryEntryId domainHistoryId,
Optional<AllocationToken> allocationToken,
DateTime now) {
return new BillingEvent.OneTime.Builder()
@@ -355,27 +347,27 @@ public final class DomainRenewFlow implements TransactionalFlow {
.map(AllocationToken::createVKey)
.orElse(null))
.setBillingTime(now.plus(Registry.get(tld).getRenewGracePeriodLength()))
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
}
private ImmutableList<FeeTransformResponseExtension> createResponseExtensions(
FeesAndCredits feesAndCredits, Optional<FeeRenewCommandExtension> feeRenew) {
return feeRenew.isPresent()
? ImmutableList.of(
feeRenew
.get()
.createResponseBuilder()
.setCurrency(feesAndCredits.getCurrency())
.setFees(
ImmutableList.of(
Fee.create(
feesAndCredits.getRenewCost().getAmount(),
FeeType.RENEW,
feesAndCredits.hasPremiumFeesOfType(FeeType.RENEW))))
.build())
: ImmutableList.of();
return feeRenew
.map(
feeRenewCommandExtension ->
ImmutableList.of(
feeRenewCommandExtension
.createResponseBuilder()
.setCurrency(feesAndCredits.getCurrency())
.setFees(
ImmutableList.of(
Fee.create(
feesAndCredits.getRenewCost().getAmount(),
FeeType.RENEW,
feesAndCredits.hasPremiumFeesOfType(FeeType.RENEW))))
.build()))
.orElseGet(ImmutableList::of);
}
/** The current expiration date is incorrect. */

View File

@@ -14,7 +14,7 @@
package google.registry.flows.domain;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
import static google.registry.flows.ResourceFlowUtils.verifyOptionalAuthInfo;
@@ -34,7 +34,6 @@ import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.net.InternetDomainName;
import com.googlecode.objectify.Key;
import google.registry.dns.DnsQueue;
import google.registry.flows.EppException;
import google.registry.flows.EppException.CommandUseErrorException;
@@ -52,7 +51,6 @@ import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Update;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.fee.BaseFee.FeeType;
import google.registry.model.domain.fee.Fee;
import google.registry.model.domain.fee.FeeTransformResponseExtension;
@@ -67,6 +65,7 @@ import google.registry.model.eppoutput.EppResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import java.util.Optional;
@@ -85,12 +84,12 @@ import org.joda.time.DateTime;
*
* <p>This flow is called a restore "request" because technically it is only supposed to signal that
* the registrar requests the restore, which the registry can choose to process or not based on a
* restore report that is submitted through an out of band process and details the request. However,
* in practice this flow does the restore immediately. This is allowable because all of the fields
* on a restore report are optional or have default values, and so by policy when the request comes
* in we consider it to have been accompanied by a default-initialized report which we auto-approve.
* restore report that is submitted through an out-of-band process and details the request. However,
* in practice this flow does the restore immediately. This is allowable because all the fields on a
* restore report are optional or have default values, and so by policy when the request comes in we
* consider it to have been accompanied by a default-initialized report which we auto-approve.
*
* <p>Restores cost a fixed restore fee plus a one year renewal fee for the domain. The domain is
* <p>Restores cost a fixed restore fee plus a one-year renewal fee for the domain. The domain is
* restored to a single year expiration starting at the restore time, regardless of what the
* original expiration time was.
*
@@ -147,10 +146,8 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
Optional<FeeUpdateCommandExtension> feeUpdate =
eppInput.getSingleExtension(FeeUpdateCommandExtension.class);
verifyRestoreAllowed(command, existingDomain, feeUpdate, feesAndCredits, now);
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
historyBuilder.setId(domainHistoryKey.getId());
DomainHistoryId domainHistoryId =
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId());
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder.setRevisionId(domainHistoryId.getRevisionId());
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
DateTime newExpirationTime =
@@ -175,9 +172,7 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
newAutorenewPollMessage(existingDomain)
.setEventTime(newExpirationTime)
.setAutorenewEndTime(END_OF_TIME)
.setDomainHistoryId(
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
Domain newDomain =
performRestore(
@@ -231,7 +226,7 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
if (!existingDomain.getGracePeriodStatuses().contains(GracePeriodStatus.REDEMPTION)) {
throw new DomainNotEligibleForRestoreException();
}
validateFeeChallenge(targetId, now, feeUpdate, feesAndCredits);
validateFeeChallenge(feeUpdate, feesAndCredits);
}
private static Domain performRestore(
@@ -259,17 +254,17 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
}
private OneTime createRenewBillingEvent(
DomainHistoryId domainHistoryId, Money renewCost, DateTime now) {
HistoryEntryId domainHistoryId, Money renewCost, DateTime now) {
return prepareBillingEvent(domainHistoryId, renewCost, now).setReason(Reason.RENEW).build();
}
private BillingEvent.OneTime createRestoreBillingEvent(
DomainHistoryId domainHistoryId, Money restoreCost, DateTime now) {
HistoryEntryId domainHistoryId, Money restoreCost, DateTime now) {
return prepareBillingEvent(domainHistoryId, restoreCost, now).setReason(Reason.RESTORE).build();
}
private OneTime.Builder prepareBillingEvent(
DomainHistoryId domainHistoryId, Money cost, DateTime now) {
HistoryEntryId domainHistoryId, Money cost, DateTime now) {
return new BillingEvent.OneTime.Builder()
.setTargetId(targetId)
.setRegistrarId(registrarId)
@@ -297,15 +292,16 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
FeeType.RENEW,
feesAndCredits.hasPremiumFeesOfType(FeeType.RENEW)));
}
return feeUpdate.isPresent()
? ImmutableList.of(
feeUpdate
.get()
.createResponseBuilder()
.setCurrency(feesAndCredits.getCurrency())
.setFees(fees.build())
.build())
: ImmutableList.of();
return feeUpdate
.map(
feeUpdateCommandExtension ->
ImmutableList.of(
feeUpdateCommandExtension
.createResponseBuilder()
.setCurrency(feesAndCredits.getCurrency())
.setFees(fees.build())
.build()))
.orElseGet(ImmutableList::of);
}
/** Restore command cannot have other changes specified. */

View File

@@ -15,7 +15,7 @@
package google.registry.flows.domain;
import static com.google.common.collect.Iterables.getOnlyElement;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.computeExDateForApprovalTime;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -36,7 +36,6 @@ import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -50,9 +49,9 @@ import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.domain.rgp.GracePeriodStatus;
@@ -62,12 +61,14 @@ import google.registry.model.eppinput.EppInput;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.DomainTransferData;
import google.registry.model.transfer.TransferStatus;
import java.util.Optional;
import javax.inject.Inject;
import org.joda.money.Money;
import org.joda.time.DateTime;
/**
@@ -146,8 +147,10 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
// Create a transfer billing event for 1 year, unless the superuser extension was used to set
// the transfer period to zero. There is not a transfer cost if the transfer period is zero.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
historyBuilder.setId(domainHistoryKey.getId());
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder.setRevisionId(domainHistoryId.getRevisionId());
boolean hasPackageToken = existingDomain.getCurrentPackageToken().isPresent();
Money renewalPrice = hasPackageToken ? null : existingRecurring.getRenewalPrice().orElse(null);
Optional<BillingEvent.OneTime> billingEvent =
transferData.getTransferPeriod().getValue() == 0
? Optional.empty()
@@ -163,14 +166,16 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
Registry.get(tld),
targetId,
transferData.getTransferRequestTime(),
existingRecurring)
// When removing a domain from a package it should return to the
// default recurring billing behavior so the existing recurring
// billing event should not be passed in.
hasPackageToken ? null : existingRecurring)
.getRenewCost())
.setEventTime(now)
.setBillingTime(now.plus(Registry.get(tld).getTransferGracePeriodLength()))
.setDomainHistoryId(
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build());
ImmutableList.Builder<ImmutableObject> entitiesToSave = new ImmutableList.Builder<>();
// If we are within an autorenew grace period, cancel the autorenew billing event and don't
// increase the registration time, since the transfer subsumes the autorenew's extra year.
@@ -185,20 +190,12 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
if (billingEvent.isPresent()) {
entitiesToSave.add(
BillingEvent.Cancellation.forGracePeriod(
autorenewGrace,
now,
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()),
targetId));
autorenewGrace, now, domainHistoryId, targetId));
}
}
// Close the old autorenew event and poll message at the transfer time (aka now). This may end
// up deleting the poll message.
updateAutorenewRecurrenceEndTime(
existingDomain,
existingRecurring,
now,
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()));
updateAutorenewRecurrenceEndTime(existingDomain, existingRecurring, now, domainHistoryId);
DateTime newExpirationTime =
computeExDateForApprovalTime(existingDomain, now, transferData.getTransferPeriod());
// Create a new autorenew event starting at the expiration time.
@@ -209,12 +206,13 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
.setTargetId(targetId)
.setRegistrarId(gainingRegistrarId)
.setEventTime(newExpirationTime)
.setRenewalPriceBehavior(existingRecurring.getRenewalPriceBehavior())
.setRenewalPrice(existingRecurring.getRenewalPrice().orElse(null))
.setRenewalPriceBehavior(
hasPackageToken
? RenewalPriceBehavior.DEFAULT
: existingRecurring.getRenewalPriceBehavior())
.setRenewalPrice(renewalPrice)
.setRecurrenceEndTime(END_OF_TIME)
.setDomainHistoryId(
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
// Create a new autorenew poll message.
PollMessage.Autorenew gainingClientAutorenewPollMessage =
@@ -224,9 +222,7 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
.setEventTime(newExpirationTime)
.setAutorenewEndTime(END_OF_TIME)
.setMsg("Domain was auto-renewed.")
.setDomainHistoryId(
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
// Construct the post-transfer domain.
Domain partiallyApprovedDomain =
@@ -258,13 +254,17 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
.orElseGet(ImmutableSet::of))
.setLastEppUpdateTime(now)
.setLastEppUpdateRegistrarId(registrarId)
// Even if the existing domain had a package token, that package token should be removed
// on transfer
.setCurrentPackageToken(null)
.build();
Registry registry = Registry.get(existingDomain.getTld());
DomainHistory domainHistory = buildDomainHistory(newDomain, registry, now, gainingRegistrarId);
// Create a poll message for the gaining client.
PollMessage gainingClientPollMessage =
createGainingTransferPollMessage(
targetId, newDomain.getTransferData(), newExpirationTime, now, domainHistoryKey);
targetId, newDomain.getTransferData(), newExpirationTime, now, domainHistoryId);
billingEvent.ifPresent(entitiesToSave::add);
entitiesToSave.add(
autorenewEvent,

View File

@@ -14,7 +14,7 @@
package google.registry.flows.domain;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
import static google.registry.flows.ResourceFlowUtils.verifyHasPendingTransfer;
@@ -32,7 +32,6 @@ import static google.registry.persistence.transaction.TransactionManagerFactory.
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -47,6 +46,7 @@ import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.AuthInfo;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.TransferStatus;
@@ -83,7 +83,9 @@ public final class DomainTransferCancelFlow implements TransactionalFlow {
@Inject @Superuser boolean isSuperuser;
@Inject DomainHistory.Builder historyBuilder;
@Inject EppResponse.Builder responseBuilder;
@Inject DomainTransferCancelFlow() {}
@Inject
DomainTransferCancelFlow() {}
@Override
public EppResponse run() throws EppException {
@@ -100,9 +102,9 @@ public final class DomainTransferCancelFlow implements TransactionalFlow {
}
Registry registry = Registry.get(existingDomain.getTld());
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder
.setId(domainHistoryKey.getId())
.setRevisionId(domainHistoryId.getRevisionId())
.setOtherRegistrarId(existingDomain.getTransferData().getLosingRegistrarId());
Domain newDomain =
@@ -112,12 +114,12 @@ public final class DomainTransferCancelFlow implements TransactionalFlow {
newDomain,
domainHistory,
createLosingTransferPollMessage(
targetId, newDomain.getTransferData(), null, domainHistoryKey));
targetId, newDomain.getTransferData(), null, domainHistoryId));
// Reopen the autorenew event and poll message that we closed for the implicit transfer. This
// may recreate the autorenew poll message if it was deleted when the transfer request was made.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
updateAutorenewRecurrenceEndTime(
existingDomain, existingRecurring, END_OF_TIME, domainHistory.getDomainHistoryId());
existingDomain, existingRecurring, END_OF_TIME, domainHistory.getHistoryEntryId());
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
tm().delete(existingDomain.getTransferData().getServerApproveEntities());

View File

@@ -14,7 +14,7 @@
package google.registry.flows.domain;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
import static google.registry.flows.ResourceFlowUtils.verifyHasPendingTransfer;
@@ -34,7 +34,6 @@ import static google.registry.util.CollectionUtils.union;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.FlowModule.RegistrarId;
@@ -49,6 +48,7 @@ import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.AuthInfo;
import google.registry.model.eppoutput.EppResponse;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.TransferStatus;
@@ -95,9 +95,9 @@ public final class DomainTransferRejectFlow implements TransactionalFlow {
DateTime now = tm().getTransactionTime();
Domain existingDomain = loadAndVerifyExistence(Domain.class, targetId, now);
Registry registry = Registry.get(existingDomain.getTld());
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder
.setId(domainHistoryKey.getId())
.setRevisionId(domainHistoryId.getRevisionId())
.setOtherRegistrarId(existingDomain.getTransferData().getGainingRegistrarId());
verifyOptionalAuthInfo(authInfo, existingDomain);
@@ -113,12 +113,12 @@ public final class DomainTransferRejectFlow implements TransactionalFlow {
newDomain,
domainHistory,
createGainingTransferPollMessage(
targetId, newDomain.getTransferData(), null, now, domainHistoryKey));
targetId, newDomain.getTransferData(), null, now, domainHistoryId));
// Reopen the autorenew event and poll message that we closed for the implicit transfer. This
// may end up recreating the poll message if it was deleted upon the transfer request.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
updateAutorenewRecurrenceEndTime(
existingDomain, existingRecurring, END_OF_TIME, domainHistory.getDomainHistoryId());
existingDomain, existingRecurring, END_OF_TIME, domainHistory.getHistoryEntryId());
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
tm().delete(existingDomain.getTransferData().getServerApproveEntities());

View File

@@ -14,7 +14,7 @@
package google.registry.flows.domain;
import static google.registry.flows.FlowUtils.createHistoryKey;
import static google.registry.flows.FlowUtils.createHistoryEntryId;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.computeExDateForApprovalTime;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -38,7 +38,6 @@ import static google.registry.persistence.transaction.TransactionManagerFactory.
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.batch.AsyncTaskEnqueuer;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
@@ -57,7 +56,6 @@ import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainCommand.Transfer;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.Period;
import google.registry.model.domain.fee.FeeTransferCommandExtension;
import google.registry.model.domain.fee.FeeTransformResponseExtension;
@@ -73,6 +71,7 @@ import google.registry.model.eppoutput.EppResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.DomainTransferData;
@@ -182,6 +181,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
? superuserExtension.get().getRenewalPeriod()
: ((Transfer) resourceCommand).getPeriod();
verifyTransferAllowed(existingDomain, period, now, superuserExtension);
String tld = existingDomain.getTld();
Registry registry = Registry.get(tld);
// An optional extension from the client specifying what they think the transfer should cost.
@@ -194,17 +194,27 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
}
// If the period is zero, then there is no fee for the transfer.
Recurring existingRecurring = tm().loadByKey(existingDomain.getAutorenewBillingEvent());
Optional<FeesAndCredits> feesAndCredits =
(period.getValue() == 0)
? Optional.empty()
: Optional.of(
pricingLogic.getTransferPrice(registry, targetId, now, existingRecurring));
if (feesAndCredits.isPresent()) {
validateFeeChallenge(targetId, now, feeTransfer, feesAndCredits.get());
Optional<FeesAndCredits> feesAndCredits;
if (period.getValue() == 0) {
feesAndCredits = Optional.empty();
} else if (!existingDomain.getCurrentPackageToken().isPresent()) {
feesAndCredits =
Optional.of(pricingLogic.getTransferPrice(registry, targetId, now, existingRecurring));
} else {
// If existing domain is in a package, calculate the transfer price with default renewal price
// behavior
feesAndCredits =
period.getValue() == 0
? Optional.empty()
: Optional.of(pricingLogic.getTransferPrice(registry, targetId, now, null));
}
Key<DomainHistory> domainHistoryKey = createHistoryKey(existingDomain, DomainHistory.class);
if (feesAndCredits.isPresent()) {
validateFeeChallenge(feeTransfer, feesAndCredits.get());
}
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder
.setId(domainHistoryKey.getId())
.setRevisionId(domainHistoryId.getRevisionId())
.setOtherRegistrarId(existingDomain.getCurrentSponsorRegistrarId());
DateTime automaticTransferTime =
superuserExtension
@@ -229,7 +239,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
createTransferServerApproveEntities(
automaticTransferTime,
serverApproveNewExpirationTime,
domainHistoryKey,
domainHistoryId,
existingDomain,
existingRecurring,
trid,
@@ -240,7 +250,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
DomainTransferData pendingTransferData =
createPendingTransferData(
domainAtTransferTime.getRepoId(),
domainHistoryKey.getId(),
domainHistoryId.getRevisionId(),
new DomainTransferData.Builder()
.setTransferRequestTrid(trid)
.setTransferRequestTime(now)
@@ -253,7 +263,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
// Create a poll message to notify the losing registrar that a transfer was requested.
PollMessage requestPollMessage =
createLosingTransferPollMessage(
targetId, pendingTransferData, serverApproveNewExpirationTime, domainHistoryKey)
targetId, pendingTransferData, serverApproveNewExpirationTime, domainHistoryId)
.asBuilder()
.setEventTime(now)
.build();
@@ -262,10 +272,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
// cloneProjectedAtTime() will replace these old autorenew entities with the server approve ones
// that we've created in this flow and stored in pendingTransferData.
updateAutorenewRecurrenceEndTime(
existingDomain,
existingRecurring,
automaticTransferTime,
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()));
existingDomain, existingRecurring, automaticTransferTime, domainHistoryId);
Domain newDomain =
existingDomain
.asBuilder()
@@ -384,7 +391,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
private static ImmutableList<FeeTransformResponseExtension> createResponseExtensions(
Optional<FeesAndCredits> feesAndCredits, Optional<FeeTransferCommandExtension> feeTransfer) {
return (feeTransfer.isPresent() && feesAndCredits.isPresent())
return feeTransfer.isPresent() && feesAndCredits.isPresent()
? ImmutableList.of(
feeTransfer
.get()

View File

@@ -20,20 +20,19 @@ import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.billing.BillingEvent.Recurring;
import google.registry.model.billing.BillingEvent.RenewalPriceBehavior;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.Period;
import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.eppcommon.Trid;
import google.registry.model.poll.PendingActionNotificationResponse.DomainPendingActionNotificationResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.tld.Registry;
import google.registry.model.transfer.DomainTransferData;
import google.registry.model.transfer.TransferData;
@@ -109,7 +108,7 @@ public final class DomainTransferUtils {
public static ImmutableSet<TransferServerApproveEntity> createTransferServerApproveEntities(
DateTime automaticTransferTime,
DateTime serverApproveNewExpirationTime,
Key<DomainHistory> domainHistoryKey,
HistoryEntryId domainHistoryId,
Domain existingDomain,
Recurring existingRecurring,
Trid trid,
@@ -135,38 +134,44 @@ public final class DomainTransferUtils {
builder.add(
createTransferBillingEvent(
automaticTransferTime,
domainHistoryKey,
domainHistoryId,
targetId,
gainingRegistrarId,
registry,
cost)));
createOptionalAutorenewCancellation(
automaticTransferTime, now, domainHistoryKey, targetId, existingDomain, transferCost)
automaticTransferTime, now, domainHistoryId, targetId, existingDomain, transferCost)
.ifPresent(builder::add);
return builder
.add(
createGainingClientAutorenewEvent(
existingRecurring,
existingDomain.getCurrentPackageToken().isPresent()
? existingRecurring
.asBuilder()
.setRenewalPriceBehavior(RenewalPriceBehavior.DEFAULT)
.setRenewalPrice(null)
.build()
: existingRecurring,
serverApproveNewExpirationTime,
domainHistoryKey,
domainHistoryId,
targetId,
gainingRegistrarId))
.add(
createGainingClientAutorenewPollMessage(
serverApproveNewExpirationTime, domainHistoryKey, targetId, gainingRegistrarId))
serverApproveNewExpirationTime, domainHistoryId, targetId, gainingRegistrarId))
.add(
createGainingTransferPollMessage(
targetId,
serverApproveTransferData,
serverApproveNewExpirationTime,
now,
domainHistoryKey))
domainHistoryId))
.add(
createLosingTransferPollMessage(
targetId,
serverApproveTransferData,
serverApproveNewExpirationTime,
domainHistoryKey))
domainHistoryId))
.build();
}
@@ -176,7 +181,7 @@ public final class DomainTransferUtils {
TransferData transferData,
@Nullable DateTime extendedRegistrationExpirationTime,
DateTime now,
Key<DomainHistory> domainHistoryKey) {
HistoryEntryId domainHistoryId) {
return new PollMessage.OneTime.Builder()
.setRegistrarId(transferData.getGainingRegistrarId())
.setEventTime(transferData.getPendingTransferExpirationTime())
@@ -189,8 +194,7 @@ public final class DomainTransferUtils {
transferData.getTransferStatus().isApproved(),
transferData.getTransferRequestTrid(),
now)))
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
}
@@ -199,7 +203,7 @@ public final class DomainTransferUtils {
String targetId,
TransferData transferData,
@Nullable DateTime extendedRegistrationExpirationTime,
Key<DomainHistory> domainHistoryKey) {
HistoryEntryId domainHistoryId) {
return new PollMessage.OneTime.Builder()
.setRegistrarId(transferData.getLosingRegistrarId())
.setEventTime(transferData.getPendingTransferExpirationTime())
@@ -207,8 +211,7 @@ public final class DomainTransferUtils {
.setResponseData(
ImmutableList.of(
createTransferResponse(targetId, transferData, extendedRegistrationExpirationTime)))
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
}
@@ -218,7 +221,7 @@ public final class DomainTransferUtils {
TransferData transferData,
@Nullable DateTime extendedRegistrationExpirationTime) {
return new DomainTransferResponse.Builder()
.setFullyQualifiedDomainName(targetId)
.setDomainName(targetId)
.setGainingRegistrarId(transferData.getGainingRegistrarId())
.setLosingRegistrarId(transferData.getLosingRegistrarId())
.setPendingTransferExpirationTime(transferData.getPendingTransferExpirationTime())
@@ -230,7 +233,7 @@ public final class DomainTransferUtils {
private static PollMessage.Autorenew createGainingClientAutorenewPollMessage(
DateTime serverApproveNewExpirationTime,
Key<DomainHistory> domainHistoryKey,
HistoryEntryId domainHistoryId,
String targetId,
String gainingRegistrarId) {
return new PollMessage.Autorenew.Builder()
@@ -239,15 +242,14 @@ public final class DomainTransferUtils {
.setEventTime(serverApproveNewExpirationTime)
.setAutorenewEndTime(END_OF_TIME)
.setMsg("Domain was auto-renewed.")
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
}
private static BillingEvent.Recurring createGainingClientAutorenewEvent(
Recurring existingRecurring,
DateTime serverApproveNewExpirationTime,
Key<DomainHistory> domainHistoryKey,
HistoryEntryId domainHistoryId,
String targetId,
String gainingRegistrarId) {
return new BillingEvent.Recurring.Builder()
@@ -259,8 +261,7 @@ public final class DomainTransferUtils {
.setRecurrenceEndTime(END_OF_TIME)
.setRenewalPriceBehavior(existingRecurring.getRenewalPriceBehavior())
.setRenewalPrice(existingRecurring.getRenewalPrice().orElse(null))
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
}
@@ -274,7 +275,7 @@ public final class DomainTransferUtils {
* renewal, we must issue a cancellation for the autorenew, so that the losing registrar will not
* be charged (essentially, the gaining registrar takes on the cost of the year of registration
* that the autorenew just added). But, if the superuser extension is used to request a transfer
* without an additional year then the gaining registrar is not charged for the one year renewal
* without an additional year then the gaining registrar is not charged for the one-year renewal
* and the losing registrar still needs to be charged for the auto-renew.
*
* <p>For details on the policy justification, see b/19430703#comment17 and <a
@@ -283,7 +284,7 @@ public final class DomainTransferUtils {
private static Optional<BillingEvent.Cancellation> createOptionalAutorenewCancellation(
DateTime automaticTransferTime,
DateTime now,
Key<DomainHistory> domainHistoryKey,
HistoryEntryId domainHistoryId,
String targetId,
Domain existingDomain,
Optional<Money> transferCost) {
@@ -294,11 +295,7 @@ public final class DomainTransferUtils {
if (autorenewGracePeriod != null && transferCost.isPresent()) {
return Optional.of(
BillingEvent.Cancellation.forGracePeriod(
autorenewGracePeriod,
now,
new DomainHistoryId(
domainHistoryKey.getParent().getName(), domainHistoryKey.getId()),
targetId)
autorenewGracePeriod, now, domainHistoryId, targetId)
.asBuilder()
.setEventTime(automaticTransferTime)
.build());
@@ -308,7 +305,7 @@ public final class DomainTransferUtils {
private static BillingEvent.OneTime createTransferBillingEvent(
DateTime automaticTransferTime,
Key<DomainHistory> domainHistoryKey,
HistoryEntryId domainHistoryId,
String targetId,
String gainingRegistrarId,
Registry registry,
@@ -321,8 +318,7 @@ public final class DomainTransferUtils {
.setPeriodYears(1)
.setEventTime(automaticTransferTime)
.setBillingTime(automaticTransferTime.plus(registry.getTransferGracePeriodLength()))
.setDomainHistoryId(
new DomainHistoryId(domainHistoryKey.getParent().getName(), domainHistoryKey.getId()))
.setDomainHistoryId(domainHistoryId)
.build();
}

View File

@@ -74,7 +74,7 @@ import google.registry.model.domain.DomainCommand.Update.Change;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.fee.FeeUpdateCommandExtension;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.domain.secdns.DelegationSignerData;
import google.registry.model.domain.secdns.DomainDsData;
import google.registry.model.domain.secdns.SecDnsUpdateExtension;
import google.registry.model.domain.superuser.DomainUpdateSuperuserExtension;
import google.registry.model.eppcommon.AuthInfo;
@@ -87,6 +87,7 @@ import google.registry.model.poll.PendingActionNotificationResponse.DomainPendin
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.model.tld.Registry;
import java.util.Objects;
import java.util.Optional;
import javax.inject.Inject;
import org.joda.time.DateTime;
@@ -181,7 +182,9 @@ public final class DomainUpdateFlow implements TransactionalFlow {
DomainHistory domainHistory =
historyBuilder.setType(DOMAIN_UPDATE).setDomain(newDomain).build();
validateNewState(newDomain);
dnsQueue.addDomainRefreshTask(targetId);
if (requiresDnsUpdate(existingDomain, newDomain)) {
dnsQueue.addDomainRefreshTask(targetId);
}
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
entitiesToSave.add(newDomain, domainHistory);
Optional<BillingEvent.OneTime> statusUpdateBillingEvent =
@@ -203,6 +206,13 @@ public final class DomainUpdateFlow implements TransactionalFlow {
return responseBuilder.build();
}
/** Determines if any of the changes to new domain should trigger DNS update. */
private boolean requiresDnsUpdate(Domain existingDomain, Domain newDomain) {
return existingDomain.shouldPublishToDns() != newDomain.shouldPublishToDns()
|| !Objects.equals(newDomain.getDsData(), existingDomain.getDsData())
|| !Objects.equals(newDomain.getNsHosts(), existingDomain.getNsHosts());
}
/** Fail if the object doesn't exist or was deleted. */
private void verifyUpdateAllowed(Update command, Domain existingDomain, DateTime now)
throws EppException {
@@ -229,8 +239,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
validateContactsHaveTypes(add.getContacts());
validateContactsHaveTypes(remove.getContacts());
validateRegistrantAllowedOnTld(tld, command.getInnerChange().getRegistrantContactId());
validateNameserversAllowedOnTld(
tld, add.getNameserverFullyQualifiedHostNames());
validateNameserversAllowedOnTld(tld, add.getNameserverHostNames());
}
private Domain performUpdate(Update command, Domain domain, DateTime now) throws EppException {
@@ -260,7 +269,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
secDnsUpdate.isPresent()
? updateDsData(
domain.getDsData().stream()
.map(DelegationSignerData::cloneWithoutDomainRepoId)
.map(DomainDsData::cloneWithoutDomainRepoId)
.collect(toImmutableSet()),
secDnsUpdate.get())
: domain.getDsData())
@@ -268,12 +277,18 @@ public final class DomainUpdateFlow implements TransactionalFlow {
.setLastEppUpdateRegistrarId(registrarId)
.addStatusValues(add.getStatusValues())
.removeStatusValues(remove.getStatusValues())
.addNameservers(add.getNameservers().stream().collect(toImmutableSet()))
.removeNameservers(remove.getNameservers().stream().collect(toImmutableSet()))
.removeContacts(remove.getContacts())
.addContacts(add.getContacts())
.setRegistrant(firstNonNull(change.getRegistrant(), domain.getRegistrant()))
.setAuthInfo(firstNonNull(change.getAuthInfo(), domain.getAuthInfo()));
if (!add.getNameservers().isEmpty()) {
domainBuilder.addNameservers(add.getNameservers().stream().collect(toImmutableSet()));
}
if (!remove.getNameservers().isEmpty()) {
domainBuilder.removeNameservers(remove.getNameservers().stream().collect(toImmutableSet()));
}
Optional<DomainUpdateSuperuserExtension> superuserExt =
eppInput.getSingleExtension(DomainUpdateSuperuserExtension.class);
if (superuserExt.isPresent()) {
@@ -345,7 +360,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
.map(StatusValue::getXmlName)
.collect(toImmutableSortedSet(Ordering.natural()));
String msg = "";
String msg;
if (addedServerStatuses.size() > 0 && removedServerStatuses.size() > 0) {
msg =
String.format(

View File

@@ -15,7 +15,6 @@
package google.registry.flows.domain.token;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.base.Strings;
@@ -36,7 +35,7 @@ import google.registry.model.domain.token.AllocationToken.TokenBehavior;
import google.registry.model.domain.token.AllocationToken.TokenStatus;
import google.registry.model.domain.token.AllocationToken.TokenType;
import google.registry.model.domain.token.AllocationTokenExtension;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.model.tld.Registry;
import google.registry.persistence.VKey;
import java.util.List;
@@ -95,12 +94,11 @@ public class AllocationTokenFlowUtils {
}
/** Redeems a SINGLE_USE {@link AllocationToken}, returning the redeemed copy. */
public AllocationToken redeemToken(
AllocationToken token, VKey<? extends HistoryEntry> redemptionHistoryEntry) {
public AllocationToken redeemToken(AllocationToken token, HistoryEntryId redemptionHistoryId) {
checkArgument(
TokenType.SINGLE_USE.equals(token.getTokenType()),
"Only SINGLE_USE tokens can be marked as redeemed");
return token.asBuilder().setRedemptionHistoryEntry(redemptionHistoryEntry).build();
return token.asBuilder().setRedemptionHistoryId(redemptionHistoryId).build();
}
/**
@@ -110,7 +108,7 @@ public class AllocationTokenFlowUtils {
*
* @throws EppException if the token is invalid in any way
*/
private void validateToken(
private static void validateToken(
InternetDomainName domainName, AllocationToken token, String registrarId, DateTime now)
throws EppException {
@@ -138,7 +136,7 @@ public class AllocationTokenFlowUtils {
}
/** Loads a given token and validates that it is not redeemed */
private AllocationToken loadToken(String token) throws EppException {
private static AllocationToken loadToken(String token) throws EppException {
if (Strings.isNullOrEmpty(token)) {
// We load the token directly from the input XML. If it's null or empty we should throw
// an InvalidAllocationTokenException before the database load attempt fails.
@@ -152,7 +150,7 @@ public class AllocationTokenFlowUtils {
}
maybeTokenEntity =
tm().transact(() -> tm().loadByKeyIfPresent(VKey.createSql(AllocationToken.class, token)));
tm().transact(() -> tm().loadByKeyIfPresent(VKey.create(AllocationToken.class, token)));
if (!maybeTokenEntity.isPresent()) {
throw new InvalidAllocationTokenException();
@@ -175,11 +173,7 @@ public class AllocationTokenFlowUtils {
return Optional.empty();
}
AllocationToken tokenEntity = loadToken(extension.get().getAllocationToken());
validateToken(
InternetDomainName.from(command.getFullyQualifiedDomainName()),
tokenEntity,
registrarId,
now);
validateToken(InternetDomainName.from(command.getDomainName()), tokenEntity, registrarId, now);
return Optional.of(
tokenCustomLogic.validateToken(command, tokenEntity, registry, registrarId, now));
}
@@ -234,8 +228,8 @@ public class AllocationTokenFlowUtils {
// the Recurring billing event is reloaded later in the renew flow, so we synchronize changed
// RecurringBillingEvent with storage manually
tm().put(newRecurringBillingEvent);
jpaTm().getEntityManager().flush();
jpaTm().getEntityManager().clear();
tm().getEntityManager().flush();
tm().getEntityManager().clear();
// Remove current package token
return domain
@@ -293,11 +287,11 @@ public class AllocationTokenFlowUtils {
}
}
/** The __REMOVEPACKAGE__ token is missing on a renew package domain command */
/** The __REMOVEPACKAGE__ token is missing on a package domain command */
public static class MissingRemovePackageTokenOnPackageDomainException
extends AssociationProhibitsOperationException {
MissingRemovePackageTokenOnPackageDomainException() {
super("Domains that are inside packages cannot be explicitly renewed");
super("Domains that are inside packages cannot be explicitly renewed or transferred");
}
}

View File

@@ -27,7 +27,6 @@ import static google.registry.persistence.transaction.TransactionManagerFactory.
import static google.registry.util.CollectionUtils.isNullOrEmpty;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.config.RegistryConfig.Config;
import google.registry.dns.DnsQueue;
import google.registry.flows.EppException;
@@ -49,7 +48,6 @@ import google.registry.model.eppoutput.EppResponse;
import google.registry.model.host.Host;
import google.registry.model.host.HostCommand.Create;
import google.registry.model.host.HostHistory;
import google.registry.model.index.EppResourceIndex;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import java.util.Optional;
import javax.inject.Inject;
@@ -106,7 +104,7 @@ public final class HostCreateFlow implements TransactionalFlow {
DateTime now = tm().getTransactionTime();
verifyResourceDoesNotExist(Host.class, targetId, now, registrarId);
// The superordinate domain of the host object if creating an in-bailiwick host, or null if
// creating an external host. This is looked up before we actually create the Host object so
// creating an external host. This is looked up before we actually create the Host object, so
// we can detect error conditions earlier.
Optional<Domain> superordinateDomain =
lookupSuperordinateDomain(validateHostName(targetId), now);
@@ -130,17 +128,13 @@ public final class HostCreateFlow implements TransactionalFlow {
.setSuperordinateDomain(superordinateDomain.map(Domain::createVKey).orElse(null))
.build();
historyBuilder.setType(HOST_CREATE).setHost(newHost);
ImmutableSet<ImmutableObject> entitiesToSave =
ImmutableSet.of(
newHost,
historyBuilder.build(),
EppResourceIndex.create(Key.create(newHost)));
ImmutableSet<ImmutableObject> entitiesToSave = ImmutableSet.of(newHost, historyBuilder.build());
if (superordinateDomain.isPresent()) {
tm().update(
superordinateDomain
.get()
.asBuilder()
.addSubordinateHost(command.getFullyQualifiedHostName())
.addSubordinateHost(command.getHostName())
.build());
// Only update DNS if this is a subordinate host. External hosts have no glue to write, so
// they are only written as NS records from the referencing domain.
@@ -152,14 +146,14 @@ public final class HostCreateFlow implements TransactionalFlow {
/** Subordinate hosts must have an ip address. */
static class SubordinateHostMustHaveIpException extends RequiredParameterMissingException {
public SubordinateHostMustHaveIpException() {
SubordinateHostMustHaveIpException() {
super("Subordinate hosts must have an ip address");
}
}
/** External hosts must not have ip addresses. */
static class UnexpectedExternalHostIpException extends ParameterValueRangeErrorException {
public UnexpectedExternalHostIpException() {
UnexpectedExternalHostIpException() {
super("External hosts must not have ip addresses");
}
}

View File

@@ -24,7 +24,6 @@ import static google.registry.model.eppoutput.Result.Code.SUCCESS;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import google.registry.batch.AsyncTaskEnqueuer;
import google.registry.dns.DnsQueue;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
@@ -79,7 +78,6 @@ public final class HostDeleteFlow implements TransactionalFlow {
@Inject Trid trid;
@Inject @Superuser boolean isSuperuser;
@Inject HostHistory.Builder historyBuilder;
@Inject AsyncTaskEnqueuer asyncTaskEnqueuer;
@Inject EppResponse.Builder responseBuilder;
@Inject

View File

@@ -80,26 +80,26 @@ public final class HostInfoFlow implements Flow {
tm().transact(
() -> tm().loadByKey(host.getSuperordinateDomain()).cloneProjectedAtTime(now));
hostInfoDataBuilder
.setCurrentSponsorClientId(superordinateDomain.getCurrentSponsorRegistrarId())
.setCurrentSponsorRegistrarId(superordinateDomain.getCurrentSponsorRegistrarId())
.setLastTransferTime(host.computeLastTransferTime(superordinateDomain));
if (superordinateDomain.getStatusValues().contains(StatusValue.PENDING_TRANSFER)) {
statusValues.add(StatusValue.PENDING_TRANSFER);
}
} else {
hostInfoDataBuilder
.setCurrentSponsorClientId(host.getPersistedCurrentSponsorRegistrarId())
.setCurrentSponsorRegistrarId(host.getPersistedCurrentSponsorRegistrarId())
.setLastTransferTime(host.getLastTransferTime());
}
return responseBuilder
.setResData(
hostInfoDataBuilder
.setFullyQualifiedHostName(host.getHostName())
.setHostName(host.getHostName())
.setRepoId(host.getRepoId())
.setStatusValues(statusValues.build())
.setInetAddresses(host.getInetAddresses())
.setCreationClientId(host.getCreationRegistrarId())
.setCreationRegistrarId(host.getCreationRegistrarId())
.setCreationTime(host.getCreationTime())
.setLastEppUpdateClientId(host.getLastEppUpdateRegistrarId())
.setLastEppUpdateRegistrarId(host.getLastEppUpdateRegistrarId())
.setLastEppUpdateTime(host.getLastEppUpdateTime())
.build())
.build();

View File

@@ -16,6 +16,8 @@ package google.registry.flows.host;
import static com.google.common.base.MoreObjects.firstNonNull;
import static com.google.common.collect.Sets.union;
import static google.registry.dns.RefreshDnsOnHostRenameAction.PARAM_HOST_KEY;
import static google.registry.dns.RefreshDnsOnHostRenameAction.QUEUE_HOST_RENAME;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.checkSameValuesNotAddedAndRemoved;
import static google.registry.flows.ResourceFlowUtils.loadAndVerifyExistence;
@@ -26,14 +28,16 @@ import static google.registry.flows.host.HostFlowUtils.lookupSuperordinateDomain
import static google.registry.flows.host.HostFlowUtils.validateHostName;
import static google.registry.flows.host.HostFlowUtils.verifySuperordinateDomainNotInPendingDelete;
import static google.registry.flows.host.HostFlowUtils.verifySuperordinateDomainOwnership;
import static google.registry.model.index.ForeignKeyIndex.loadAndGetKey;
import static google.registry.model.reporting.HistoryEntry.Type.HOST_UPDATE;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.isNullOrEmpty;
import com.google.cloud.tasks.v2.Task;
import com.google.common.collect.ImmutableMultimap;
import com.google.common.collect.ImmutableSet;
import google.registry.batch.AsyncTaskEnqueuer;
import google.registry.dns.DnsQueue;
import google.registry.dns.RefreshDnsOnHostRenameAction;
import google.registry.flows.EppException;
import google.registry.flows.EppException.ObjectAlreadyExistsException;
import google.registry.flows.EppException.ParameterValueRangeErrorException;
@@ -46,6 +50,7 @@ import google.registry.flows.TransactionalFlow;
import google.registry.flows.annotations.ReportingSpec;
import google.registry.flows.exceptions.ResourceHasClientUpdateProhibitedException;
import google.registry.model.EppResource;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.ImmutableObject;
import google.registry.model.domain.Domain;
import google.registry.model.domain.metadata.MetadataExtension;
@@ -59,6 +64,8 @@ import google.registry.model.host.HostCommand.Update.Change;
import google.registry.model.host.HostHistory;
import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
import google.registry.persistence.VKey;
import google.registry.request.Action.Service;
import google.registry.util.CloudTasksUtils;
import java.util.Objects;
import java.util.Optional;
import javax.inject.Inject;
@@ -107,9 +114,8 @@ public final class HostUpdateFlow implements TransactionalFlow {
* requires special checking, since you must be able to clear the status off the object with an
* update.
*/
private static final ImmutableSet<StatusValue> DISALLOWED_STATUSES = ImmutableSet.of(
StatusValue.PENDING_DELETE,
StatusValue.SERVER_UPDATE_PROHIBITED);
private static final ImmutableSet<StatusValue> DISALLOWED_STATUSES =
ImmutableSet.of(StatusValue.PENDING_DELETE, StatusValue.SERVER_UPDATE_PROHIBITED);
@Inject ResourceCommand resourceCommand;
@Inject ExtensionManager extensionManager;
@@ -120,7 +126,10 @@ public final class HostUpdateFlow implements TransactionalFlow {
@Inject AsyncTaskEnqueuer asyncTaskEnqueuer;
@Inject DnsQueue dnsQueue;
@Inject EppResponse.Builder responseBuilder;
@Inject HostUpdateFlow() {}
@Inject CloudTasksUtils cloudTasksUtils;
@Inject
HostUpdateFlow() {}
@Override
public EppResponse run() throws EppException {
@@ -129,7 +138,7 @@ public final class HostUpdateFlow implements TransactionalFlow {
extensionManager.validate();
Update command = (Update) resourceCommand;
Change change = command.getInnerChange();
String suppliedNewHostName = change.getFullyQualifiedHostName();
String suppliedNewHostName = change.getHostName();
DateTime now = tm().getTransactionTime();
validateHostName(targetId);
Host existingHost = loadAndVerifyExistence(Host.class, targetId, now);
@@ -147,7 +156,7 @@ public final class HostUpdateFlow implements TransactionalFlow {
EppResource owningResource = firstNonNull(oldSuperordinateDomain, existingHost);
verifyUpdateAllowed(
command, existingHost, newSuperordinateDomain.orElse(null), owningResource, isHostRename);
if (isHostRename && loadAndGetKey(Host.class, newHostName, now) != null) {
if (isHostRename && ForeignKeyUtils.load(Host.class, newHostName, now) != null) {
throw new HostAlreadyExistsException(newHostName);
}
AddRemove add = command.getInnerAdd();
@@ -260,7 +269,7 @@ public final class HostUpdateFlow implements TransactionalFlow {
dnsQueue.addHostRefreshTask(existingHost.getHostName());
}
// In case of a rename, there are many updates we need to queue up.
if (((Update) resourceCommand).getInnerChange().getFullyQualifiedHostName() != null) {
if (((Update) resourceCommand).getInnerChange().getHostName() != null) {
// If the renamed host is also subordinate, then we must enqueue an update to write the new
// glue.
if (newHost.isSubordinate()) {
@@ -268,7 +277,12 @@ public final class HostUpdateFlow implements TransactionalFlow {
}
// We must also enqueue updates for all domains that use this host as their nameserver so
// that their NS records can be updated to point at the new name.
asyncTaskEnqueuer.enqueueAsyncDnsRefresh(existingHost, tm().getTransactionTime());
Task task =
cloudTasksUtils.createPostTask(
RefreshDnsOnHostRenameAction.PATH,
Service.BACKEND.toString(),
ImmutableMultimap.of(PARAM_HOST_KEY, existingHost.createVKey().stringify()));
cloudTasksUtils.enqueue(QUEUE_HOST_RENAME, task);
}
}

View File

@@ -108,7 +108,7 @@ public final class PollAckFlow implements TransactionalFlow {
// acked, then we return a special status code indicating that. Note that the query will
// include the message being acked.
int messageCount = tm().doTransactionless(() -> getPollMessageCount(registrarId, now));
int messageCount = tm().transact(() -> getPollMessageCount(registrarId, now));
if (messageCount <= 0) {
return responseBuilder.setResultFromCode(SUCCESS_WITH_NO_MESSAGES).build();
}

View File

@@ -17,7 +17,7 @@ package google.registry.groups;
import com.google.api.services.admin.directory.Directory;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule.DelegatedCredential;
import google.registry.config.CredentialModule.AdcDelegatedCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.util.GoogleCredentialsBundle;
@@ -27,7 +27,7 @@ public final class DirectoryModule {
@Provides
static Directory provideDirectory(
@DelegatedCredential GoogleCredentialsBundle credentialsBundle,
@AdcDelegatedCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Directory.Builder(
credentialsBundle.getHttpTransport(),

View File

@@ -17,7 +17,7 @@ package google.registry.groups;
import com.google.api.services.groupssettings.Groupssettings;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule.DelegatedCredential;
import google.registry.config.CredentialModule.AdcDelegatedCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.util.GoogleCredentialsBundle;
@@ -27,7 +27,7 @@ public final class GroupssettingsModule {
@Provides
static Groupssettings provideDirectory(
@DelegatedCredential GoogleCredentialsBundle credentialsBundle,
@AdcDelegatedCredential GoogleCredentialsBundle credentialsBundle,
@Config("projectId") String projectId) {
return new Groupssettings.Builder(
credentialsBundle.getHttpTransport(),

View File

@@ -120,10 +120,4 @@ public final class KeyModule {
static String provideSafeBrowsingAPIKey(Keyring keyring) {
return keyring.getSafeBrowsingAPIKey();
}
@Provides
@Key("jsonCredential")
static String provideJsonCredential(Keyring keyring) {
return keyring.getJsonCredential();
}
}

View File

@@ -1,41 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.keyring.kms;
import com.google.auto.value.AutoValue;
import com.google.common.annotations.VisibleForTesting;
/**
* A value type class containing a Cloud KMS encrypted and encoded ciphertext, and the name of the
* CryptoKeyVersion used to encrypt it.
*/
@AutoValue
abstract class EncryptResponse {
static EncryptResponse create(
com.google.api.services.cloudkms.v1.model.EncryptResponse cloudKmsEncryptResponse) {
return new AutoValue_EncryptResponse(
cloudKmsEncryptResponse.getCiphertext(), cloudKmsEncryptResponse.getName());
}
@VisibleForTesting
static EncryptResponse create(String ciphertext, String cryptoKeyVersionName) {
return new AutoValue_EncryptResponse(ciphertext, cryptoKeyVersionName);
}
abstract String ciphertext();
abstract String cryptoKeyVersionName();
}

View File

@@ -1,48 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.keyring.kms;
import google.registry.keyring.api.KeyringException;
/** An abstraction to simplify Cloud KMS operations. */
public interface KmsConnection {
/**
* The maximum allowable secret size, as set by Cloud KMS.
*
* @see <a
* href="https://cloud.google.com/kms/docs/reference/rest/v1/projects.locations.keyRings.cryptoKeys/encrypt#request-body">projects.locations.keyRings.cryptoKeys.encrypt</a>
*/
int MAX_SECRET_SIZE_BYTES = 64 * 1024;
/**
* Encrypts a plaintext with CryptoKey {@code cryptoKeyName} on KeyRing {@code keyRingName}.
*
* <p>The latest CryptoKeyVersion is used to encrypt the value. The value must not be larger than
* {@code MAX_SECRET_SIZE_BYTES}.
*
* <p>If no applicable CryptoKey or CryptoKeyVersion exist, they will be created.
*
* @throws KeyringException on encryption failure.
*/
EncryptResponse encrypt(String cryptoKeyName, byte[] plaintext);
/**
* Decrypts a Cloud KMS encrypted and encoded value with CryptoKey {@code cryptoKeyName}.
*
* @throws KeyringException on decryption failure.
*/
byte[] decrypt(String cryptoKeyName, String encodedCiphertext);
}

Some files were not shown because too many files have changed in this diff Show More