1
0
mirror of https://github.com/google/nomulus synced 2026-05-20 23:01:53 +00:00

Compare commits

...

79 Commits

Author SHA1 Message Date
gbrodman
886cdfa39b Update NPM dependency based on Github security warning (#941) 2021-01-25 23:04:30 -05:00
sarahcaseybot
beefa9364b Use CertificateChecker on login (#936)
* Use CertificateChecker on login

* Add actual enforcement of requirements in sandbox

* Add new Exceptions

* add validation command to RegistryToolComponent

* Fix error messages

* Add a test for production behavior

* check logs in test

* move loghandler
2021-01-22 16:32:15 -05:00
gbrodman
73210e4b09 Convert (most) HistoryEntry ofy calls to tm (#933)
* Convert (most) HistoryEntry ofy calls to tm

As part of this change, it was necessary to do changes in the JPATM that
are similar (but the opposite) of the changes that we did in
DatastoreTM with regards to converting HistoryEntries to and from the
*History classes.

We leave the ofy() calls in the MapReduce ResaveAllHistoryEntriesAction
for now; that can be converted during the Beam pipeline transition.

Some other tests required registrar-name fixes as well -- because
*History objects have a foreign key on the Registrar table, we have to
use a "real" registrar name in tests.

* Add simple HistoryEntryDaoTest
2021-01-22 14:43:34 -05:00
Ben McIlwain
08cec96a93 Correct containsMatch() -> contains() for non-regexes (#940)
* Correct containsMatch() -> contains() for non-regexes
2021-01-22 14:31:38 -05:00
Ben McIlwain
31ef402c50 Require an override flag to allow updating pending delete domains (#939)
* Require an override flag to allow updating pending delete domains

Needing to update pending delete domains is an uncommon situation, yet currently
we are allowing superusers to do so without any extra validation (which has led
to errors). This adds a new override flag to gate the update of pending delete
domains; without it, the update will fail.
2021-01-22 14:31:13 -05:00
Michael Muller
e89cc4406a Fix another "extra parens" warning (#938)
* Fix another "extra parens" warning

Same place as the last one, but I missed it :-(
2021-01-22 13:39:30 -05:00
Shicong Huang
48de5d8375 Convert ofy() to tm() for all contact transfer flows (#937)
* Convert ofy() to tm() for all contact transfer flows

* Resolve comments
2021-01-22 09:38:51 -05:00
Ben McIlwain
59abc1d154 Put else if on same line to fix build style warning (#935)
* Put else if on same line to fix build style warning
2021-01-21 10:50:29 -05:00
Shicong Huang
6794c6fbd7 Resolve remaining TODO(shicong) (#932) 2021-01-20 19:27:48 -05:00
Ben McIlwain
0c384adc22 Change java.util.Optional.isEmpty() to !isPresent() (#934)
isEmpty() is not available in the version of Java GAE uses and is throwing
runtime errors (!!). I think these got into our codebases because people don't
have the language version set correctly in IntelliJ; they show as outright
errors for me (I'm on language level 8).
2021-01-20 09:38:52 -05:00
sarahcaseybot
3b679058b0 Validate Certificate on Login (#919)
* Check certificate matches saved one on login

* Add tests

* refactoring

* fix warning messages
2021-01-19 17:06:26 -05:00
Shicong Huang
9b5805f145 Resolve TODOs under Shicong's name (#930) 2021-01-15 14:22:31 -05:00
Michael Muller
9e6f99face Add object comparison to replay tests (#925)
* Add object comparison to replay tests

Allow optional object comparison in the replay test extension and enable it
for the DomainCreateFlow test.

To faciliate this, add two new field annotations to ImmutableObject:
DoNotCompare, to be used for fields that are not relevant to either database,
and Insignificant, to be used for fields that are mutated after they have been
accessed and therefore violate immutability (there is currently only one of
these, however we might discover more in the course of adding more comparisons
to the replay test.

* Revert commented out premium price error log

* Added static create methods for ReplayExtension
2021-01-15 14:20:55 -05:00
Shicong Huang
554e675303 Add Sara and Legina to CONTRIBUTORS (#931)
* Add Sara and Legina to CONTRIBUTORS

* Add a new line at the end of the file
2021-01-14 20:16:55 -05:00
Michael Muller
3d33c81475 Remove some unnecessary parens (#929)
* Remove some unnecessary parens

Remove extra parens that cause a compile warning.

* Removed the correct paren this time.
2021-01-14 17:09:27 -05:00
Shicong Huang
56e384aa4f Restore symmetric VKey referenced by BillingEvent.Cancellation (#928) 2021-01-14 13:16:24 -05:00
Shicong Huang
f669e3ca59 Convert remaining Contact flows to tm() (#924)
* Convert remaining Contact flows to tm()

* Add a test to verify street fileds get populated from XML
2021-01-13 13:50:23 -05:00
Shicong Huang
c45129f9ac Convert some Host and Contact flows to tm() (#921)
* Convert HostDeleteFlow to tm()

* Add query for contact linked domain
2021-01-13 10:56:53 -05:00
Weimin Yu
84d2b82050 Update the Datastore to SQL migration pipeline (#927)
* Update the Datastore to SQL migration pipeline

The pipeline now includes all entity types to be migrated by it, and has
completed successfully using the Sandbox data set. The running time in Sandbox
is about 3 hours, extrapolating by entity count to a 12-hour run with
production data. However, actual running time is likely to be longer since
throughput is lower with domains, which accounts for a higher percentage
of the total in production. More optimization will be needed.

The migrated data has not been validated.
2021-01-12 18:05:46 -05:00
Shicong Huang
0109d5e473 Convert HostUpdateFlow to tm() (#923) 2021-01-08 10:28:22 -05:00
Ben McIlwain
9e03ae453c Use better null-handling around registrar certificates (#922)
* Use better null-handling around registrar certificates

Now with Optional it's always very clear whether they do or do not have values.
isNullOrEmpty() shouldn't be necessary anymore (indeed it wasn't necessary prior
to this either, as the relevant setters in the Registrar builder already coerced
empty strings to null). And also the cert hash is a required HTTP header, so it
will error out in the Dagger component if null or empty long before getting to
any other code.

* Merge branch 'master' into optional-get-certs
2021-01-07 19:30:09 -05:00
Weimin Yu
7a62aa0602 Allow BEAM pipeline to choose JDBC isolation levels (#916)
* Allow BEAM pipeline to choose JDBC isolation levels

Some BEAM pipelines may only perform READ-ONLY (e.g., reporting) or
blind-write (datastore to sql data migration) operations, which do not
need the default TRANSACTION_SERIALIZABLE isolation level. In such
cases, a less strict level allows better performance.
2021-01-07 11:00:36 -05:00
Weimin Yu
6a1e86ff33 Add a TODO to a non-functioning JPA annotation (#917)
* Add a TODO to a non-functioning JPA annotation
2021-01-06 13:28:53 -05:00
gbrodman
5bf618e671 Refactor naming and behavior of bulk load methods in TransactionManager (#918)
* Refactor naming and behavior of bulk load methods in TransactionManager

The contract of loadByKeys(Iterable<VKey>) specifies that the method will
throw a NoSuchElementException if any of the specified keys don't exist.
We don't do that before this PR, but now we do.

Existing calls (when necessary) were converted to the new load*
methods, which have the same behavior as the previous methods.

Existing methods were also renamed to be more clear -- see b/176239831
for more details and discussion.
2021-01-06 11:55:59 -05:00
Weimin Yu
b4676a9836 Remove unnecessary method (#920)
* Remove unnecessary method

The 'id' property no longer exists in the entity hierarchy
2021-01-06 11:18:37 -05:00
Ben McIlwain
ef9f3aeada Remove a couple unused variables (#913)
* Remove a couple unused variables
2020-12-23 17:19:02 -05:00
Shicong Huang
9c43aab8cd Convert HostCreateFlow and HostCheckFlow to tm() (#910) 2020-12-22 21:02:02 -05:00
Weimin Yu
cb63c3dd80 Add unique constraints on domain_hosts (#911)
* Add unique constraints on domain_hosts

Add unique constraints on DomainHost (child of DomainBase) and
DomainHistoryHost (child of DomainHistory). DomainHost is non-entity
embedded object and Hibernate does not define indexes automatically.

This should improve read and write performance of the parent entities.
2020-12-21 18:22:24 -05:00
Ben McIlwain
2cf190e448 Add a fast mode to the ResaveAllEppResourcesAction mapreduce (#912)
* Add a fast mode to the ResaveAllEppResourcesAction mapreduce

This new mode avoids writing no-op mutations for entities that don't actually
have any changes to write. The cronjobs use fast mode by default, but manual
invocations do not, as manual invocations are often used to trigger @OnLoad
migrations, and fast mode won't pick up on those changes.
2020-12-21 18:07:59 -05:00
gbrodman
e550c94cbc Convert AllocationToken-related classes to tm() (#909)
* Convert AllocationToken-related classes to tm()

For the most part this is a fairly simple converstion -- changing Key
references to VKey references, using JPA transactions when necessary,
and using the TransactionManager interface. There's a bit of cleanup too
in related code
2020-12-21 15:56:49 -05:00
Shicong Huang
6e2bbd1a7e Add BillingVKey to restore symmetric VKey in GracePeriodBase (#902)
* Use PollMessageVKey to replace VKey<PollMessage> in DomainBase

* Revert changes to DomainContent

* Use BillingVKey in GracePeriodBase to restore symmetric vkey

* Rebase on HEAD
2020-12-17 14:13:47 -05:00
Weimin Yu
495d7176d8 Validate SQL credentials in Secret Manager (#907)
* Validate SQL credentials in Secret Manager

Load SQL credentials from the SecretManager and compare them with the
ones currently in use in Nomulus server, beam pipeline, and the registry
tool. Normal operations are not affected by failures related to the
SecretManager, be it IOException, insufficient permission , or wrong or
missing credential.

The appengine and compute engine default service accounts must be
granted the permission to access the secret data. In the short term, we
will grant the secretmanager.secretAccessor role to these accounts. In
the long term, with the proposed privilege service, access will be granted
on per-secret basis.
2020-12-16 10:57:03 -05:00
Michael Muller
d7aab524e5 Make config/presubmits.py use explicit encodings (#908)
For some reason, our docker build image has started using a non-utf8 default
encoding.  Specify the encoding explicitly on python "open()" to override.

Note that this might not entirely fix the build: it's possible that this
problem may affect other portions of the build.
2020-12-16 10:03:32 -05:00
sarahcaseybot
c5bfe31b73 Modify SignedMarkRevocationList to throw Cloud SQL failures in unit tests (#898)
* Modify SignedMarkRevocationList to not swallow CloudSQL failures in unittests

* restore package-lock.json

* Added suppressExceptionUnlessInTest()

* Add a DatabaseMigrationUtils class

* small changes
2020-12-15 17:34:38 -05:00
sarahcaseybot
9975bc2195 Modify proxy to pass full certificate before login (#896)
* Modify proxy to pass full certificate until partner is logged in

* refactor tests

* revert package-lock.json

* add sample cert string to tests
2020-12-15 16:36:39 -05:00
gbrodman
cb16a7649f Add a scrap command to backfill Spec11 threats (#897)
This parses through all pre-existing Spec11 files in GCS (starting at
2019-01-01 which is basically when the new format started) and maps them
to the new Spec11ThreatMatch objects.

Because the old format stored domain names only and the new format stores
names + repo IDs, we need to retrieve the DomainBase objects from the
point in time of the scan (failing if they don't exist). Because the
same domains appear multiple times (we estimate a total of 100k+ entries
but only 1-2k unique domains) we cache the DomainBase objects that we
retrieve from Datastore.
2020-12-15 16:18:27 -05:00
Michael Muller
d7e2b24468 Allow disabling UpdateAutoTimestamp updates (#906)
* Allow disabling UpdateAutoTimestamp updates

Allow us to disable timestamp updates within a try-with-resources block for a
given thread.  This functionality will be needed for transaction replays both
to and from datastore.

As part of this, also upgrade the UpdateAutoTimestampTest to a
DualDatabaseTest so we can verify that the functionality works both on
Datastore and Cloud SQL.
2020-12-15 10:34:52 -05:00
gbrodman
7c364b4471 Add SetSqlReplayCheckpoint command for SQL replay (#895)
* Add SetSqlReplayCheckpoint command for SQL replay

We should set this to the same time that we initially populate the SQL
database from Datastore.
2020-12-11 17:41:06 -05:00
Shicong Huang
b5137c3d05 Convert HostResourceTest to work with Cloud SQL (#905) 2020-12-11 13:17:55 -05:00
Ben McIlwain
6a9929019a Use 10 workers instead of the default 100 for re-save all EPP resources (#904)
* Use 10 workers instead of the default 100 for re-save all EPP resources

The intended/desired effect is to have a larger number of GCS commit log diffs
spread out over a longer period of time, with each diff itself being
significantly smaller. This should retain roughly the same amount of total work
for the async Cloud SQL replication action to have to deal with, but spread
across 10X as much time.
2020-12-10 15:25:25 -05:00
Weimin Yu
83ed448741 Add a credential store backed by Secret Manager (#901)
* Add a credential store backed by Secret Manager

Added a SqlCredentialStore that stores user credentials with one level
of indirection: for each credential, an addtional secret is used to
identify the 'live' version of the credential. This is a work in
progress and the overall design is explained in
go/dr-sql-security.

Also added two nomulus commands for credential management. They are
stop-gap measures that will be deprecated by the planned privilege
management system.
2020-12-10 11:29:44 -05:00
gbrodman
2c6ee6dae9 Parameterize the serialization of objects being written to SQL (#892)
* Parameterize the serialization of objects being written to SQL

We shouldn't require that objects written to SQL during a Beam pipeline
be VersionedEntity objects -- they may be non-Objectify entities. As a
result, we should allow the user to specify what the objects are that
should be written to SQL.

Note: we will need to clean up the Spec11PipelineTest more but that can
be out of the scope of this PR.

* Overload the method and add a bit of javadoc

* Actually use the overloaded function
2020-12-09 15:52:56 -05:00
Shicong Huang
a181d6a720 Add a command to remove Registry 1.0 key in DomainBase (#900) 2020-12-09 10:30:23 -05:00
Michael Muller
db19f9ea4f Resurrect symmetric vkeys when possible. (#899)
* Resurrect symmetric vkeys when possible.

AbstractVKeyConverter had never been updated to generate symmetric VKeys for
simple (i.e. non-composite) VKey types.  Update it to default to generating a
VKey with a simple Ofy key.

With this change, composite VKeys must specify the "compositeKey = true" field
in the With*VKey annotations.  Doing so creates an asymmetric (SQL only) VKey
that can triggers higher-level logic to generate a corrected VKey containing
the composite Key.
2020-12-08 15:13:44 -05:00
Michael Muller
c7e6192929 Add replay extension to more domain tests (#893)
* Add replay extension to more domain tests

Add replay to DomainRenewFlowTest and DomainUpdateFlowTest.

* Formatting fix
2020-12-04 14:15:24 -05:00
gbrodman
a8effe8a1e Add an action to replay commit logs to SQL (#887)
* Add an action to replay commit logs to SQL

- We import from the commit-log GCS files so that we are sure that we
have a consistent snapshot.
- We use the object weighting (moved to ObjectWeights) to verify that we
are replaying objects in the correct order.
- We add a config setting (default false) of whether or not to replay
the logs.
- The action is triggered after the export if the aforementioned config
setting is on.

* Responses to CR

- Remove triggering of replay from the export action and remove the test
changes
- Add a method to load commit log diffs by transaction
- Replay one Datastore transaction at a time, per SQL transaction
- Minor logging / comment changes
- Change ObjectWeights to EntityWritePriorities and flesh out javadoc

* More CR responses

- Use one transaction per GCS diff file
- Fix up comments minutiae

* Add a class-level javadoc

* Add a log message and some periods

* bit of formatting

* Merge remote-tracking branch 'origin/master' into replayAction

* Handle toSqlEntity rather than toSqlEntities
2020-12-03 16:50:41 -05:00
gbrodman
4f0189c162 Convert DatastoreEntity/SqlEntity to return Optional, not List (#894)
* Convert DatastoreEntity/SqlEntity to return Optional, not List

We don't have any entities that convert to more than one entity, so we
can use an Optional instead for clarity and simplicity.
2020-12-02 17:29:01 -05:00
sarahcaseybot
59c852d812 Add an HTTP header to response from Nomulus after successful login (#879)
* Add a logged-in response header

* small fixes

* Refactor EPP test cases to check for headers

* small change
2020-12-01 19:24:56 -05:00
sarahcaseybot
2621448f5e Remove ability to set only the certificate hash for a registrar (#891) 2020-12-01 14:28:45 -05:00
Weimin Yu
94ef81dca4 Script to rolling-start Nomulus (#888)
* Script to rolling-start Nomulus

Add a script to restart Nomulus non-disruptively. This can be used after
a configuration change to external resources (e.g.,  Cloud SQL
credential) to make Nomulus pick up the latest config.

Also added proper support to paging based List api methods, replacing the
current hack that forces the server to return everything in one response.
The List method for instances has a lower limit on page size than others
which is not sufficient for our project.
2020-12-01 10:14:05 -05:00
Michael Muller
64e1a4b345 Make Domain -> BillingEvent FK deferred (#890)
* Make Domain -> BillingEvent FK deferred

It appears that Hibernate can sporadically introduce FK constraint failures
when updating a Domain to reference a new BillingEvent and then deleting the
old BillingEvent, causing a flakey test failure in DomainDeleteFlowTest.  This
may be due to the fact that this FK relationships is not known to hibernate.

An alternate solution appears to be to flush after every update, but that
likely has some pretty serious performance implications.
2020-11-30 18:06:07 -05:00
Michael Muller
cde1c78f5e Add replay-testing to DomainDeleteFlowTest (#886)
* Minor fixes:

- Initialize "requestedByRegistrar" to false (it's non-nullable).
- Store test entities (registrar, hosts and contacts) in JPA.

* Flyway changes

* Add ReplayExtension to DomainDeleteFlowTest

* Check in latest ER diagrams
2020-11-25 11:15:10 -05:00
gbrodman
3b1c198c11 Ensure that all relevant Keys can be converted to VKeys (#883)
* Ensure that all relevant Keys can be converted to VKeys

When replaying commit logs to SQL (specifically deletes) we will need to
convert Datastore Keys to SQL VKeys in order to know what (if anything)
to delete.

The test added to EntityTest (and the associated code changes) mean that
for every relevant object we'll be able to call
VKeyTranslatorFactory.createVKey(Key<?>) for all possible keys that we
care about. Note that we do not care about entities that only exist in
Datastore or entities that are non-replicated -- by their nature,
deletes for those types of objects in Datastore are not relevant in SQL.

* Responses to code review

- changing comments / method names
- using ModelUtils
2020-11-24 14:33:06 -05:00
Michael Muller
3afcc0dcb4 Fix DatastoreHelper -> DatabaseHelper in comments (#885)
* Fix DatastoreHelper -> DatabaseHelper in comments

Fix a few comments that still make reference to DatastoreHelper.
2020-11-24 14:32:15 -05:00
Michael Muller
67c6d73a18 Fix DomainHistory merge issues (#884)
* Reproduce DomainHistory double write failure

* Add fix for cascade sets and clean up hacks

* Fix DatastoreHelper to work with name change.

* Remove Ignored entities from ofy schema
2020-11-24 11:42:00 -05:00
Weimin Yu
33499aaf9e Make sure post load work happens in GracePeriod (#878)
* Make sure post load work happens in GracePeriod

The GracePeriod method with ofy @OnLoad annotation is not called.

Apparently Ofy only checks for @OnLoad on first-class entities,
not embedded ones.

Added a call to this method from DomainContent's OnLoad method.

Reproduced issue with a test and verified that the fix works.
2020-11-23 13:47:27 -05:00
Ben McIlwain
fcd79e7c18 Add a full ASCII StringGenerator alphabet (#882)
* Add a full ASCII StringGenerator alphabet

This is intended for the purpose of generating maximally secure passwords for
PostgreSQL and others. We may need to remove a few of these punctuation
characters if they prove to be more trouble than they're worth (e.g. backtick).
2020-11-19 18:08:04 -05:00
Ben McIlwain
d9c06ce54c Fix pseudo typo (#880)
* Fix pseudo typo
2020-11-19 17:25:19 -05:00
Ben McIlwain
b4b7fdbc07 Rename DatastoreHelper -> DatabaseHelper (#881)
* Rename DatastoreHelper -> DatabaseHelper

It already contains some functionality for dealing with Cloud SQL and will
increasingly contain more, so it should be renamed so that it does not falsely
imply it is specific only to Datastore.
2020-11-19 17:24:53 -05:00
gbrodman
ef6d3890bc Allow addition of extra entity classes for VKey conversion (#877)
* Allow addition of extra entity classes for VKey conversion

This allows us to create VKeys from Keys for test objects that may not
be part of the original codebase.

This isn't used anywhere directly yet but it will be useful in the
future when testing the replay of SQL transactions.
2020-11-19 13:38:28 -05:00
Shicong Huang
1c7c202a80 Display concrete test method name for @@DualDatabaseTest (#876) 2020-11-18 20:56:08 -05:00
Weimin Yu
e1ce357904 Drop foreign key constraints on PollMessages (#875)
* Drop foreign key constraints on PollMessages

PollMessages are deleted after being acked by recipients.
Other tables cannot have foreign key constraints on them.
2020-11-18 20:28:56 -05:00
Shicong Huang
1d91a8e647 Use DomainHistoryVKey to restore symmetric VKey (#874)
* Use DomainHistoryVKey to restore symmetric VKey

* Rebase on HEAD
2020-11-17 16:32:32 -05:00
Michael Muller
ab7ee51fb2 Add an extension to verify transaction replay (#857)
* Add an extension to verify transaction replay

Add ReplayExtension, which can be applied to test suites to verify that
transactions committed to datastore can be replayed to SQL.

This introduces a ReplayQueue class, which serves as a stand-in for the
current lack of replay-from-commit-logs.  It also includes replay logic in
TransactionInfo which introduces the concept of "entity class weights."
Entity weighting allows us store and delete objects in an order that is
consistent with the direction of foreign key and deferred foreign key
relationships.  As a general rule, lower weight classes must have no direct or
indirect non-deferred foreign key relationships on higher weight classes.

It is expected that much of this code will change when the final replay
mechanism is implemented.

* Minor fixes:

- Initialize "requestedByRegistrar" to false (it's non-nullable). [reverted
  during rebase: non-nullable was removed in another PR]
- Store test entities (registrar, hosts and contacts) in JPA.

* Make testbed save replay

This changes the replay system to make datastore saves initiated from the
testbed (as opposed to just the tested code) replay when the ReplayExtension
is enabled.  This requires modifications to DatastoreHelper and the
AppEngineExtension that the ReplayExtension can plug into.

This changes also has some necessary fixes to objects that are persisted by
the testbed (such as PremiumList).
2020-11-17 13:29:50 -05:00
Shicong Huang
c8159e7b35 Convert RegistrarTest to working with Postgresql (#865)
* Convert RegistrarTest to working with Postgresql

* Resolve comments
2020-11-16 12:18:28 -05:00
Weimin Yu
51942fcaad Make some columns nullable in History tables (#873)
* Make some columns nullable in History tables

xmlBytes is made nullable in all history tables since changes performed
by backend actions would not have it. In addition, epp requests are not saved to
ContactHistory since data may contain PII.

requestedByRegistrar in all history tables are made nullable. This
property is set from metadata in epp requests. Null means not provided.
2020-11-13 15:25:54 -05:00
Weimin Yu
ae6b414b82 Add a Secret Manager client for Nomulus (#872)
* Add a Secret Manager client for Nomulus
2020-11-12 17:12:52 -05:00
Shicong Huang
de20334a66 Add SQL schema for GracePeriodHistory (#746)
* Add schema for GracePeriodHistory

Rebase on HEAD

Rebase on HEAD

Rebase on HEAD and rename column

Use OfyService to generate id

Refactor GracePeriodsSubject

Rebase on HEAD

Remove GracePeriodSubject and GracePeriodsSubject

Rebase on HEAD

Rebase on HEAD

Rebase on HEAD

Add gracePeriodHistoryRevisionId and remove some foreign key

* Rebase on HEAD
2020-11-12 10:06:58 -05:00
Michael Muller
78bc967357 Replace ofyOrJpaTm with tm().isOfy() conditionals (#870)
* Replace ofyOrJpaTm with tm().isOfy() conditionals

Replace existing ofyOrJpaTm() calls with conditionals (either "if" statements
or ternary expressions) gated on tm().isOfy().
2020-11-11 07:40:43 -05:00
gbrodman
88f4f36678 Maintain only one row in the TmchCrl table at all times (#869)
* Maintain only one row in the TmchCrl table at all times
2020-11-10 17:09:26 -05:00
gbrodman
f1eb1a8fe7 Add SQL replay checkpoint object to SQL (#868)
* Add SQL replay checkpoint object to Datastore

This will be part of the asynchronous commit-log replay to SQL. Whenever
we successfully export commits up to a particular time, we should
persist that time so we don't replay the same commits again (it is not
idempotent)

* Move SqlReplayCheckpoint from DS to SQL

* Responses to CR
2020-11-10 17:09:18 -05:00
Michael Muller
82cc7b59fe Small SQL persistence fixes to model classes (#863)
* Small SQL persistence fixes to model classes

- Add a createVKey() method to Registry (Registry vkeys are composite)
- Add/fix toSqlEntities() methods in premium and reserved list classes.

* Remove fixes addressed by #866
2020-11-10 11:12:09 -05:00
Michael Muller
633377a594 Generate ER diagrams in :nom:generate_golden_file (#867)
Generate entity relationship diagrams during the same step in which we
generate the golden schema.
2020-11-10 10:03:28 -05:00
sarahcaseybot
c729a30d38 Add ECDSA key validation to Certificate Checker (#855)
* Add ecdsa key validation

* Add some comments

* fix merge conflicts

* change variable names

* Separate tests

* separate curve tests
2020-11-09 15:28:48 -05:00
gbrodman
acce1a7d3b Add DuallyWrittenEntity convenience interface (#866)
* Add DuallyWrittenEntity convenience interface

For classes that are dually written we don't need to worry about replay
conversion, so this just allows us to remove unnecessary unimportant
methods

* Rename DWE to NonReplicatedEntity and enable test
2020-11-09 15:13:40 -05:00
gbrodman
0c6363c04f Persist two singleton entities in SQL tables (#860)
* Persist two singleton entities in SQL tables

A table might not be the best place to store singleton entities, but by
doing this we ensure we can easily inspect them later and use the same
sort of persistence logic for these that we do elsewhere.

ServerSecret is stored upon retrieval so that we make sure that the same
secret is used in both Datastore and SQL (we wouldn't want to change
it).

* Responses to CR

* Don't have a separate ID for the singleton entities

* Rename secret UUID

* Rename and regenerate
2020-11-09 13:47:42 -05:00
Shicong Huang
cb764b5d30 Convert Registry realted tests to working with SQL (#862) 2020-11-09 12:10:55 -05:00
Shicong Huang
420f3bf380 Add missing foreign key constraints (#845)
* Add missing foreign key constraints

* Fix failed unit tests
2020-11-09 10:55:29 -05:00
Michael Muller
7097b0f5e6 Allow InjectExtension to apply in "before each" (#864)
* Allow InjectExtension to apply in "before each"

InjectExtension is non-standard in that, while it cleans up the changes it has
applied in its "after each" method, those changes must be applied by the test
driver or by another extension.  This breaks extension ordering, which is
something we're going to need to make heavy use of for the ReplayExtension.

Allow changes to be specified (but not activated) after construction and apply
all unactivated changes during "before each."
2020-11-09 07:52:55 -05:00
670 changed files with 22774 additions and 11484 deletions

View File

@@ -34,3 +34,5 @@ Guy Bensky <guyben@google.com>
Weimin Yu <weiminyu@google.com>
Shicong Huang <shicong@google.com>
Gustav Brodman <gbrodman@google.com>
Sarah Botwinick <sarahbot@google.com>
Legina Chen <legina@google.com>

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -50,7 +50,7 @@ org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -50,7 +50,7 @@ org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -50,7 +50,7 @@ org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -50,7 +50,7 @@ org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -50,7 +50,7 @@ org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -58,7 +58,7 @@ org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.hamcrest:hamcrest-core:1.3
org.json:json:20160212
org.junit.jupiter:junit-jupiter-api:5.6.2

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -58,7 +58,7 @@ org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.hamcrest:hamcrest-core:1.3
org.json:json:20160212
org.junit.jupiter:junit-jupiter-api:5.6.2

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -58,7 +58,7 @@ org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.hamcrest:hamcrest-core:1.3
org.json:json:20160212
org.junit.jupiter:junit-jupiter-api:5.6.2

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -58,7 +58,7 @@ org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.hamcrest:hamcrest-core:1.3
org.json:json:20160212
org.junit.jupiter:junit-jupiter-api:5.6.2

View File

@@ -4,9 +4,9 @@
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0

View File

@@ -4,9 +4,9 @@
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0

View File

@@ -4,9 +4,9 @@
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0

View File

@@ -4,9 +4,9 @@
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0

View File

@@ -4,9 +4,9 @@
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0

View File

@@ -4,9 +4,9 @@
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0

View File

@@ -6,7 +6,7 @@ com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -17,7 +17,7 @@ joda-time:joda-time:2.9.2
junit:junit:4.12
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
org.junit.jupiter:junit-jupiter-api:5.6.2

View File

@@ -6,7 +6,7 @@ com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -17,7 +17,7 @@ joda-time:joda-time:2.9.2
junit:junit:4.12
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
org.junit.jupiter:junit-jupiter-api:5.6.2

View File

@@ -7,7 +7,7 @@ com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.1
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -18,7 +18,7 @@ joda-time:joda-time:2.9.2
junit:junit:4.12
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
org.junit.jupiter:junit-jupiter-api:5.6.2

View File

@@ -7,7 +7,7 @@ com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.1
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -18,7 +18,7 @@ joda-time:joda-time:2.9.2
junit:junit:4.12
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
org.junit.jupiter:junit-jupiter-api:5.6.2

View File

@@ -6,7 +6,7 @@ com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -16,6 +16,6 @@ javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3

View File

@@ -6,7 +6,7 @@ com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -16,6 +16,6 @@ javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3

View File

@@ -7,7 +7,7 @@ com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.1
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -17,6 +17,6 @@ javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3

View File

@@ -7,7 +7,7 @@ com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.1
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -17,6 +17,6 @@ javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3

View File

@@ -0,0 +1,47 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.testing.truth;
import static com.google.common.truth.Truth.assertWithMessage;
import com.google.common.truth.Truth;
import javax.annotation.Nullable;
/** Utils class containing helper functions for {@link Truth}. */
public class TruthUtils {
/** Asserts that both of the given objects are either null or nonnull. */
public static void assertNullnessParity(@Nullable Object thisObj, @Nullable Object thatObj) {
if (thisObj == null) {
assertWithMessage("Expects both objects are null but thatObj is not null")
.that(thatObj)
.isNull();
} else {
assertWithMessage("Expects both objects are not null but thatObj is null")
.that(thatObj)
.isNotNull();
}
}
/** Asserts that both of the given objects are either null or nonnull. */
public static void assertNullnessParity(
@Nullable Object thisObj, @Nullable Object thatObj, String errorMessage) {
if (thisObj == null) {
assertWithMessage(errorMessage).that(thatObj).isNull();
} else {
assertWithMessage(errorMessage).that(thatObj).isNotNull();
}
}
}

View File

@@ -300,9 +300,15 @@ def do_pseudo_task(task: str) -> None:
f'{root}/db/src/main/resources/sql/schema/'
'nomulus.golden.sql')
if subprocess.call([f'{root}/gradlew', ':db:test']):
print('\033[31mERROR:\033[0m Golden file test failed after '
'copying schema. Please check your flyway files.')
# Rerun :db:test and regenerate the ER diagram (at "warning" log
# level so it doesn't generate pages of messaging)
if subprocess.call([f'{root}/gradlew', ':db:test', 'devTool',
'--args=-e localhost --log_level=WARNING '
'generate_sql_er_diagram -o '
f'{root}/db/src/main/resources/sql/er_diagram']):
print('\033[31mERROR:\033[0m Golden file test or ER diagram '
'generation failed after copying schema. Please check your '
'flyway files.')
raise Abort()
else:
print(f'\033[31mERROR:\033[0m Unknown task {task}')
@@ -377,6 +383,7 @@ def main(args) -> int:
# See if there are any special ":nom:" pseudo-tasks specified.
got_non_pseudo_tasks = False
got_pseudo_tasks = False
for arg in args.non_flag_args[1:]:
if arg.startswith(':nom:'):
if got_non_pseudo_tasks:
@@ -388,13 +395,14 @@ def main(args) -> int:
'specified prior to all actual gradle tasks. Aborting.')
return 1
do_pseudo_task(arg)
got_pseudo_tasks = True
else:
got_non_pseudo_tasks = True
non_flag_args = [
arg for arg in args.non_flag_args[1:] if not arg.startswith(':nom:')]
if not non_flag_args:
if not got_non_pseudo_tasks:
if not got_pseudo_tasks:
print('\033[33mWARNING:\033[0m No tasks specified. Not '
'doing anything')
return 0

View File

@@ -113,7 +113,10 @@ class MyTest(unittest.TestCase):
nom_build.main(['nom_build', ':nom:generate_golden_file'])
self.call_mock.assert_has_calls([
mock.call([GRADLEW, ':db:test']),
mock.call([GRADLEW, ':db:test'])
mock.call([GRADLEW, ':db:test', 'devTool',
'--args=-e localhost --log_level=WARNING '
'generate_sql_er_diagram -o '
'/tmp/rootdir/db/src/main/resources/sql/er_diagram'])
])
def test_generate_golden_file_nofail(self):
@@ -122,5 +125,3 @@ class MyTest(unittest.TestCase):
self.call_mock.assert_has_calls([mock.call([GRADLEW, ':db:test'])])
unittest.main()

View File

@@ -65,7 +65,7 @@ class PresubmitCheck:
for pattern in self.skipped_patterns:
if pattern in file:
return False
with open(file, "r") as f:
with open(file, "r", encoding='utf8') as f:
file_content = f.read()
matches = re.match(self.regex, file_content, re.DOTALL)
if self.regex_type == FORBIDDEN:
@@ -241,7 +241,7 @@ def verify_flyway_index():
# Remove the sequence numbers and compare against the index file contents.
files = [filename[1] for filename in sorted(files)]
with open('db/src/main/resources/sql/flyway.txt') as index:
with open('db/src/main/resources/sql/flyway.txt', encoding='utf8') as index:
indexed_files = index.read().splitlines()
if files != indexed_files:
unindexed = set(files) - set(indexed_files)

View File

@@ -171,6 +171,8 @@ dependencies {
testRuntime files(sourceSets.test.resources.srcDirs)
compile deps['com.beust:jcommander']
compile deps['com.google.api:gax']
compile deps['com.google.api.grpc:proto-google-cloud-secretmanager-v1']
compile deps['com.google.api-client:google-api-client']
compile deps['com.google.api-client:google-api-client-appengine']
compile deps['com.google.api-client:google-api-client-servlet']
@@ -196,6 +198,7 @@ dependencies {
compile deps['com.google.auth:google-auth-library-oauth2-http']
compile deps['com.google.cloud.sql:jdbc-socket-factory-core']
runtimeOnly deps['com.google.cloud.sql:postgres-socket-factory']
compile deps['com.google.cloud:google-cloud-secretmanager']
compile deps['com.google.code.gson:gson']
compile deps['com.google.auto.value:auto-value-annotations']
compile deps['com.google.code.findbugs:jsr305']
@@ -215,6 +218,7 @@ dependencies {
compile deps['com.google.oauth-client:google-oauth-client-jetty']
compile deps['com.google.oauth-client:google-oauth-client-appengine']
compile deps['com.google.oauth-client:google-oauth-client-servlet']
compile deps['com.google.protobuf:protobuf-java']
compile deps['com.google.re2j:re2j']
compile deps['com.google.template:soy']
compile deps['com.googlecode.json-simple:json-simple']
@@ -596,7 +600,6 @@ task compileProdJS(type: JavaExec) {
closureArgs << "--js=${jsDir}/soyutils_usegoog.js"
closureArgs << "--js=${cssSourceDir}/registrar_bin.css.js"
closureArgs << "--js=${jsSourceDir}/**.js"
// TODO(shicong) Verify the compiled JS file works in Alpha
closureArgs << "--js=${externsDir}/json.js"
closureArgs << "--js=${soySourceDir}/**.js"
args closureArgs
@@ -824,9 +827,8 @@ task buildToolImage(dependsOn: nomulus, type: Exec) {
commandLine 'docker', 'build', '-t', 'nomulus-tool', '.'
}
task generateInitSqlPipelineGraph(type: Test) {
include "**/InitSqlPipelineGraphTest.*"
testNameIncludePatterns = ["**createPipeline_compareGraph"]
task generateInitSqlPipelineGraph(type: FilteringTest) {
tests = ['InitSqlPipelineGraphTest.createPipeline_compareGraph']
ignoreFailures = true
}

View File

@@ -19,7 +19,7 @@ com.google.errorprone:error_prone_type_annotations:2.3.4
com.google.errorprone:javac-shaded:9-dev-r4023-3
com.google.googlejavaformat:google-java-format:1.5
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.protobuf:protobuf-java:3.4.0
@@ -30,7 +30,7 @@ javax.inject:javax.inject:1
javax.persistence:javax.persistence-api:2.2
net.ltgt.gradle.incap:incap:0.2
org.checkerframework:checker-compat-qual:2.5.3
org.checkerframework:checker-qual:3.0.0
org.checkerframework:checker-qual:3.5.0
org.checkerframework:dataflow:3.0.0
org.checkerframework:javacutil:3.0.0
org.jetbrains.kotlin:kotlin-stdlib-common:1.3.61

View File

@@ -7,7 +7,7 @@ args4j:args4j:2.33
cglib:cglib-nodep:2.2
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.10.2
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-core:2.11.3
com.fasterxml.jackson.core:jackson-databind:2.10.2
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
@@ -36,15 +36,17 @@ com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:1.9.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:1.9.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.85.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.85.1
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.2.3
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.2.3
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-v1:1.49.1
com.google.api.grpc:proto-google-common-protos:1.18.0
com.google.api.grpc:proto-google-iam-v1:0.13.0
com.google.api:api-common:1.8.1
com.google.api:gax-grpc:1.54.0
com.google.api.grpc:proto-google-common-protos:2.0.0
com.google.api.grpc:proto-google-iam-v1:1.0.1
com.google.api:api-common:1.10.1
com.google.api:gax-grpc:1.60.0
com.google.api:gax-httpjson:0.70.1
com.google.api:gax:1.54.0
com.google.api:gax:1.60.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20191211-1.30.9
@@ -67,9 +69,9 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.81
com.google.appengine:appengine-remote-api:1.9.81
com.google.appengine:appengine-testing:1.9.81
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7.1
com.google.auth:google-auth-library-credentials:0.22.0
com.google.auth:google-auth-library-oauth2-http:0.22.0
com.google.auto.value:auto-value-annotations:1.7.4
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:2.1.3
com.google.cloud.bigdataoss:util:2.1.3
@@ -82,23 +84,24 @@ com.google.cloud:google-cloud-bigtable:1.9.1
com.google.cloud:google-cloud-core-grpc:1.92.2
com.google.cloud:google-cloud-core-http:1.92.4
com.google.cloud:google-cloud-core:1.92.5
com.google.cloud:google-cloud-secretmanager:1.2.3
com.google.cloud:google-cloud-spanner:1.49.1
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_annotations:2.4.0
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
com.google.http-client:google-http-client-jackson2:1.34.2
com.google.http-client:google-http-client-jackson2:1.37.0
com.google.http-client:google-http-client-protobuf:1.33.0
com.google.http-client:google-http-client:1.34.2
com.google.http-client:google-http-client:1.37.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -110,8 +113,8 @@ com.google.oauth-client:google-oauth-client-java6:1.30.6
com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.30.5
com.google.oauth-client:google-oauth-client:1.30.6
com.google.protobuf:protobuf-java-util:3.11.4
com.google.protobuf:protobuf-java:3.11.4
com.google.protobuf:protobuf-java-util:3.13.0
com.google.protobuf:protobuf-java:3.13.0
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.googlecode.charts4j:charts4j:1.3
@@ -132,18 +135,18 @@ dnsjava:dnsjava:2.1.7
io.dropwizard.metrics:metrics-core:3.2.6
io.github.classgraph:classgraph:4.8.65
io.grpc:grpc-all:1.27.2
io.grpc:grpc-alts:1.29.0
io.grpc:grpc-api:1.29.0
io.grpc:grpc-auth:1.29.0
io.grpc:grpc-context:1.29.0
io.grpc:grpc-core:1.29.0
io.grpc:grpc-grpclb:1.29.0
io.grpc:grpc-netty-shaded:1.29.0
io.grpc:grpc-alts:1.32.2
io.grpc:grpc-api:1.32.2
io.grpc:grpc-auth:1.32.2
io.grpc:grpc-context:1.32.2
io.grpc:grpc-core:1.32.2
io.grpc:grpc-grpclb:1.32.2
io.grpc:grpc-netty-shaded:1.32.2
io.grpc:grpc-netty:1.27.2
io.grpc:grpc-okhttp:1.27.2
io.grpc:grpc-protobuf-lite:1.29.0
io.grpc:grpc-protobuf:1.29.0
io.grpc:grpc-stub:1.29.0
io.grpc:grpc-protobuf-lite:1.32.2
io.grpc:grpc-protobuf:1.32.2
io.grpc:grpc-stub:1.32.2
io.grpc:grpc-testing:1.27.2
io.netty:netty-buffer:4.1.42.Final
io.netty:netty-codec-http2:4.1.42.Final
@@ -198,7 +201,7 @@ org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.beam:beam-vendor-sdks-java-extensions-protobuf:2.23.0
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.13
org.apache.logging.log4j:log4j-api:2.6.2
org.bouncycastle:bcpg-jdk15on:1.61
@@ -208,7 +211,7 @@ org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.codehaus.mojo:animal-sniffer-annotations:1.19
org.conscrypt:conscrypt-openjdk-uber:2.2.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
@@ -246,7 +249,7 @@ org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.4.1
org.threeten:threetenbp:1.4.5
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4

View File

@@ -7,7 +7,7 @@ args4j:args4j:2.33
cglib:cglib-nodep:2.2
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.10.2
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-core:2.11.3
com.fasterxml.jackson.core:jackson-databind:2.10.2
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
@@ -18,7 +18,6 @@ com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.android:annotations:4.1.1.4
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.30.9
com.google.api-client:google-api-client-java6:1.30.9
@@ -36,15 +35,17 @@ com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:1.9.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:1.9.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.85.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.85.1
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.2.3
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.2.3
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-v1:1.49.1
com.google.api.grpc:proto-google-common-protos:1.18.0
com.google.api.grpc:proto-google-iam-v1:0.13.0
com.google.api:api-common:1.8.1
com.google.api:gax-grpc:1.54.0
com.google.api.grpc:proto-google-common-protos:2.0.0
com.google.api.grpc:proto-google-iam-v1:1.0.1
com.google.api:api-common:1.10.1
com.google.api:gax-grpc:1.60.0
com.google.api:gax-httpjson:0.70.1
com.google.api:gax:1.54.0
com.google.api:gax:1.60.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20191211-1.30.9
@@ -67,9 +68,9 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.81
com.google.appengine:appengine-remote-api:1.9.81
com.google.appengine:appengine-testing:1.9.81
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7.1
com.google.auth:google-auth-library-credentials:0.22.0
com.google.auth:google-auth-library-oauth2-http:0.22.0
com.google.auto.value:auto-value-annotations:1.7.4
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:2.1.3
com.google.cloud.bigdataoss:util:2.1.3
@@ -82,22 +83,23 @@ com.google.cloud:google-cloud-bigtable:1.9.1
com.google.cloud:google-cloud-core-grpc:1.92.2
com.google.cloud:google-cloud-core-http:1.92.4
com.google.cloud:google-cloud-core:1.92.5
com.google.cloud:google-cloud-secretmanager:1.2.3
com.google.cloud:google-cloud-spanner:1.49.1
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_annotations:2.4.0
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
com.google.http-client:google-http-client-jackson2:1.34.2
com.google.http-client:google-http-client-jackson2:1.37.0
com.google.http-client:google-http-client-protobuf:1.33.0
com.google.http-client:google-http-client:1.34.2
com.google.http-client:google-http-client:1.37.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -110,7 +112,7 @@ com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.30.5
com.google.oauth-client:google-oauth-client:1.30.6
com.google.protobuf:protobuf-java-util:3.11.4
com.google.protobuf:protobuf-java:3.11.4
com.google.protobuf:protobuf-java:3.13.0
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.googlecode.charts4j:charts4j:1.3
@@ -131,18 +133,18 @@ dnsjava:dnsjava:2.1.7
io.dropwizard.metrics:metrics-core:3.2.6
io.github.classgraph:classgraph:4.8.65
io.grpc:grpc-all:1.27.2
io.grpc:grpc-alts:1.29.0
io.grpc:grpc-api:1.29.0
io.grpc:grpc-auth:1.29.0
io.grpc:grpc-context:1.29.0
io.grpc:grpc-core:1.29.0
io.grpc:grpc-grpclb:1.29.0
io.grpc:grpc-netty-shaded:1.29.0
io.grpc:grpc-alts:1.32.2
io.grpc:grpc-api:1.32.2
io.grpc:grpc-auth:1.32.2
io.grpc:grpc-context:1.32.2
io.grpc:grpc-core:1.32.2
io.grpc:grpc-grpclb:1.32.2
io.grpc:grpc-netty-shaded:1.32.2
io.grpc:grpc-netty:1.27.2
io.grpc:grpc-okhttp:1.27.2
io.grpc:grpc-protobuf-lite:1.29.0
io.grpc:grpc-protobuf:1.29.0
io.grpc:grpc-stub:1.29.0
io.grpc:grpc-protobuf-lite:1.32.2
io.grpc:grpc-protobuf:1.32.2
io.grpc:grpc-stub:1.32.2
io.grpc:grpc-testing:1.27.2
io.netty:netty-buffer:4.1.42.Final
io.netty:netty-codec-http2:4.1.42.Final
@@ -158,7 +160,6 @@ io.netty:netty-transport:4.1.42.Final
io.opencensus:opencensus-api:0.24.0
io.opencensus:opencensus-contrib-grpc-util:0.24.0
io.opencensus:opencensus-contrib-http-util:0.24.0
io.perfmark:perfmark-api:0.19.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
@@ -196,17 +197,16 @@ org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.beam:beam-vendor-sdks-java-extensions-protobuf:2.23.0
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.13
org.apache.logging.log4j:log4j-api:2.6.2
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcpkix-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.conscrypt:conscrypt-openjdk-uber:2.2.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
@@ -243,7 +243,7 @@ org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.4.1
org.threeten:threetenbp:1.4.5
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4

View File

@@ -11,7 +11,7 @@ com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.10.2
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-core:2.11.3
com.fasterxml.jackson.core:jackson-databind:2.10.2
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
@@ -40,15 +40,17 @@ com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:1.9.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:1.9.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.85.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.85.1
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.2.3
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.2.3
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-v1:1.49.1
com.google.api.grpc:proto-google-common-protos:1.18.0
com.google.api.grpc:proto-google-iam-v1:0.13.0
com.google.api:api-common:1.8.1
com.google.api:gax-grpc:1.54.0
com.google.api.grpc:proto-google-common-protos:2.0.0
com.google.api.grpc:proto-google-iam-v1:1.0.1
com.google.api:api-common:1.10.1
com.google.api:gax-grpc:1.60.0
com.google.api:gax-httpjson:0.70.1
com.google.api:gax:1.54.0
com.google.api:gax:1.60.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20191211-1.30.9
@@ -71,9 +73,9 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.81
com.google.appengine:appengine-remote-api:1.9.81
com.google.appengine:appengine-testing:1.9.81
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7.1
com.google.auth:google-auth-library-credentials:0.22.0
com.google.auth:google-auth-library-oauth2-http:0.22.0
com.google.auto.value:auto-value-annotations:1.7.4
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:2.1.3
com.google.cloud.bigdataoss:util:2.1.3
@@ -87,23 +89,24 @@ com.google.cloud:google-cloud-bigtable:1.9.1
com.google.cloud:google-cloud-core-grpc:1.92.2
com.google.cloud:google-cloud-core-http:1.92.4
com.google.cloud:google-cloud-core:1.92.5
com.google.cloud:google-cloud-secretmanager:1.2.3
com.google.cloud:google-cloud-spanner:1.49.1
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_annotations:2.4.0
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
com.google.http-client:google-http-client-jackson2:1.34.2
com.google.http-client:google-http-client-jackson2:1.37.0
com.google.http-client:google-http-client-protobuf:1.33.0
com.google.http-client:google-http-client:1.34.2
com.google.http-client:google-http-client:1.37.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -115,8 +118,8 @@ com.google.oauth-client:google-oauth-client-java6:1.30.6
com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.30.5
com.google.oauth-client:google-oauth-client:1.30.6
com.google.protobuf:protobuf-java-util:3.11.4
com.google.protobuf:protobuf-java:3.11.4
com.google.protobuf:protobuf-java-util:3.13.0
com.google.protobuf:protobuf-java:3.13.0
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.googlecode.charts4j:charts4j:1.3
@@ -140,18 +143,18 @@ guru.nidi:graphviz-java:0.17.0
io.dropwizard.metrics:metrics-core:3.2.6
io.github.classgraph:classgraph:4.8.65
io.grpc:grpc-all:1.27.2
io.grpc:grpc-alts:1.29.0
io.grpc:grpc-api:1.29.0
io.grpc:grpc-auth:1.29.0
io.grpc:grpc-context:1.29.0
io.grpc:grpc-core:1.29.0
io.grpc:grpc-grpclb:1.29.0
io.grpc:grpc-netty-shaded:1.29.0
io.grpc:grpc-alts:1.32.2
io.grpc:grpc-api:1.32.2
io.grpc:grpc-auth:1.32.2
io.grpc:grpc-context:1.32.2
io.grpc:grpc-core:1.32.2
io.grpc:grpc-grpclb:1.32.2
io.grpc:grpc-netty-shaded:1.32.2
io.grpc:grpc-netty:1.27.2
io.grpc:grpc-okhttp:1.27.2
io.grpc:grpc-protobuf-lite:1.29.0
io.grpc:grpc-protobuf:1.29.0
io.grpc:grpc-stub:1.29.0
io.grpc:grpc-protobuf-lite:1.32.2
io.grpc:grpc-protobuf:1.32.2
io.grpc:grpc-stub:1.32.2
io.grpc:grpc-testing:1.27.2
io.netty:netty-buffer:4.1.42.Final
io.netty:netty-codec-http2:4.1.42.Final
@@ -208,7 +211,7 @@ org.apache.beam:beam-vendor-sdks-java-extensions-protobuf:2.23.0
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.13
org.apache.logging.log4j:log4j-api:2.13.3
org.apache.logging.log4j:log4j-core:2.13.3
@@ -219,7 +222,7 @@ org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.codehaus.mojo:animal-sniffer-annotations:1.19
org.conscrypt:conscrypt-openjdk-uber:2.2.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
@@ -260,7 +263,7 @@ org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.4.1
org.threeten:threetenbp:1.4.5
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3

View File

@@ -11,7 +11,7 @@ com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.10.2
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-core:2.11.3
com.fasterxml.jackson.core:jackson-databind:2.10.2
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
@@ -40,15 +40,17 @@ com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:1.9.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:1.9.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.85.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.85.1
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.2.3
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.2.3
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-v1:1.49.1
com.google.api.grpc:proto-google-common-protos:1.18.0
com.google.api.grpc:proto-google-iam-v1:0.13.0
com.google.api:api-common:1.8.1
com.google.api:gax-grpc:1.54.0
com.google.api.grpc:proto-google-common-protos:2.0.0
com.google.api.grpc:proto-google-iam-v1:1.0.1
com.google.api:api-common:1.10.1
com.google.api:gax-grpc:1.60.0
com.google.api:gax-httpjson:0.70.1
com.google.api:gax:1.54.0
com.google.api:gax:1.60.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20191211-1.30.9
@@ -71,9 +73,9 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.81
com.google.appengine:appengine-remote-api:1.9.81
com.google.appengine:appengine-testing:1.9.81
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7.1
com.google.auth:google-auth-library-credentials:0.22.0
com.google.auth:google-auth-library-oauth2-http:0.22.0
com.google.auto.value:auto-value-annotations:1.7.4
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:2.1.3
com.google.cloud.bigdataoss:util:2.1.3
@@ -87,23 +89,24 @@ com.google.cloud:google-cloud-bigtable:1.9.1
com.google.cloud:google-cloud-core-grpc:1.92.2
com.google.cloud:google-cloud-core-http:1.92.4
com.google.cloud:google-cloud-core:1.92.5
com.google.cloud:google-cloud-secretmanager:1.2.3
com.google.cloud:google-cloud-spanner:1.49.1
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_annotations:2.4.0
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
com.google.http-client:google-http-client-jackson2:1.34.2
com.google.http-client:google-http-client-jackson2:1.37.0
com.google.http-client:google-http-client-protobuf:1.33.0
com.google.http-client:google-http-client:1.34.2
com.google.http-client:google-http-client:1.37.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -115,8 +118,8 @@ com.google.oauth-client:google-oauth-client-java6:1.30.6
com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.30.5
com.google.oauth-client:google-oauth-client:1.30.6
com.google.protobuf:protobuf-java-util:3.11.4
com.google.protobuf:protobuf-java:3.11.4
com.google.protobuf:protobuf-java-util:3.13.0
com.google.protobuf:protobuf-java:3.13.0
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.googlecode.charts4j:charts4j:1.3
@@ -140,18 +143,18 @@ guru.nidi:graphviz-java:0.17.0
io.dropwizard.metrics:metrics-core:3.2.6
io.github.classgraph:classgraph:4.8.65
io.grpc:grpc-all:1.27.2
io.grpc:grpc-alts:1.29.0
io.grpc:grpc-api:1.29.0
io.grpc:grpc-auth:1.29.0
io.grpc:grpc-context:1.29.0
io.grpc:grpc-core:1.29.0
io.grpc:grpc-grpclb:1.29.0
io.grpc:grpc-netty-shaded:1.29.0
io.grpc:grpc-alts:1.32.2
io.grpc:grpc-api:1.32.2
io.grpc:grpc-auth:1.32.2
io.grpc:grpc-context:1.32.2
io.grpc:grpc-core:1.32.2
io.grpc:grpc-grpclb:1.32.2
io.grpc:grpc-netty-shaded:1.32.2
io.grpc:grpc-netty:1.27.2
io.grpc:grpc-okhttp:1.27.2
io.grpc:grpc-protobuf-lite:1.29.0
io.grpc:grpc-protobuf:1.29.0
io.grpc:grpc-stub:1.29.0
io.grpc:grpc-protobuf-lite:1.32.2
io.grpc:grpc-protobuf:1.32.2
io.grpc:grpc-stub:1.32.2
io.grpc:grpc-testing:1.27.2
io.netty:netty-buffer:4.1.42.Final
io.netty:netty-codec-http2:4.1.42.Final
@@ -207,7 +210,7 @@ org.apache.beam:beam-vendor-sdks-java-extensions-protobuf:2.23.0
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.13
org.apache.logging.log4j:log4j-api:2.13.3
org.apache.logging.log4j:log4j-core:2.13.3
@@ -218,7 +221,7 @@ org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.codehaus.mojo:animal-sniffer-annotations:1.19
org.conscrypt:conscrypt-openjdk-uber:2.2.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
@@ -258,7 +261,7 @@ org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.4.1
org.threeten:threetenbp:1.4.5
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3

View File

@@ -7,7 +7,7 @@ args4j:args4j:2.33
cglib:cglib-nodep:2.2
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.10.2
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-core:2.11.3
com.fasterxml.jackson.core:jackson-databind:2.10.2
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
@@ -36,15 +36,17 @@ com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:1.9.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:1.9.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.85.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.85.1
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.2.3
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.2.3
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-v1:1.49.1
com.google.api.grpc:proto-google-common-protos:1.18.0
com.google.api.grpc:proto-google-iam-v1:0.13.0
com.google.api:api-common:1.8.1
com.google.api:gax-grpc:1.54.0
com.google.api.grpc:proto-google-common-protos:2.0.0
com.google.api.grpc:proto-google-iam-v1:1.0.1
com.google.api:api-common:1.10.1
com.google.api:gax-grpc:1.60.0
com.google.api:gax-httpjson:0.70.1
com.google.api:gax:1.54.0
com.google.api:gax:1.60.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20191211-1.30.9
@@ -67,9 +69,9 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.81
com.google.appengine:appengine-remote-api:1.9.81
com.google.appengine:appengine-testing:1.9.81
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7.1
com.google.auth:google-auth-library-credentials:0.22.0
com.google.auth:google-auth-library-oauth2-http:0.22.0
com.google.auto.value:auto-value-annotations:1.7.4
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:2.1.3
com.google.cloud.bigdataoss:util:2.1.3
@@ -82,23 +84,24 @@ com.google.cloud:google-cloud-bigtable:1.9.1
com.google.cloud:google-cloud-core-grpc:1.92.2
com.google.cloud:google-cloud-core-http:1.92.4
com.google.cloud:google-cloud-core:1.92.5
com.google.cloud:google-cloud-secretmanager:1.2.3
com.google.cloud:google-cloud-spanner:1.49.1
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_annotations:2.4.0
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
com.google.http-client:google-http-client-jackson2:1.34.2
com.google.http-client:google-http-client-jackson2:1.37.0
com.google.http-client:google-http-client-protobuf:1.33.0
com.google.http-client:google-http-client:1.34.2
com.google.http-client:google-http-client:1.37.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -110,8 +113,8 @@ com.google.oauth-client:google-oauth-client-java6:1.30.6
com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.30.5
com.google.oauth-client:google-oauth-client:1.30.6
com.google.protobuf:protobuf-java-util:3.11.4
com.google.protobuf:protobuf-java:3.11.4
com.google.protobuf:protobuf-java-util:3.13.0
com.google.protobuf:protobuf-java:3.13.0
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.googlecode.charts4j:charts4j:1.3
@@ -132,18 +135,18 @@ dnsjava:dnsjava:2.1.7
io.dropwizard.metrics:metrics-core:3.2.6
io.github.classgraph:classgraph:4.8.65
io.grpc:grpc-all:1.27.2
io.grpc:grpc-alts:1.29.0
io.grpc:grpc-api:1.29.0
io.grpc:grpc-auth:1.29.0
io.grpc:grpc-context:1.29.0
io.grpc:grpc-core:1.29.0
io.grpc:grpc-grpclb:1.29.0
io.grpc:grpc-netty-shaded:1.29.0
io.grpc:grpc-alts:1.32.2
io.grpc:grpc-api:1.32.2
io.grpc:grpc-auth:1.32.2
io.grpc:grpc-context:1.32.2
io.grpc:grpc-core:1.32.2
io.grpc:grpc-grpclb:1.32.2
io.grpc:grpc-netty-shaded:1.32.2
io.grpc:grpc-netty:1.27.2
io.grpc:grpc-okhttp:1.27.2
io.grpc:grpc-protobuf-lite:1.29.0
io.grpc:grpc-protobuf:1.29.0
io.grpc:grpc-stub:1.29.0
io.grpc:grpc-protobuf-lite:1.32.2
io.grpc:grpc-protobuf:1.32.2
io.grpc:grpc-stub:1.32.2
io.grpc:grpc-testing:1.27.2
io.netty:netty-buffer:4.1.42.Final
io.netty:netty-codec-http2:4.1.42.Final
@@ -198,7 +201,7 @@ org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.beam:beam-vendor-sdks-java-extensions-protobuf:2.23.0
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.13
org.apache.logging.log4j:log4j-api:2.6.2
org.bouncycastle:bcpg-jdk15on:1.61
@@ -208,7 +211,7 @@ org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.codehaus.mojo:animal-sniffer-annotations:1.19
org.conscrypt:conscrypt-openjdk-uber:2.2.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
@@ -246,7 +249,7 @@ org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.4.1
org.threeten:threetenbp:1.4.5
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4

View File

@@ -7,7 +7,7 @@ args4j:args4j:2.33
cglib:cglib-nodep:2.2
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.10.2
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-core:2.11.3
com.fasterxml.jackson.core:jackson-databind:2.10.2
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
@@ -18,7 +18,6 @@ com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.android:annotations:4.1.1.4
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.30.9
com.google.api-client:google-api-client-java6:1.30.9
@@ -36,15 +35,17 @@ com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:1.9.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:1.9.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.85.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.85.1
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.2.3
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.2.3
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-v1:1.49.1
com.google.api.grpc:proto-google-common-protos:1.18.0
com.google.api.grpc:proto-google-iam-v1:0.13.0
com.google.api:api-common:1.8.1
com.google.api:gax-grpc:1.54.0
com.google.api.grpc:proto-google-common-protos:2.0.0
com.google.api.grpc:proto-google-iam-v1:1.0.1
com.google.api:api-common:1.10.1
com.google.api:gax-grpc:1.60.0
com.google.api:gax-httpjson:0.70.1
com.google.api:gax:1.54.0
com.google.api:gax:1.60.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20191211-1.30.9
@@ -67,9 +68,9 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.81
com.google.appengine:appengine-remote-api:1.9.81
com.google.appengine:appengine-testing:1.9.81
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7.1
com.google.auth:google-auth-library-credentials:0.22.0
com.google.auth:google-auth-library-oauth2-http:0.22.0
com.google.auto.value:auto-value-annotations:1.7.4
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:2.1.3
com.google.cloud.bigdataoss:util:2.1.3
@@ -82,22 +83,23 @@ com.google.cloud:google-cloud-bigtable:1.9.1
com.google.cloud:google-cloud-core-grpc:1.92.2
com.google.cloud:google-cloud-core-http:1.92.4
com.google.cloud:google-cloud-core:1.92.5
com.google.cloud:google-cloud-secretmanager:1.2.3
com.google.cloud:google-cloud-spanner:1.49.1
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_annotations:2.4.0
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
com.google.http-client:google-http-client-jackson2:1.34.2
com.google.http-client:google-http-client-jackson2:1.37.0
com.google.http-client:google-http-client-protobuf:1.33.0
com.google.http-client:google-http-client:1.34.2
com.google.http-client:google-http-client:1.37.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -110,7 +112,7 @@ com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.30.5
com.google.oauth-client:google-oauth-client:1.30.6
com.google.protobuf:protobuf-java-util:3.11.4
com.google.protobuf:protobuf-java:3.11.4
com.google.protobuf:protobuf-java:3.13.0
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.googlecode.charts4j:charts4j:1.3
@@ -131,18 +133,18 @@ dnsjava:dnsjava:2.1.7
io.dropwizard.metrics:metrics-core:3.2.6
io.github.classgraph:classgraph:4.8.65
io.grpc:grpc-all:1.27.2
io.grpc:grpc-alts:1.29.0
io.grpc:grpc-api:1.29.0
io.grpc:grpc-auth:1.29.0
io.grpc:grpc-context:1.29.0
io.grpc:grpc-core:1.29.0
io.grpc:grpc-grpclb:1.29.0
io.grpc:grpc-netty-shaded:1.29.0
io.grpc:grpc-alts:1.32.2
io.grpc:grpc-api:1.32.2
io.grpc:grpc-auth:1.32.2
io.grpc:grpc-context:1.32.2
io.grpc:grpc-core:1.32.2
io.grpc:grpc-grpclb:1.32.2
io.grpc:grpc-netty-shaded:1.32.2
io.grpc:grpc-netty:1.27.2
io.grpc:grpc-okhttp:1.27.2
io.grpc:grpc-protobuf-lite:1.29.0
io.grpc:grpc-protobuf:1.29.0
io.grpc:grpc-stub:1.29.0
io.grpc:grpc-protobuf-lite:1.32.2
io.grpc:grpc-protobuf:1.32.2
io.grpc:grpc-stub:1.32.2
io.grpc:grpc-testing:1.27.2
io.netty:netty-buffer:4.1.42.Final
io.netty:netty-codec-http2:4.1.42.Final
@@ -158,7 +160,6 @@ io.netty:netty-transport:4.1.42.Final
io.opencensus:opencensus-api:0.24.0
io.opencensus:opencensus-contrib-grpc-util:0.24.0
io.opencensus:opencensus-contrib-http-util:0.24.0
io.perfmark:perfmark-api:0.19.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
@@ -197,17 +198,16 @@ org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.beam:beam-vendor-sdks-java-extensions-protobuf:2.23.0
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.13
org.apache.logging.log4j:log4j-api:2.6.2
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcpkix-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.conscrypt:conscrypt-openjdk-uber:2.2.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
@@ -245,7 +245,7 @@ org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.4.1
org.threeten:threetenbp:1.4.5
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4

View File

@@ -11,7 +11,7 @@ com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.10.2
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-core:2.11.3
com.fasterxml.jackson.core:jackson-databind:2.10.2
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
@@ -40,15 +40,17 @@ com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:1.9.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:1.9.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.85.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.85.1
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.2.3
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.2.3
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-v1:1.49.1
com.google.api.grpc:proto-google-common-protos:1.18.0
com.google.api.grpc:proto-google-iam-v1:0.13.0
com.google.api:api-common:1.8.1
com.google.api:gax-grpc:1.54.0
com.google.api.grpc:proto-google-common-protos:2.0.0
com.google.api.grpc:proto-google-iam-v1:1.0.1
com.google.api:api-common:1.10.1
com.google.api:gax-grpc:1.60.0
com.google.api:gax-httpjson:0.70.1
com.google.api:gax:1.54.0
com.google.api:gax:1.60.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20191211-1.30.9
@@ -71,9 +73,9 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.81
com.google.appengine:appengine-remote-api:1.9.81
com.google.appengine:appengine-testing:1.9.81
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7.1
com.google.auth:google-auth-library-credentials:0.22.0
com.google.auth:google-auth-library-oauth2-http:0.22.0
com.google.auto.value:auto-value-annotations:1.7.4
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:2.1.3
com.google.cloud.bigdataoss:util:2.1.3
@@ -86,23 +88,24 @@ com.google.cloud:google-cloud-bigtable:1.9.1
com.google.cloud:google-cloud-core-grpc:1.92.2
com.google.cloud:google-cloud-core-http:1.92.4
com.google.cloud:google-cloud-core:1.92.5
com.google.cloud:google-cloud-secretmanager:1.2.3
com.google.cloud:google-cloud-spanner:1.49.1
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_annotations:2.4.0
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
com.google.http-client:google-http-client-jackson2:1.34.2
com.google.http-client:google-http-client-jackson2:1.37.0
com.google.http-client:google-http-client-protobuf:1.33.0
com.google.http-client:google-http-client:1.34.2
com.google.http-client:google-http-client:1.37.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -114,8 +117,8 @@ com.google.oauth-client:google-oauth-client-java6:1.30.6
com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.30.5
com.google.oauth-client:google-oauth-client:1.30.6
com.google.protobuf:protobuf-java-util:3.11.4
com.google.protobuf:protobuf-java:3.11.4
com.google.protobuf:protobuf-java-util:3.13.0
com.google.protobuf:protobuf-java:3.13.0
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.googlecode.charts4j:charts4j:1.3
@@ -139,18 +142,18 @@ guru.nidi:graphviz-java:0.17.0
io.dropwizard.metrics:metrics-core:3.2.6
io.github.classgraph:classgraph:4.8.65
io.grpc:grpc-all:1.27.2
io.grpc:grpc-alts:1.29.0
io.grpc:grpc-api:1.29.0
io.grpc:grpc-auth:1.29.0
io.grpc:grpc-context:1.29.0
io.grpc:grpc-core:1.29.0
io.grpc:grpc-grpclb:1.29.0
io.grpc:grpc-netty-shaded:1.29.0
io.grpc:grpc-alts:1.32.2
io.grpc:grpc-api:1.32.2
io.grpc:grpc-auth:1.32.2
io.grpc:grpc-context:1.32.2
io.grpc:grpc-core:1.32.2
io.grpc:grpc-grpclb:1.32.2
io.grpc:grpc-netty-shaded:1.32.2
io.grpc:grpc-netty:1.27.2
io.grpc:grpc-okhttp:1.27.2
io.grpc:grpc-protobuf-lite:1.29.0
io.grpc:grpc-protobuf:1.29.0
io.grpc:grpc-stub:1.29.0
io.grpc:grpc-protobuf-lite:1.32.2
io.grpc:grpc-protobuf:1.32.2
io.grpc:grpc-stub:1.32.2
io.grpc:grpc-testing:1.27.2
io.netty:netty-buffer:4.1.42.Final
io.netty:netty-codec-http2:4.1.42.Final
@@ -207,7 +210,7 @@ org.apache.beam:beam-vendor-sdks-java-extensions-protobuf:2.23.0
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.13
org.apache.logging.log4j:log4j-api:2.13.3
org.apache.logging.log4j:log4j-core:2.13.3
@@ -218,7 +221,7 @@ org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.codehaus.mojo:animal-sniffer-annotations:1.19
org.conscrypt:conscrypt-openjdk-uber:2.2.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
@@ -258,7 +261,7 @@ org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.4.1
org.threeten:threetenbp:1.4.5
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3

View File

@@ -11,7 +11,7 @@ com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.10.2
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-core:2.11.3
com.fasterxml.jackson.core:jackson-databind:2.10.2
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
@@ -40,15 +40,17 @@ com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:1.9.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:1.9.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.85.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.85.1
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.2.3
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.2.3
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-v1:1.49.1
com.google.api.grpc:proto-google-common-protos:1.18.0
com.google.api.grpc:proto-google-iam-v1:0.13.0
com.google.api:api-common:1.8.1
com.google.api:gax-grpc:1.54.0
com.google.api.grpc:proto-google-common-protos:2.0.0
com.google.api.grpc:proto-google-iam-v1:1.0.1
com.google.api:api-common:1.10.1
com.google.api:gax-grpc:1.60.0
com.google.api:gax-httpjson:0.70.1
com.google.api:gax:1.54.0
com.google.api:gax:1.60.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20191211-1.30.9
@@ -71,9 +73,9 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.81
com.google.appengine:appengine-remote-api:1.9.81
com.google.appengine:appengine-testing:1.9.81
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7.1
com.google.auth:google-auth-library-credentials:0.22.0
com.google.auth:google-auth-library-oauth2-http:0.22.0
com.google.auto.value:auto-value-annotations:1.7.4
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:2.1.3
com.google.cloud.bigdataoss:util:2.1.3
@@ -86,23 +88,24 @@ com.google.cloud:google-cloud-bigtable:1.9.1
com.google.cloud:google-cloud-core-grpc:1.92.2
com.google.cloud:google-cloud-core-http:1.92.4
com.google.cloud:google-cloud-core:1.92.5
com.google.cloud:google-cloud-secretmanager:1.2.3
com.google.cloud:google-cloud-spanner:1.49.1
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_annotations:2.4.0
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
com.google.http-client:google-http-client-jackson2:1.34.2
com.google.http-client:google-http-client-jackson2:1.37.0
com.google.http-client:google-http-client-protobuf:1.33.0
com.google.http-client:google-http-client:1.34.2
com.google.http-client:google-http-client:1.37.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -114,8 +117,8 @@ com.google.oauth-client:google-oauth-client-java6:1.30.6
com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.30.5
com.google.oauth-client:google-oauth-client:1.30.6
com.google.protobuf:protobuf-java-util:3.11.4
com.google.protobuf:protobuf-java:3.11.4
com.google.protobuf:protobuf-java-util:3.13.0
com.google.protobuf:protobuf-java:3.13.0
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.googlecode.charts4j:charts4j:1.3
@@ -139,18 +142,18 @@ guru.nidi:graphviz-java:0.17.0
io.dropwizard.metrics:metrics-core:3.2.6
io.github.classgraph:classgraph:4.8.65
io.grpc:grpc-all:1.27.2
io.grpc:grpc-alts:1.29.0
io.grpc:grpc-api:1.29.0
io.grpc:grpc-auth:1.29.0
io.grpc:grpc-context:1.29.0
io.grpc:grpc-core:1.29.0
io.grpc:grpc-grpclb:1.29.0
io.grpc:grpc-netty-shaded:1.29.0
io.grpc:grpc-alts:1.32.2
io.grpc:grpc-api:1.32.2
io.grpc:grpc-auth:1.32.2
io.grpc:grpc-context:1.32.2
io.grpc:grpc-core:1.32.2
io.grpc:grpc-grpclb:1.32.2
io.grpc:grpc-netty-shaded:1.32.2
io.grpc:grpc-netty:1.27.2
io.grpc:grpc-okhttp:1.27.2
io.grpc:grpc-protobuf-lite:1.29.0
io.grpc:grpc-protobuf:1.29.0
io.grpc:grpc-stub:1.29.0
io.grpc:grpc-protobuf-lite:1.32.2
io.grpc:grpc-protobuf:1.32.2
io.grpc:grpc-stub:1.32.2
io.grpc:grpc-testing:1.27.2
io.netty:netty-buffer:4.1.42.Final
io.netty:netty-codec-http2:4.1.42.Final
@@ -207,7 +210,7 @@ org.apache.beam:beam-vendor-sdks-java-extensions-protobuf:2.23.0
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.13
org.apache.logging.log4j:log4j-api:2.13.3
org.apache.logging.log4j:log4j-core:2.13.3
@@ -218,7 +221,7 @@ org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.codehaus.mojo:animal-sniffer-annotations:1.19
org.conscrypt:conscrypt-openjdk-uber:2.2.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
@@ -258,7 +261,7 @@ org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.4.1
org.threeten:threetenbp:1.4.5
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3

View File

@@ -11,7 +11,7 @@ com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.10.2
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-core:2.11.3
com.fasterxml.jackson.core:jackson-databind:2.10.2
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
@@ -40,15 +40,17 @@ com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:1.9.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:1.9.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.85.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.85.1
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.2.3
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.2.3
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-v1:1.49.1
com.google.api.grpc:proto-google-common-protos:1.18.0
com.google.api.grpc:proto-google-iam-v1:0.13.0
com.google.api:api-common:1.8.1
com.google.api:gax-grpc:1.54.0
com.google.api.grpc:proto-google-common-protos:2.0.0
com.google.api.grpc:proto-google-iam-v1:1.0.1
com.google.api:api-common:1.10.1
com.google.api:gax-grpc:1.60.0
com.google.api:gax-httpjson:0.70.1
com.google.api:gax:1.54.0
com.google.api:gax:1.60.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20191211-1.30.9
@@ -71,9 +73,9 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.81
com.google.appengine:appengine-remote-api:1.9.81
com.google.appengine:appengine-testing:1.9.81
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7.1
com.google.auth:google-auth-library-credentials:0.22.0
com.google.auth:google-auth-library-oauth2-http:0.22.0
com.google.auto.value:auto-value-annotations:1.7.4
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:2.1.3
com.google.cloud.bigdataoss:util:2.1.3
@@ -86,23 +88,24 @@ com.google.cloud:google-cloud-bigtable:1.9.1
com.google.cloud:google-cloud-core-grpc:1.92.2
com.google.cloud:google-cloud-core-http:1.92.4
com.google.cloud:google-cloud-core:1.92.5
com.google.cloud:google-cloud-secretmanager:1.2.3
com.google.cloud:google-cloud-spanner:1.49.1
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_annotations:2.4.0
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
com.google.http-client:google-http-client-jackson2:1.34.2
com.google.http-client:google-http-client-jackson2:1.37.0
com.google.http-client:google-http-client-protobuf:1.33.0
com.google.http-client:google-http-client:1.34.2
com.google.http-client:google-http-client:1.37.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -114,8 +117,8 @@ com.google.oauth-client:google-oauth-client-java6:1.30.6
com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.30.5
com.google.oauth-client:google-oauth-client:1.30.6
com.google.protobuf:protobuf-java-util:3.11.4
com.google.protobuf:protobuf-java:3.11.4
com.google.protobuf:protobuf-java-util:3.13.0
com.google.protobuf:protobuf-java:3.13.0
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.googlecode.charts4j:charts4j:1.3
@@ -139,18 +142,18 @@ guru.nidi:graphviz-java:0.17.0
io.dropwizard.metrics:metrics-core:3.2.6
io.github.classgraph:classgraph:4.8.65
io.grpc:grpc-all:1.27.2
io.grpc:grpc-alts:1.29.0
io.grpc:grpc-api:1.29.0
io.grpc:grpc-auth:1.29.0
io.grpc:grpc-context:1.29.0
io.grpc:grpc-core:1.29.0
io.grpc:grpc-grpclb:1.29.0
io.grpc:grpc-netty-shaded:1.29.0
io.grpc:grpc-alts:1.32.2
io.grpc:grpc-api:1.32.2
io.grpc:grpc-auth:1.32.2
io.grpc:grpc-context:1.32.2
io.grpc:grpc-core:1.32.2
io.grpc:grpc-grpclb:1.32.2
io.grpc:grpc-netty-shaded:1.32.2
io.grpc:grpc-netty:1.27.2
io.grpc:grpc-okhttp:1.27.2
io.grpc:grpc-protobuf-lite:1.29.0
io.grpc:grpc-protobuf:1.29.0
io.grpc:grpc-stub:1.29.0
io.grpc:grpc-protobuf-lite:1.32.2
io.grpc:grpc-protobuf:1.32.2
io.grpc:grpc-stub:1.32.2
io.grpc:grpc-testing:1.27.2
io.netty:netty-buffer:4.1.42.Final
io.netty:netty-codec-http2:4.1.42.Final
@@ -207,7 +210,7 @@ org.apache.beam:beam-vendor-sdks-java-extensions-protobuf:2.23.0
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.13
org.apache.logging.log4j:log4j-api:2.13.3
org.apache.logging.log4j:log4j-core:2.13.3
@@ -218,7 +221,7 @@ org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.codehaus.mojo:animal-sniffer-annotations:1.19
org.conscrypt:conscrypt-openjdk-uber:2.2.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
@@ -258,7 +261,7 @@ org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.4.1
org.threeten:threetenbp:1.4.5
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3

View File

@@ -11,7 +11,7 @@ com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.10.2
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-core:2.11.3
com.fasterxml.jackson.core:jackson-databind:2.10.2
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
@@ -40,15 +40,17 @@ com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:1.9.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:1.9.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.85.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.85.1
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.2.3
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.2.3
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-v1:1.49.1
com.google.api.grpc:proto-google-common-protos:1.18.0
com.google.api.grpc:proto-google-iam-v1:0.13.0
com.google.api:api-common:1.8.1
com.google.api:gax-grpc:1.54.0
com.google.api.grpc:proto-google-common-protos:2.0.0
com.google.api.grpc:proto-google-iam-v1:1.0.1
com.google.api:api-common:1.10.1
com.google.api:gax-grpc:1.60.0
com.google.api:gax-httpjson:0.70.1
com.google.api:gax:1.54.0
com.google.api:gax:1.60.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20191211-1.30.9
@@ -71,9 +73,9 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.81
com.google.appengine:appengine-remote-api:1.9.81
com.google.appengine:appengine-testing:1.9.81
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7.1
com.google.auth:google-auth-library-credentials:0.22.0
com.google.auth:google-auth-library-oauth2-http:0.22.0
com.google.auto.value:auto-value-annotations:1.7.4
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:2.1.3
com.google.cloud.bigdataoss:util:2.1.3
@@ -87,23 +89,24 @@ com.google.cloud:google-cloud-bigtable:1.9.1
com.google.cloud:google-cloud-core-grpc:1.92.2
com.google.cloud:google-cloud-core-http:1.92.4
com.google.cloud:google-cloud-core:1.92.5
com.google.cloud:google-cloud-secretmanager:1.2.3
com.google.cloud:google-cloud-spanner:1.49.1
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_annotations:2.4.0
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
com.google.http-client:google-http-client-jackson2:1.34.2
com.google.http-client:google-http-client-jackson2:1.37.0
com.google.http-client:google-http-client-protobuf:1.33.0
com.google.http-client:google-http-client:1.34.2
com.google.http-client:google-http-client:1.37.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -115,8 +118,8 @@ com.google.oauth-client:google-oauth-client-java6:1.30.6
com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.30.5
com.google.oauth-client:google-oauth-client:1.30.6
com.google.protobuf:protobuf-java-util:3.11.4
com.google.protobuf:protobuf-java:3.11.4
com.google.protobuf:protobuf-java-util:3.13.0
com.google.protobuf:protobuf-java:3.13.0
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.googlecode.charts4j:charts4j:1.3
@@ -140,18 +143,18 @@ guru.nidi:graphviz-java:0.17.0
io.dropwizard.metrics:metrics-core:3.2.6
io.github.classgraph:classgraph:4.8.65
io.grpc:grpc-all:1.27.2
io.grpc:grpc-alts:1.29.0
io.grpc:grpc-api:1.29.0
io.grpc:grpc-auth:1.29.0
io.grpc:grpc-context:1.29.0
io.grpc:grpc-core:1.29.0
io.grpc:grpc-grpclb:1.29.0
io.grpc:grpc-netty-shaded:1.29.0
io.grpc:grpc-alts:1.32.2
io.grpc:grpc-api:1.32.2
io.grpc:grpc-auth:1.32.2
io.grpc:grpc-context:1.32.2
io.grpc:grpc-core:1.32.2
io.grpc:grpc-grpclb:1.32.2
io.grpc:grpc-netty-shaded:1.32.2
io.grpc:grpc-netty:1.27.2
io.grpc:grpc-okhttp:1.27.2
io.grpc:grpc-protobuf-lite:1.29.0
io.grpc:grpc-protobuf:1.29.0
io.grpc:grpc-stub:1.29.0
io.grpc:grpc-protobuf-lite:1.32.2
io.grpc:grpc-protobuf:1.32.2
io.grpc:grpc-stub:1.32.2
io.grpc:grpc-testing:1.27.2
io.netty:netty-buffer:4.1.42.Final
io.netty:netty-codec-http2:4.1.42.Final
@@ -207,7 +210,7 @@ org.apache.beam:beam-vendor-sdks-java-extensions-protobuf:2.23.0
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.13
org.apache.logging.log4j:log4j-api:2.13.3
org.apache.logging.log4j:log4j-core:2.13.3
@@ -218,7 +221,7 @@ org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.codehaus.mojo:animal-sniffer-annotations:1.19
org.conscrypt:conscrypt-openjdk-uber:2.2.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
@@ -258,7 +261,7 @@ org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.4.1
org.threeten:threetenbp:1.4.5
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3

View File

@@ -19,7 +19,7 @@ com.google.errorprone:error_prone_type_annotations:2.3.4
com.google.errorprone:javac-shaded:9-dev-r4023-3
com.google.googlejavaformat:google-java-format:1.5
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:29.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.protobuf:protobuf-java:3.4.0
@@ -30,7 +30,7 @@ javax.inject:javax.inject:1
javax.persistence:javax.persistence-api:2.2
net.ltgt.gradle.incap:incap:0.2
org.checkerframework:checker-compat-qual:2.5.3
org.checkerframework:checker-qual:3.0.0
org.checkerframework:checker-qual:3.5.0
org.checkerframework:dataflow:3.0.0
org.checkerframework:javacutil:3.0.0
org.jetbrains.kotlin:kotlin-stdlib-common:1.3.61

View File

@@ -7,7 +7,7 @@ args4j:args4j:2.33
cglib:cglib-nodep:2.2
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.11.2
com.fasterxml.jackson.core:jackson-core:2.11.2
com.fasterxml.jackson.core:jackson-core:2.11.3
com.fasterxml.jackson.core:jackson-databind:2.11.2
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
@@ -36,15 +36,17 @@ com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:1.9.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:1.9.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.85.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.85.1
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.2.3
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.2.3
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-v1:1.49.1
com.google.api.grpc:proto-google-common-protos:1.18.0
com.google.api.grpc:proto-google-iam-v1:0.13.0
com.google.api:api-common:1.8.1
com.google.api:gax-grpc:1.54.0
com.google.api.grpc:proto-google-common-protos:2.0.0
com.google.api.grpc:proto-google-iam-v1:1.0.1
com.google.api:api-common:1.10.1
com.google.api:gax-grpc:1.60.0
com.google.api:gax-httpjson:0.70.1
com.google.api:gax:1.54.0
com.google.api:gax:1.60.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20191211-1.30.9
@@ -68,9 +70,9 @@ com.google.appengine:appengine-api-1.0-sdk:1.9.81
com.google.appengine:appengine-api-stubs:1.9.81
com.google.appengine:appengine-remote-api:1.9.81
com.google.appengine:appengine-testing:1.9.81
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7.1
com.google.auth:google-auth-library-credentials:0.22.0
com.google.auth:google-auth-library-oauth2-http:0.22.0
com.google.auto.value:auto-value-annotations:1.7.4
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:2.1.3
com.google.cloud.bigdataoss:util:2.1.3
@@ -83,24 +85,25 @@ com.google.cloud:google-cloud-bigtable:1.9.1
com.google.cloud:google-cloud-core-grpc:1.92.2
com.google.cloud:google-cloud-core-http:1.92.4
com.google.cloud:google-cloud-core:1.92.5
com.google.cloud:google-cloud-secretmanager:1.2.3
com.google.cloud:google-cloud-spanner:1.49.1
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_annotations:2.4.0
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava-testlib:29.0-jre
com.google.guava:guava:29.0-jre
com.google.guava:guava-testlib:30.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
com.google.http-client:google-http-client-jackson2:1.34.2
com.google.http-client:google-http-client-jackson2:1.37.0
com.google.http-client:google-http-client-protobuf:1.33.0
com.google.http-client:google-http-client:1.34.2
com.google.http-client:google-http-client:1.37.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -113,8 +116,8 @@ com.google.oauth-client:google-oauth-client-java6:1.30.6
com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.30.5
com.google.oauth-client:google-oauth-client:1.30.6
com.google.protobuf:protobuf-java-util:3.11.4
com.google.protobuf:protobuf-java:3.11.4
com.google.protobuf:protobuf-java-util:3.13.0
com.google.protobuf:protobuf-java:3.13.0
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.google.truth.extensions:truth-java8-extension:1.0
@@ -140,18 +143,18 @@ dnsjava:dnsjava:2.1.7
io.dropwizard.metrics:metrics-core:3.2.6
io.github.classgraph:classgraph:4.8.65
io.grpc:grpc-all:1.27.2
io.grpc:grpc-alts:1.29.0
io.grpc:grpc-api:1.29.0
io.grpc:grpc-auth:1.29.0
io.grpc:grpc-context:1.29.0
io.grpc:grpc-core:1.29.0
io.grpc:grpc-grpclb:1.29.0
io.grpc:grpc-netty-shaded:1.29.0
io.grpc:grpc-alts:1.32.2
io.grpc:grpc-api:1.32.2
io.grpc:grpc-auth:1.32.2
io.grpc:grpc-context:1.32.2
io.grpc:grpc-core:1.32.2
io.grpc:grpc-grpclb:1.32.2
io.grpc:grpc-netty-shaded:1.32.2
io.grpc:grpc-netty:1.27.2
io.grpc:grpc-okhttp:1.27.2
io.grpc:grpc-protobuf-lite:1.29.0
io.grpc:grpc-protobuf:1.29.0
io.grpc:grpc-stub:1.29.0
io.grpc:grpc-protobuf-lite:1.32.2
io.grpc:grpc-protobuf:1.32.2
io.grpc:grpc-stub:1.32.2
io.grpc:grpc-testing:1.27.2
io.netty:netty-buffer:4.1.42.Final
io.netty:netty-codec-http2:4.1.42.Final
@@ -212,7 +215,7 @@ org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.ftpserver:ftplet-api:1.0.6
org.apache.ftpserver:ftpserver-core:1.0.6
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.13
org.apache.logging.log4j:log4j-api:2.6.2
org.apache.mina:mina-core:2.0.4
@@ -228,7 +231,7 @@ org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.codehaus.mojo:animal-sniffer-annotations:1.19
org.conscrypt:conscrypt-openjdk-uber:2.2.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
@@ -294,7 +297,7 @@ org.testcontainers:junit-jupiter:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:selenium:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.4.1
org.threeten:threetenbp:1.4.5
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4

View File

@@ -7,7 +7,7 @@ args4j:args4j:2.33
cglib:cglib-nodep:2.2
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.11.2
com.fasterxml.jackson.core:jackson-core:2.11.2
com.fasterxml.jackson.core:jackson-core:2.11.3
com.fasterxml.jackson.core:jackson-databind:2.11.2
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
@@ -18,7 +18,6 @@ com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.android:annotations:4.1.1.4
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.30.9
com.google.api-client:google-api-client-java6:1.30.9
@@ -36,15 +35,17 @@ com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:1.9.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:1.9.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.85.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.85.1
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.2.3
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.2.3
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-v1:1.49.1
com.google.api.grpc:proto-google-common-protos:1.18.0
com.google.api.grpc:proto-google-iam-v1:0.13.0
com.google.api:api-common:1.8.1
com.google.api:gax-grpc:1.54.0
com.google.api.grpc:proto-google-common-protos:2.0.0
com.google.api.grpc:proto-google-iam-v1:1.0.1
com.google.api:api-common:1.10.1
com.google.api:gax-grpc:1.60.0
com.google.api:gax-httpjson:0.70.1
com.google.api:gax:1.54.0
com.google.api:gax:1.60.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20191211-1.30.9
@@ -68,9 +69,9 @@ com.google.appengine:appengine-api-1.0-sdk:1.9.81
com.google.appengine:appengine-api-stubs:1.9.81
com.google.appengine:appengine-remote-api:1.9.81
com.google.appengine:appengine-testing:1.9.81
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7.1
com.google.auth:google-auth-library-credentials:0.22.0
com.google.auth:google-auth-library-oauth2-http:0.22.0
com.google.auto.value:auto-value-annotations:1.7.4
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:2.1.3
com.google.cloud.bigdataoss:util:2.1.3
@@ -83,23 +84,24 @@ com.google.cloud:google-cloud-bigtable:1.9.1
com.google.cloud:google-cloud-core-grpc:1.92.2
com.google.cloud:google-cloud-core-http:1.92.4
com.google.cloud:google-cloud-core:1.92.5
com.google.cloud:google-cloud-secretmanager:1.2.3
com.google.cloud:google-cloud-spanner:1.49.1
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_annotations:2.4.0
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava-testlib:29.0-jre
com.google.guava:guava:29.0-jre
com.google.guava:guava-testlib:30.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
com.google.http-client:google-http-client-jackson2:1.34.2
com.google.http-client:google-http-client-jackson2:1.37.0
com.google.http-client:google-http-client-protobuf:1.33.0
com.google.http-client:google-http-client:1.34.2
com.google.http-client:google-http-client:1.37.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -113,7 +115,7 @@ com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.30.5
com.google.oauth-client:google-oauth-client:1.30.6
com.google.protobuf:protobuf-java-util:3.11.4
com.google.protobuf:protobuf-java:3.11.4
com.google.protobuf:protobuf-java:3.13.0
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.google.truth.extensions:truth-java8-extension:1.0
@@ -139,18 +141,18 @@ dnsjava:dnsjava:2.1.7
io.dropwizard.metrics:metrics-core:3.2.6
io.github.classgraph:classgraph:4.8.65
io.grpc:grpc-all:1.27.2
io.grpc:grpc-alts:1.29.0
io.grpc:grpc-api:1.29.0
io.grpc:grpc-auth:1.29.0
io.grpc:grpc-context:1.29.0
io.grpc:grpc-core:1.29.0
io.grpc:grpc-grpclb:1.29.0
io.grpc:grpc-netty-shaded:1.29.0
io.grpc:grpc-alts:1.32.2
io.grpc:grpc-api:1.32.2
io.grpc:grpc-auth:1.32.2
io.grpc:grpc-context:1.32.2
io.grpc:grpc-core:1.32.2
io.grpc:grpc-grpclb:1.32.2
io.grpc:grpc-netty-shaded:1.32.2
io.grpc:grpc-netty:1.27.2
io.grpc:grpc-okhttp:1.27.2
io.grpc:grpc-protobuf-lite:1.29.0
io.grpc:grpc-protobuf:1.29.0
io.grpc:grpc-stub:1.29.0
io.grpc:grpc-protobuf-lite:1.32.2
io.grpc:grpc-protobuf:1.32.2
io.grpc:grpc-stub:1.32.2
io.grpc:grpc-testing:1.27.2
io.netty:netty-buffer:4.1.42.Final
io.netty:netty-codec-http2:4.1.42.Final
@@ -166,7 +168,6 @@ io.netty:netty-transport:4.1.42.Final
io.opencensus:opencensus-api:0.24.0
io.opencensus:opencensus-contrib-grpc-util:0.24.0
io.opencensus:opencensus-contrib-http-util:0.24.0
io.perfmark:perfmark-api:0.19.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
@@ -211,7 +212,7 @@ org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.ftpserver:ftplet-api:1.0.6
org.apache.ftpserver:ftpserver-core:1.0.6
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.13
org.apache.logging.log4j:log4j-api:2.6.2
org.apache.mina:mina-core:2.0.4
@@ -224,10 +225,9 @@ org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcpkix-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.11.1
org.checkerframework:checker-qual:3.5.0
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.conscrypt:conscrypt-openjdk-uber:2.2.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
@@ -293,7 +293,7 @@ org.testcontainers:junit-jupiter:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:selenium:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.4.1
org.threeten:threetenbp:1.4.5
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4

View File

@@ -11,7 +11,7 @@ com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.11.2
com.fasterxml.jackson.core:jackson-core:2.11.2
com.fasterxml.jackson.core:jackson-core:2.11.3
com.fasterxml.jackson.core:jackson-databind:2.11.2
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
@@ -40,15 +40,17 @@ com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:1.9.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:1.9.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.85.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.85.1
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.2.3
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.2.3
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-v1:1.49.1
com.google.api.grpc:proto-google-common-protos:1.18.0
com.google.api.grpc:proto-google-iam-v1:0.13.0
com.google.api:api-common:1.8.1
com.google.api:gax-grpc:1.54.0
com.google.api.grpc:proto-google-common-protos:2.0.0
com.google.api.grpc:proto-google-iam-v1:1.0.1
com.google.api:api-common:1.10.1
com.google.api:gax-grpc:1.60.0
com.google.api:gax-httpjson:0.70.1
com.google.api:gax:1.54.0
com.google.api:gax:1.60.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20191211-1.30.9
@@ -72,9 +74,9 @@ com.google.appengine:appengine-api-1.0-sdk:1.9.81
com.google.appengine:appengine-api-stubs:1.9.81
com.google.appengine:appengine-remote-api:1.9.81
com.google.appengine:appengine-testing:1.9.81
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7.1
com.google.auth:google-auth-library-credentials:0.22.0
com.google.auth:google-auth-library-oauth2-http:0.22.0
com.google.auto.value:auto-value-annotations:1.7.4
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:2.1.3
com.google.cloud.bigdataoss:util:2.1.3
@@ -88,24 +90,25 @@ com.google.cloud:google-cloud-bigtable:1.9.1
com.google.cloud:google-cloud-core-grpc:1.92.2
com.google.cloud:google-cloud-core-http:1.92.4
com.google.cloud:google-cloud-core:1.92.5
com.google.cloud:google-cloud-secretmanager:1.2.3
com.google.cloud:google-cloud-spanner:1.49.1
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_annotations:2.4.0
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava-testlib:29.0-jre
com.google.guava:guava:29.0-jre
com.google.guava:guava-testlib:30.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
com.google.http-client:google-http-client-jackson2:1.34.2
com.google.http-client:google-http-client-jackson2:1.37.0
com.google.http-client:google-http-client-protobuf:1.33.0
com.google.http-client:google-http-client:1.34.2
com.google.http-client:google-http-client:1.37.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -118,8 +121,8 @@ com.google.oauth-client:google-oauth-client-java6:1.30.6
com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.30.5
com.google.oauth-client:google-oauth-client:1.30.6
com.google.protobuf:protobuf-java-util:3.11.4
com.google.protobuf:protobuf-java:3.11.4
com.google.protobuf:protobuf-java-util:3.13.0
com.google.protobuf:protobuf-java:3.13.0
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.google.truth.extensions:truth-java8-extension:1.0
@@ -149,18 +152,18 @@ io.dropwizard.metrics:metrics-core:3.2.6
io.github.classgraph:classgraph:4.8.65
io.github.java-diff-utils:java-diff-utils:4.0
io.grpc:grpc-all:1.27.2
io.grpc:grpc-alts:1.29.0
io.grpc:grpc-api:1.29.0
io.grpc:grpc-auth:1.29.0
io.grpc:grpc-context:1.29.0
io.grpc:grpc-core:1.29.0
io.grpc:grpc-grpclb:1.29.0
io.grpc:grpc-netty-shaded:1.29.0
io.grpc:grpc-alts:1.32.2
io.grpc:grpc-api:1.32.2
io.grpc:grpc-auth:1.32.2
io.grpc:grpc-context:1.32.2
io.grpc:grpc-core:1.32.2
io.grpc:grpc-grpclb:1.32.2
io.grpc:grpc-netty-shaded:1.32.2
io.grpc:grpc-netty:1.27.2
io.grpc:grpc-okhttp:1.27.2
io.grpc:grpc-protobuf-lite:1.29.0
io.grpc:grpc-protobuf:1.29.0
io.grpc:grpc-stub:1.29.0
io.grpc:grpc-protobuf-lite:1.32.2
io.grpc:grpc-protobuf:1.32.2
io.grpc:grpc-stub:1.32.2
io.grpc:grpc-testing:1.27.2
io.netty:netty-buffer:4.1.42.Final
io.netty:netty-codec-http2:4.1.42.Final
@@ -222,7 +225,7 @@ org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.ftpserver:ftplet-api:1.0.6
org.apache.ftpserver:ftpserver-core:1.0.6
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.13
org.apache.logging.log4j:log4j-api:2.13.3
org.apache.logging.log4j:log4j-core:2.13.3
@@ -239,7 +242,7 @@ org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.codehaus.mojo:animal-sniffer-annotations:1.19
org.conscrypt:conscrypt-openjdk-uber:2.2.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
@@ -308,7 +311,7 @@ org.testcontainers:junit-jupiter:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:selenium:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.4.1
org.threeten:threetenbp:1.4.5
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3

View File

@@ -11,7 +11,7 @@ com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.11.2
com.fasterxml.jackson.core:jackson-core:2.11.2
com.fasterxml.jackson.core:jackson-core:2.11.3
com.fasterxml.jackson.core:jackson-databind:2.11.2
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
@@ -40,15 +40,17 @@ com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:1.9.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:1.9.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.85.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.85.1
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.2.3
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.2.3
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.49.1
com.google.api.grpc:proto-google-cloud-spanner-v1:1.49.1
com.google.api.grpc:proto-google-common-protos:1.18.0
com.google.api.grpc:proto-google-iam-v1:0.13.0
com.google.api:api-common:1.8.1
com.google.api:gax-grpc:1.54.0
com.google.api.grpc:proto-google-common-protos:2.0.0
com.google.api.grpc:proto-google-iam-v1:1.0.1
com.google.api:api-common:1.10.1
com.google.api:gax-grpc:1.60.0
com.google.api:gax-httpjson:0.70.1
com.google.api:gax:1.54.0
com.google.api:gax:1.60.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20191211-1.30.9
@@ -72,9 +74,9 @@ com.google.appengine:appengine-api-1.0-sdk:1.9.81
com.google.appengine:appengine-api-stubs:1.9.81
com.google.appengine:appengine-remote-api:1.9.81
com.google.appengine:appengine-testing:1.9.81
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7.1
com.google.auth:google-auth-library-credentials:0.22.0
com.google.auth:google-auth-library-oauth2-http:0.22.0
com.google.auto.value:auto-value-annotations:1.7.4
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:2.1.3
com.google.cloud.bigdataoss:util:2.1.3
@@ -88,24 +90,25 @@ com.google.cloud:google-cloud-bigtable:1.9.1
com.google.cloud:google-cloud-core-grpc:1.92.2
com.google.cloud:google-cloud-core-http:1.92.4
com.google.cloud:google-cloud-core:1.92.5
com.google.cloud:google-cloud-secretmanager:1.2.3
com.google.cloud:google-cloud-spanner:1.49.1
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_annotations:2.4.0
com.google.flogger:flogger-system-backend:0.5.1
com.google.flogger:flogger:0.5.1
com.google.flogger:google-extensions:0.5.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava-testlib:29.0-jre
com.google.guava:guava:29.0-jre
com.google.guava:guava-testlib:30.0-jre
com.google.guava:guava:30.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
com.google.http-client:google-http-client-jackson2:1.34.2
com.google.http-client:google-http-client-jackson2:1.37.0
com.google.http-client:google-http-client-protobuf:1.33.0
com.google.http-client:google-http-client:1.34.2
com.google.http-client:google-http-client:1.37.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -118,8 +121,8 @@ com.google.oauth-client:google-oauth-client-java6:1.30.6
com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.30.5
com.google.oauth-client:google-oauth-client:1.30.6
com.google.protobuf:protobuf-java-util:3.11.4
com.google.protobuf:protobuf-java:3.11.4
com.google.protobuf:protobuf-java-util:3.13.0
com.google.protobuf:protobuf-java:3.13.0
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.google.truth.extensions:truth-java8-extension:1.0
@@ -149,18 +152,18 @@ io.dropwizard.metrics:metrics-core:3.2.6
io.github.classgraph:classgraph:4.8.65
io.github.java-diff-utils:java-diff-utils:4.0
io.grpc:grpc-all:1.27.2
io.grpc:grpc-alts:1.29.0
io.grpc:grpc-api:1.29.0
io.grpc:grpc-auth:1.29.0
io.grpc:grpc-context:1.29.0
io.grpc:grpc-core:1.29.0
io.grpc:grpc-grpclb:1.29.0
io.grpc:grpc-netty-shaded:1.29.0
io.grpc:grpc-alts:1.32.2
io.grpc:grpc-api:1.32.2
io.grpc:grpc-auth:1.32.2
io.grpc:grpc-context:1.32.2
io.grpc:grpc-core:1.32.2
io.grpc:grpc-grpclb:1.32.2
io.grpc:grpc-netty-shaded:1.32.2
io.grpc:grpc-netty:1.27.2
io.grpc:grpc-okhttp:1.27.2
io.grpc:grpc-protobuf-lite:1.29.0
io.grpc:grpc-protobuf:1.29.0
io.grpc:grpc-stub:1.29.0
io.grpc:grpc-protobuf-lite:1.32.2
io.grpc:grpc-protobuf:1.32.2
io.grpc:grpc-stub:1.32.2
io.grpc:grpc-testing:1.27.2
io.netty:netty-buffer:4.1.42.Final
io.netty:netty-codec-http2:4.1.42.Final
@@ -222,7 +225,7 @@ org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.ftpserver:ftplet-api:1.0.6
org.apache.ftpserver:ftpserver-core:1.0.6
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.13
org.apache.logging.log4j:log4j-api:2.13.3
org.apache.logging.log4j:log4j-core:2.13.3
@@ -239,7 +242,7 @@ org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.5.0
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.codehaus.mojo:animal-sniffer-annotations:1.19
org.conscrypt:conscrypt-openjdk-uber:2.2.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
@@ -309,7 +312,7 @@ org.testcontainers:junit-jupiter:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:selenium:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.4.1
org.threeten:threetenbp:1.4.5
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3

View File

@@ -15,10 +15,10 @@
package google.registry.backup;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.backup.BackupUtils.createDeserializingIterator;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Streams;
import google.registry.model.ImmutableObject;
import google.registry.model.ofy.CommitLogCheckpoint;
import google.registry.model.ofy.CommitLogManifest;
@@ -31,7 +31,6 @@ import java.io.InputStream;
import java.nio.channels.Channels;
import java.nio.channels.ReadableByteChannel;
import java.util.Iterator;
import java.util.stream.Stream;
/**
* Helpers for reading CommitLog records from a file.
@@ -43,6 +42,56 @@ public final class CommitLogImports {
private CommitLogImports() {}
/**
* Returns entities in an {@code inputStream} (from a single CommitLog file) as an {@link
* ImmutableList} of {@link ImmutableList}s of {@link VersionedEntity} records where the inner
* lists each consist of one transaction. Upon completion the {@code inputStream} is closed.
*
* <p>The returned list may be empty, since CommitLogs are written at fixed intervals regardless
* if actual changes exist. Each sublist, however, will not be empty.
*
* <p>A CommitLog file starts with a {@link CommitLogCheckpoint}, followed by (repeated)
* subsequences of [{@link CommitLogManifest}, [{@link CommitLogMutation}] ...]. Each subsequence
* represents the changes in one transaction. The {@code CommitLogManifest} contains deleted
* entity keys, whereas each {@code CommitLogMutation} contains one whole entity.
*/
public static ImmutableList<ImmutableList<VersionedEntity>> loadEntitiesByTransaction(
InputStream inputStream) {
try (AppEngineEnvironment appEngineEnvironment = new AppEngineEnvironment();
InputStream input = new BufferedInputStream(inputStream)) {
Iterator<ImmutableObject> commitLogs = createDeserializingIterator(input);
checkState(commitLogs.hasNext());
checkState(commitLogs.next() instanceof CommitLogCheckpoint);
ImmutableList.Builder<ImmutableList<VersionedEntity>> resultBuilder =
new ImmutableList.Builder<>();
ImmutableList.Builder<VersionedEntity> currentTransactionBuilder =
new ImmutableList.Builder<>();
while (commitLogs.hasNext()) {
ImmutableObject currentObject = commitLogs.next();
if (currentObject instanceof CommitLogManifest) {
// CommitLogManifest means we are starting a new transaction
addIfNonempty(resultBuilder, currentTransactionBuilder);
currentTransactionBuilder = new ImmutableList.Builder<>();
VersionedEntity.fromManifest((CommitLogManifest) currentObject)
.forEach(currentTransactionBuilder::add);
} else if (currentObject instanceof CommitLogMutation) {
currentTransactionBuilder.add(
VersionedEntity.fromMutation((CommitLogMutation) currentObject));
} else {
throw new IllegalStateException(
String.format("Unknown entity type %s in commit logs", currentObject.getClass()));
}
}
// Add the last transaction in (if it's not empty)
addIfNonempty(resultBuilder, currentTransactionBuilder);
return resultBuilder.build();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
/**
* Returns entities in an {@code inputStream} (from a single CommitLog file) as an {@link
* ImmutableList} of {@link VersionedEntity} records. Upon completion the {@code inputStream} is
@@ -57,23 +106,9 @@ public final class CommitLogImports {
* entity keys, whereas each {@code CommitLogMutation} contains one whole entity.
*/
public static ImmutableList<VersionedEntity> loadEntities(InputStream inputStream) {
try (AppEngineEnvironment appEngineEnvironment = new AppEngineEnvironment();
InputStream input = new BufferedInputStream(inputStream)) {
Iterator<ImmutableObject> commitLogs = createDeserializingIterator(input);
checkState(commitLogs.hasNext());
checkState(commitLogs.next() instanceof CommitLogCheckpoint);
return Streams.stream(commitLogs)
.map(
e ->
e instanceof CommitLogManifest
? VersionedEntity.fromManifest((CommitLogManifest) e)
: Stream.of(VersionedEntity.fromMutation((CommitLogMutation) e)))
.flatMap(s -> s)
.collect(ImmutableList.toImmutableList());
} catch (IOException e) {
throw new RuntimeException(e);
}
return loadEntitiesByTransaction(inputStream).stream()
.flatMap(ImmutableList::stream)
.collect(toImmutableList());
}
/** Covenience method that adapts {@link #loadEntities(InputStream)} to a {@link File}. */
@@ -92,4 +127,13 @@ public final class CommitLogImports {
public static ImmutableList<VersionedEntity> loadEntities(ReadableByteChannel channel) {
return loadEntities(Channels.newInputStream(channel));
}
private static void addIfNonempty(
ImmutableList.Builder<ImmutableList<VersionedEntity>> resultBuilder,
ImmutableList.Builder<VersionedEntity> currentTransactionBuilder) {
ImmutableList<VersionedEntity> currentTransaction = currentTransactionBuilder.build();
if (!currentTransaction.isEmpty()) {
resultBuilder.add(currentTransaction);
}
}
}

View File

@@ -122,7 +122,7 @@ class GcsDiffFileLister {
// Reconstruct the sequence of files by traversing backwards from "lastUpperBoundTime" (i.e. the
// last file that we found) and finding its previous file until we either run out of files or
// get to one that preceeds "fromTime".
// get to one that precedes "fromTime".
//
// GCS file listing is eventually consistent, so it's possible that we are missing a file. The
// metadata of a file is sufficient to identify the preceding file, so if we start from the

View File

@@ -0,0 +1,188 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import static google.registry.backup.ExportCommitLogDiffAction.DIFF_FILE_PREFIX;
import static google.registry.model.ofy.EntityWritePriorities.getEntityPriority;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static javax.servlet.http.HttpServletResponse.SC_NO_CONTENT;
import static org.joda.time.Duration.standardHours;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.Key;
import com.google.appengine.tools.cloudstorage.GcsFileMetadata;
import com.google.appengine.tools.cloudstorage.GcsService;
import com.google.common.collect.ImmutableList;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryConfig;
import google.registry.model.server.Lock;
import google.registry.model.translators.VKeyTranslatorFactory;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.schema.replay.DatastoreEntity;
import google.registry.schema.replay.DatastoreOnlyEntity;
import google.registry.schema.replay.NonReplicatedEntity;
import google.registry.schema.replay.SqlReplayCheckpoint;
import google.registry.util.RequestStatusChecker;
import java.io.IOException;
import java.io.InputStream;
import java.nio.channels.Channels;
import java.util.Optional;
import javax.inject.Inject;
import javax.servlet.http.HttpServletResponse;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/** Action that replays commit logs to Cloud SQL to keep it up to date. */
@Action(
service = Action.Service.BACKEND,
path = ReplayCommitLogsToSqlAction.PATH,
method = Action.Method.POST,
automaticallyPrintOk = true,
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
public class ReplayCommitLogsToSqlAction implements Runnable {
static final String PATH = "/_dr/task/replayCommitLogsToSql";
private static final int BLOCK_SIZE =
1024 * 1024; // Buffer 1mb at a time, for no particular reason.
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final Duration LEASE_LENGTH = standardHours(1);
@Inject GcsService gcsService;
@Inject Response response;
@Inject RequestStatusChecker requestStatusChecker;
@Inject GcsDiffFileLister diffLister;
@Inject
ReplayCommitLogsToSqlAction() {}
@Override
public void run() {
if (!RegistryConfig.getCloudSqlReplayCommitLogs()) {
String message = "ReplayCommitLogsToSqlAction was called but disabled in the config.";
logger.atWarning().log(message);
// App Engine will retry on any non-2xx status code, which we don't want in this case.
response.setStatus(SC_NO_CONTENT);
response.setPayload(message);
return;
}
Optional<Lock> lock =
Lock.acquire(
this.getClass().getSimpleName(), null, LEASE_LENGTH, requestStatusChecker, false);
if (!lock.isPresent()) {
String message = "Can't acquire SQL commit log replay lock, aborting.";
logger.atSevere().log(message);
// App Engine will retry on any non-2xx status code, which we don't want in this case.
// Let the next run after the next export happen naturally.
response.setStatus(SC_NO_CONTENT);
response.setPayload(message);
return;
}
try {
replayFiles();
response.setStatus(HttpServletResponse.SC_OK);
logger.atInfo().log("ReplayCommitLogsToSqlAction completed successfully.");
} finally {
lock.ifPresent(Lock::release);
}
}
private void replayFiles() {
// Start at the first millisecond we haven't seen yet
DateTime fromTime = jpaTm().transact(() -> SqlReplayCheckpoint.get().plusMillis(1));
// If there's an inconsistent file set, this will throw IllegalStateException and the job
// will try later -- this is likely because an export hasn't finished yet.
ImmutableList<GcsFileMetadata> commitLogFiles =
diffLister.listDiffFiles(fromTime, /* current time */ null);
for (GcsFileMetadata metadata : commitLogFiles) {
// One transaction per GCS file
jpaTm().transact(() -> processFile(metadata));
}
logger.atInfo().log("Replayed %d commit log files to SQL successfully.", commitLogFiles.size());
}
private void processFile(GcsFileMetadata metadata) {
try (InputStream input =
Channels.newInputStream(
gcsService.openPrefetchingReadChannel(metadata.getFilename(), 0, BLOCK_SIZE))) {
// Load and process the Datastore transactions one at a time
ImmutableList<ImmutableList<VersionedEntity>> allTransactions =
CommitLogImports.loadEntitiesByTransaction(input);
allTransactions.forEach(this::replayTransaction);
// if we succeeded, set the last-seen time
DateTime checkpoint =
DateTime.parse(
metadata.getFilename().getObjectName().substring(DIFF_FILE_PREFIX.length()));
SqlReplayCheckpoint.set(checkpoint);
logger.atInfo().log("Replayed %d transactions from commit log file.", allTransactions.size());
} catch (IOException e) {
throw new RuntimeException(e);
}
}
private void replayTransaction(ImmutableList<VersionedEntity> transaction) {
transaction.stream()
.sorted(ReplayCommitLogsToSqlAction::compareByWeight)
.forEach(
versionedEntity ->
versionedEntity
.getEntity()
.ifPresentOrElse(
this::handleEntityPut, () -> handleEntityDelete(versionedEntity)));
}
private void handleEntityPut(Entity entity) {
Object ofyPojo = ofy().toPojo(entity);
if (ofyPojo instanceof DatastoreEntity) {
DatastoreEntity datastoreEntity = (DatastoreEntity) ofyPojo;
datastoreEntity.toSqlEntity().ifPresent(jpaTm()::put);
} else {
// this should never happen, but we shouldn't fail on it
logger.atSevere().log(
"%s does not implement DatastoreEntity, which is necessary for SQL replay.",
ofyPojo.getClass());
}
}
private void handleEntityDelete(VersionedEntity entityToDelete) {
Key key = entityToDelete.key();
VKey<?> entityVKey;
try {
entityVKey = VKeyTranslatorFactory.createVKey(key);
} catch (RuntimeException e) {
// This means that the key wasn't convertible to VKey through the standard methods or via
// a createVKey method. This means that the object isn't persisted in SQL so we ignore it.
logger.atInfo().log(
"Skipping SQL delete for kind %s since it is not convertible.", key.getKind());
return;
}
Class<?> entityClass = entityVKey.getKind();
// Delete the key iff the class represents a JPA entity that is replicated
if (!NonReplicatedEntity.class.isAssignableFrom(entityClass)
&& !DatastoreOnlyEntity.class.isAssignableFrom(entityClass)
&& entityClass.getAnnotation(javax.persistence.Entity.class) != null) {
jpaTm().delete(entityVKey);
}
}
private static int compareByWeight(VersionedEntity a, VersionedEntity b) {
return getEntityPriority(a.key().getKind(), !a.getEntity().isPresent())
- getEntityPriority(b.key().getKind(), !b.getEntity().isPresent());
}
}

View File

@@ -345,7 +345,7 @@ public class DeleteContactsAndHostsAction implements Runnable {
String resourceClientId = resource.getPersistedCurrentSponsorClientId();
if (resource instanceof HostResource && ((HostResource) resource).isSubordinate()) {
resourceClientId =
tm().load(((HostResource) resource).getSuperordinateDomain())
tm().loadByKey(((HostResource) resource).getSuperordinateDomain())
.cloneProjectedAtTime(now)
.getCurrentSponsorClientId();
}
@@ -465,7 +465,7 @@ public class DeleteContactsAndHostsAction implements Runnable {
if (host.isSubordinate()) {
dnsQueue.addHostRefreshTask(host.getHostName());
tm().put(
tm().load(host.getSuperordinateDomain())
tm().loadByKey(host.getSuperordinateDomain())
.asBuilder()
.removeSubordinateHost(host.getHostName())
.build());

View File

@@ -14,6 +14,7 @@
package google.registry.batch;
import static google.registry.mapreduce.MapreduceRunner.PARAM_FAST;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
@@ -24,6 +25,7 @@ import google.registry.mapreduce.MapreduceRunner;
import google.registry.mapreduce.inputs.EppResourceInputs;
import google.registry.model.EppResource;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import javax.inject.Inject;
@@ -39,6 +41,14 @@ import javax.inject.Inject;
* <p>Because there are no auth settings in the {@link Action} annotation, this command can only be
* run internally, or by pretending to be internal by setting the X-AppEngine-QueueName header,
* which only admin users can do.
*
* <p>If the <code>?fast=true</code> querystring parameter is passed, then entities that are not
* changed by {@link EppResource#cloneProjectedAtTime} will not be re-saved. This helps prevent
* mutation load on the DB and has the beneficial side effect of writing out smaller commit logs.
* Note that this does NOT pick up mutations caused by migrations using the {@link
* com.googlecode.objectify.annotation.OnLoad} annotation, so if you are running a one-off schema
* migration, do not use fast mode. Fast mode defaults to false for this reason, but is used by the
* monthly invocation of the mapreduce.
*/
@Action(
service = Action.Service.BACKEND,
@@ -48,15 +58,31 @@ public class ResaveAllEppResourcesAction implements Runnable {
@Inject MapreduceRunner mrRunner;
@Inject Response response;
@Inject ResaveAllEppResourcesAction() {}
@Inject
@Parameter(PARAM_FAST)
boolean isFast;
@Inject
ResaveAllEppResourcesAction() {}
/**
* The number of shards to run the map-only mapreduce on.
*
* <p>This is less than the default of 100 because we only run this action monthly and can afford
* it being slower, but we don't want to write out lots of large commit logs in a short period of
* time because they make the Cloud SQL migration tougher.
*/
private static final int NUM_SHARDS = 10;
@Override
public void run() {
mrRunner
.setJobName("Re-save all EPP resources")
.setModuleName("backend")
.setDefaultMapShards(NUM_SHARDS)
.runMapOnly(
new ResaveAllEppResourcesActionMapper(),
new ResaveAllEppResourcesActionMapper(isFast),
ImmutableList.of(EppResourceInputs.createKeyInput(EppResource.class)))
.sendLinkToMapreduceConsole(response);
}
@@ -66,23 +92,33 @@ public class ResaveAllEppResourcesAction implements Runnable {
extends Mapper<Key<EppResource>, Void, Void> {
private static final long serialVersionUID = -7721628665138087001L;
public ResaveAllEppResourcesActionMapper() {}
private final boolean isFast;
ResaveAllEppResourcesActionMapper(boolean isFast) {
this.isFast = isFast;
}
@Override
public final void map(final Key<EppResource> resourceKey) {
tm()
.transact(
() -> {
EppResource projectedResource =
ofy()
.load()
.key(resourceKey)
.now()
.cloneProjectedAtTime(tm().getTransactionTime());
ofy().save().entity(projectedResource).now();
});
getContext().incrementCounter(String.format("%s entities re-saved", resourceKey.getKind()));
boolean resaved =
tm().transact(
() -> {
EppResource originalResource = ofy().load().key(resourceKey).now();
EppResource projectedResource =
originalResource.cloneProjectedAtTime(tm().getTransactionTime());
if (isFast && originalResource.equals(projectedResource)) {
return false;
} else {
ofy().save().entity(projectedResource).now();
return true;
}
});
getContext()
.incrementCounter(
String.format(
"%s entities %s",
resourceKey.getKind(), resaved ? "re-saved" : "with no changes skipped"));
}
}
}

View File

@@ -30,7 +30,9 @@ import google.registry.keyring.kms.KmsModule;
import google.registry.persistence.PersistenceModule;
import google.registry.persistence.PersistenceModule.JdbcJpaTm;
import google.registry.persistence.PersistenceModule.SocketFactoryJpaTm;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.privileges.secretmanager.SecretManagerModule;
import google.registry.util.UtilsModule;
import java.io.BufferedReader;
import java.io.IOException;
@@ -56,6 +58,7 @@ public class BeamJpaModule {
@Nullable private final String sqlAccessInfoFile;
@Nullable private final String cloudKmsProjectId;
@Nullable private final TransactionIsolationLevel isolationOverride;
/**
* Constructs a new instance of {@link BeamJpaModule}.
@@ -72,10 +75,20 @@ public class BeamJpaModule {
* real encrypted file on GCS as returned by {@link
* BackupPaths#getCloudSQLCredentialFilePatterns} or an unencrypted file on local filesystem
* with credentials to a test database.
* @param cloudKmsProjectId the GCP project where the credential decryption key can be found
* @param isolationOverride the desired Transaction Isolation level for all JDBC connections
*/
public BeamJpaModule(@Nullable String sqlAccessInfoFile, @Nullable String cloudKmsProjectId) {
public BeamJpaModule(
@Nullable String sqlAccessInfoFile,
@Nullable String cloudKmsProjectId,
@Nullable TransactionIsolationLevel isolationOverride) {
this.sqlAccessInfoFile = sqlAccessInfoFile;
this.cloudKmsProjectId = cloudKmsProjectId;
this.isolationOverride = isolationOverride;
}
public BeamJpaModule(@Nullable String sqlAccessInfoFile, @Nullable String cloudKmsProjectId) {
this(sqlAccessInfoFile, cloudKmsProjectId, null);
}
/** Returns true if the credential file is on GCS (and therefore expected to be encrypted). */
@@ -153,6 +166,13 @@ public class BeamJpaModule {
return "nomulus-tool-keyring";
}
@Provides
@Config("beamIsolationOverride")
@Nullable
TransactionIsolationLevel providesIsolationOverride() {
return isolationOverride;
}
@Provides
@Config("beamHibernateHikariMaximumPoolSize")
static int getBeamHibernateHikariMaximumPoolSize() {
@@ -168,6 +188,7 @@ public class BeamJpaModule {
BeamJpaModule.class,
KmsModule.class,
PersistenceModule.class,
SecretManagerModule.class,
UtilsModule.class
})
public interface JpaTransactionManagerComponent {

View File

@@ -26,17 +26,15 @@ final class DomainBaseUtil {
private DomainBaseUtil() {}
/**
* Removes {@link google.registry.model.billing.BillingEvent.Recurring}, {@link
* google.registry.model.poll.PollMessage PollMessages} and {@link
* google.registry.model.host.HostResource name servers} from a Datastore {@link Entity} that
* represents an Ofy {@link google.registry.model.domain.DomainBase}. This breaks the cycle of
* foreign key constraints between these entity kinds, allowing {@code DomainBases} to be inserted
* into the SQL database. See {@link InitSqlPipeline} for a use case, where the full {@code
* DomainBases} are written again during the last stage of the pipeline.
* Removes properties that contain foreign keys from a Datastore {@link Entity} that represents an
* Ofy {@link google.registry.model.domain.DomainBase}. This breaks the cycle of foreign key
* constraints between entity kinds, allowing {@code DomainBases} to be inserted into the SQL
* database. See {@link InitSqlPipeline} for a use case, where the full {@code DomainBases} are
* written again during the last stage of the pipeline.
*
* <p>The returned object may be in bad state. Specifically, {@link
* google.registry.model.eppcommon.StatusValue#INACTIVE} is not added after name servers are
* removed. This only impacts tests.
* removed. This only impacts tests that manipulate Datastore entities directly.
*
* <p>This operation is performed on an Datastore {@link Entity} instead of Ofy Java object
* because Objectify requires access to a Datastore service when converting an Ofy object to a
@@ -70,6 +68,9 @@ final class DomainBaseUtil {
domainBase.getProperties().keySet().stream()
.filter(s -> s.startsWith("transferData."))
.forEach(s -> clone.removeProperty(s));
domainBase.getProperties().keySet().stream()
.filter(s -> s.startsWith("gracePeriods."))
.forEach(s -> clone.removeProperty(s));
return clone;
}
}

View File

@@ -34,6 +34,7 @@ import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarContact;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.HistoryEntry;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.persistence.transaction.JpaTransactionManager;
import java.io.Serializable;
import java.util.Collection;
@@ -77,14 +78,22 @@ import org.joda.time.DateTime;
* HistoryEntry}.
* <li>{@link BillingEvent.OneTime}: references {@code Registrar}, {@code DomainBase}, {@code
* BillingEvent.Recurring}, {@code HistoryEntry} and {@code AllocationToken}.
* <li>{@link BillingEvent.Modification}: SQL model TBD. Will reference {@code Registrar}, {@code
* DomainBase} and {@code BillingEvent.OneTime}.
* <li>{@link BillingEvent.Cancellation}: references {@code Registrar}, {@code DomainBase}, {@code
* BillingEvent.Recurring}, {@code BillingEvent.OneTime}, and {@code HistoryEntry}.
* <li>{@link PollMessage}: references {@code Registrar}, {@code DomainBase}, {@code
* ContactResource}, {@code HostResource}, and {@code HistoryEntry}.
* <li>{@link DomainBase}, original copy from Datastore.
* </ol>
*
* <p>This pipeline expects that the source Datastore has at least one entity in each of the types
* above. This assumption allows us to construct a simpler pipeline graph that can be visually
* examined, and is true in all intended use cases. However, tests must not violate this assumption
* when setting up data, otherwise they may run into foreign key constraint violations. The reason
* is that this pipeline uses the {@link Wait} transform to order the persistence by entity type.
* However, the wait is skipped if the target type has no data, resulting in subsequent entity types
* starting prematurely. E.g., if a Datastore has no {@code RegistrarContact} entities, the pipeline
* may start writing {@code DomainBase} entities before all {@code Registry}, {@code Registrar} and
* {@code ContactResource} entities have been persisted.
*/
public class InitSqlPipeline implements Serializable {
@@ -92,26 +101,24 @@ public class InitSqlPipeline implements Serializable {
* Datastore kinds to be written to the SQL database before the cleansed version of {@link
* DomainBase}.
*/
// TODO(weiminyu): include Registry.class when it is modeled in JPA.
private static final ImmutableList<Class<?>> PHASE_ONE_ORDERED =
ImmutableList.of(Registrar.class, ContactResource.class);
ImmutableList.of(
Registry.class, Registrar.class, ContactResource.class, RegistrarContact.class);
/**
* Datastore kinds to be written to the SQL database after the cleansed version of {@link
* DomainBase}.
*
* <p>The following entities are missing from the list:
*
* <ul>
* <li>Those not modeled in JPA yet, e.g., {@code BillingEvent.Modification}.
* <li>Those waiting for sanitation, e.g., {@code HistoryEntry}, which would have duplicate keys
* after converting to SQL model.
* <li>Those that have foreign key constraints on the above.
* </ul>
*/
// TODO(weiminyu): add more entities when available.
private static final ImmutableList<Class<?>> PHASE_TWO_ORDERED =
ImmutableList.of(HostResource.class);
ImmutableList.of(
HostResource.class,
HistoryEntry.class,
AllocationToken.class,
BillingEvent.Recurring.class,
BillingEvent.OneTime.class,
BillingEvent.Cancellation.class,
PollMessage.class,
DomainBase.class);
private final InitSqlPipelineOptions options;
@@ -227,7 +234,11 @@ public class InitSqlPipeline implements Serializable {
transformId,
options.getMaxConcurrentSqlWriters(),
options.getSqlWriteBatchSize(),
new JpaSupplierFactory(credentialFileUrl, options.getCloudKmsProjectId(), jpaGetter)));
new JpaSupplierFactory(
credentialFileUrl,
options.getCloudKmsProjectId(),
jpaGetter,
TransactionIsolationLevel.TRANSACTION_READ_UNCOMMITTED)));
}
private static ImmutableList<String> toKindStrings(Collection<Class<?>> entityClasses) {

View File

@@ -16,6 +16,7 @@ package google.registry.beam.initsql;
import google.registry.beam.initsql.BeamJpaModule.JpaTransactionManagerComponent;
import google.registry.beam.initsql.Transforms.SerializableSupplier;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.persistence.transaction.JpaTransactionManager;
import javax.annotation.Nullable;
import org.apache.beam.sdk.transforms.SerializableFunction;
@@ -28,21 +29,32 @@ public class JpaSupplierFactory implements SerializableSupplier<JpaTransactionMa
@Nullable private final String cloudKmsProjectId;
private final SerializableFunction<JpaTransactionManagerComponent, JpaTransactionManager>
jpaGetter;
@Nullable private final TransactionIsolationLevel isolationLevelOverride;
public JpaSupplierFactory(
String credentialFileUrl,
@Nullable String cloudKmsProjectId,
SerializableFunction<JpaTransactionManagerComponent, JpaTransactionManager> jpaGetter) {
this(credentialFileUrl, cloudKmsProjectId, jpaGetter, null);
}
public JpaSupplierFactory(
String credentialFileUrl,
@Nullable String cloudKmsProjectId,
SerializableFunction<JpaTransactionManagerComponent, JpaTransactionManager> jpaGetter,
@Nullable TransactionIsolationLevel isolationLevelOverride) {
this.credentialFileUrl = credentialFileUrl;
this.cloudKmsProjectId = cloudKmsProjectId;
this.jpaGetter = jpaGetter;
this.isolationLevelOverride = isolationLevelOverride;
}
@Override
public JpaTransactionManager get() {
return jpaGetter.apply(
DaggerBeamJpaModule_JpaTransactionManagerComponent.builder()
.beamJpaModule(new BeamJpaModule(credentialFileUrl, cloudKmsProjectId))
.beamJpaModule(
new BeamJpaModule(credentialFileUrl, cloudKmsProjectId, isolationLevelOverride))
.build());
}
}

View File

@@ -17,10 +17,9 @@ package google.registry.beam.initsql;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.base.Throwables.throwIfUnchecked;
import static google.registry.beam.initsql.BackupPaths.getCommitLogTimestamp;
import static google.registry.beam.initsql.BackupPaths.getExportFilePatterns;
import static google.registry.persistence.JpaRetries.isFailedTxnRetriable;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.setJpaTm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
@@ -37,28 +36,34 @@ import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.Streams;
import com.googlecode.objectify.Key;
import google.registry.backup.AppEngineEnvironment;
import google.registry.backup.CommitLogImports;
import google.registry.backup.VersionedEntity;
import google.registry.model.domain.DomainBase;
import google.registry.model.ofy.ObjectifyService;
import google.registry.model.ofy.Ofy;
import google.registry.model.reporting.HistoryEntry;
import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.schema.replay.DatastoreAndSqlEntity;
import google.registry.schema.replay.SqlEntity;
import google.registry.tools.LevelDbLogReader;
import google.registry.util.SystemSleeper;
import java.io.Serializable;
import java.util.Collection;
import java.util.Iterator;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.concurrent.ThreadLocalRandom;
import java.util.function.Supplier;
import javax.annotation.Nullable;
import org.apache.beam.sdk.coders.StringUtf8Coder;
import org.apache.beam.sdk.io.Compression;
import org.apache.beam.sdk.io.FileIO;
import org.apache.beam.sdk.io.FileIO.ReadableFile;
import org.apache.beam.sdk.io.fs.EmptyMatchTreatment;
import org.apache.beam.sdk.io.fs.MatchResult.Metadata;
import org.apache.beam.sdk.metrics.Counter;
import org.apache.beam.sdk.metrics.Metrics;
import org.apache.beam.sdk.transforms.Create;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.Flatten;
@@ -68,6 +73,7 @@ import org.apache.beam.sdk.transforms.MapElements;
import org.apache.beam.sdk.transforms.PTransform;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.transforms.ProcessFunction;
import org.apache.beam.sdk.transforms.SerializableFunction;
import org.apache.beam.sdk.values.KV;
import org.apache.beam.sdk.values.PBegin;
import org.apache.beam.sdk.values.PCollection;
@@ -77,7 +83,6 @@ import org.apache.beam.sdk.values.TupleTag;
import org.apache.beam.sdk.values.TupleTagList;
import org.apache.beam.sdk.values.TypeDescriptor;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/**
* {@link PTransform Pipeline transforms} used in pipelines that load from both Datastore export
@@ -264,9 +269,9 @@ public final class Transforms {
}
/**
* Returns a {@link PTransform} that writes a {@link PCollection} of entities to a SQL database.
* and outputs an empty {@code PCollection<Void>}. This allows other operations to {@link
* org.apache.beam.sdk.transforms.Wait wait} for the completion of this transform.
* Returns a {@link PTransform} that writes a {@link PCollection} of {@link VersionedEntity}s to a
* SQL database. and outputs an empty {@code PCollection<Void>}. This allows other operations to
* {@link org.apache.beam.sdk.transforms.Wait wait} for the completion of this transform.
*
* <p>Errors are handled according to the pipeline runner's default policy. As part of a one-time
* job, we will not add features unless proven necessary.
@@ -282,20 +287,96 @@ public final class Transforms {
int maxWriters,
int batchSize,
SerializableSupplier<JpaTransactionManager> jpaSupplier) {
return new PTransform<PCollection<VersionedEntity>, PCollection<Void>>() {
return writeToSql(
transformId,
maxWriters,
batchSize,
jpaSupplier,
Transforms::convertVersionedEntityToSqlEntity,
TypeDescriptor.of(VersionedEntity.class));
}
/**
* Returns a {@link PTransform} that writes a {@link PCollection} of entities to a SQL database.
* and outputs an empty {@code PCollection<Void>}. This allows other operations to {@link
* org.apache.beam.sdk.transforms.Wait wait} for the completion of this transform.
*
* <p>The converter and type descriptor are generics so that we can convert any type of entity to
* an object to be placed in SQL.
*
* <p>Errors are handled according to the pipeline runner's default policy. As part of a one-time
* job, we will not add features unless proven necessary.
*
* @param transformId a unique ID for an instance of the returned transform
* @param maxWriters the max number of concurrent writes to SQL, which also determines the max
* number of connection pools created
* @param batchSize the number of entities to write in each operation
* @param jpaSupplier supplier of a {@link JpaTransactionManager}
* @param jpaConverter the function that converts the input object to a JPA entity
* @param objectDescriptor the type descriptor of the input object
*/
public static <T> PTransform<PCollection<T>, PCollection<Void>> writeToSql(
String transformId,
int maxWriters,
int batchSize,
SerializableSupplier<JpaTransactionManager> jpaSupplier,
SerializableFunction<T, Object> jpaConverter,
TypeDescriptor<T> objectDescriptor) {
return new PTransform<PCollection<T>, PCollection<Void>>() {
@Override
public PCollection<Void> expand(PCollection<VersionedEntity> input) {
public PCollection<Void> expand(PCollection<T> input) {
return input
.apply(
"Shard data for " + transformId,
MapElements.into(kvs(integers(), TypeDescriptor.of(VersionedEntity.class)))
MapElements.into(kvs(integers(), objectDescriptor))
.via(ve -> KV.of(ThreadLocalRandom.current().nextInt(maxWriters), ve)))
.apply("Batch output by shard " + transformId, GroupIntoBatches.ofSize(batchSize))
.apply("Write in batch for " + transformId, ParDo.of(new SqlBatchWriter(jpaSupplier)));
.apply(
"Write in batch for " + transformId,
ParDo.of(new SqlBatchWriter<T>(transformId, jpaSupplier, jpaConverter)));
}
};
}
private static Key toOfyKey(Object ofyEntity) {
return Key.create(ofyEntity);
}
private static boolean isMigratable(Entity entity) {
if (entity.getKind().equals("HistoryEntry")) {
// DOMAIN_APPLICATION_CREATE is deprecated type and should not be migrated.
// The Enum name DOMAIN_APPLICATION_CREATE no longer exists in Java and cannot
// be deserialized.
return !Objects.equals(entity.getProperty("type"), "DOMAIN_APPLICATION_CREATE");
}
return true;
}
private static SqlEntity toSqlEntity(Object ofyEntity) {
if (ofyEntity instanceof HistoryEntry) {
HistoryEntry ofyHistory = (HistoryEntry) ofyEntity;
return (SqlEntity) ofyHistory.toChildHistoryEntity();
}
return ((DatastoreAndSqlEntity) ofyEntity).toSqlEntity().get();
}
/**
* Converts a {@link VersionedEntity} to an JPA entity for persistence.
*
* @return An object to be persisted to SQL, or null if the input is not to be migrated. (Not
* using Optional in return because as a one-use method, we do not want to invest the effort
* to make Optional work with BEAM)
*/
@Nullable
private static Object convertVersionedEntityToSqlEntity(VersionedEntity dsEntity) {
return dsEntity
.getEntity()
.filter(Transforms::isMigratable)
.map(e -> ofy().toPojo(e))
.map(Transforms::toSqlEntity)
.orElse(null);
}
/** Interface for serializable {@link Supplier suppliers}. */
public interface SerializableSupplier<T> extends Supplier<T>, Serializable {}
@@ -385,27 +466,29 @@ public final class Transforms {
* to hold the {@code JpaTransactionManager} instance, we must ensure that JpaTransactionManager
* is not changed or torn down while being used by some instance.
*/
private static class SqlBatchWriter extends DoFn<KV<Integer, Iterable<VersionedEntity>>, Void> {
private static class SqlBatchWriter<T> extends DoFn<KV<Integer, Iterable<T>>, Void> {
private static int instanceCount = 0;
private static JpaTransactionManager originalJpa;
private Counter counter;
private final SerializableSupplier<JpaTransactionManager> jpaSupplier;
private final SerializableFunction<T, Object> jpaConverter;
private transient Ofy ofy;
private transient SystemSleeper sleeper;
SqlBatchWriter(SerializableSupplier<JpaTransactionManager> jpaSupplier) {
SqlBatchWriter(
String type,
SerializableSupplier<JpaTransactionManager> jpaSupplier,
SerializableFunction<T, Object> jpaConverter) {
counter = Metrics.counter("SQL_WRITE", type);
this.jpaSupplier = jpaSupplier;
this.jpaConverter = jpaConverter;
}
@Setup
public void setup() {
sleeper = new SystemSleeper();
try (AppEngineEnvironment env = new AppEngineEnvironment()) {
ObjectifyService.initOfy();
ofy = ObjectifyService.ofy();
}
synchronized (SqlBatchWriter.class) {
@@ -429,38 +512,34 @@ public final class Transforms {
}
@ProcessElement
public void processElement(@Element KV<Integer, Iterable<VersionedEntity>> kv) {
public void processElement(@Element KV<Integer, Iterable<T>> kv) {
try (AppEngineEnvironment env = new AppEngineEnvironment()) {
ImmutableList<Object> ofyEntities =
Streams.stream(kv.getValue())
.map(VersionedEntity::getEntity)
.map(Optional::get)
.map(ofy::toPojo)
.map(this.jpaConverter::apply)
// TODO(b/177340730): post migration delete the line below.
.filter(Objects::nonNull)
.collect(ImmutableList.toImmutableList());
retry(() -> jpaTm().transact(() -> jpaTm().putAll(ofyEntities)));
try {
jpaTm().transact(() -> jpaTm().putAll(ofyEntities));
counter.inc(ofyEntities.size());
} catch (RuntimeException e) {
processSingly(ofyEntities);
}
}
}
// TODO(b/160632289): Enhance Retrier and use it here.
private void retry(Runnable runnable) {
int maxAttempts = 5;
int initialDelayMillis = 100;
double jitterRatio = 0.2;
for (int attempt = 0; attempt < maxAttempts; attempt++) {
/**
* Writes entities in a failed batch one by one to identify the first bad entity and throws a
* {@link RuntimeException} on it.
*/
private void processSingly(ImmutableList<Object> ofyEntities) {
for (Object ofyEntity : ofyEntities) {
try {
runnable.run();
return;
} catch (Throwable throwable) {
if (!isFailedTxnRetriable(throwable)) {
throwIfUnchecked(throwable);
throw new RuntimeException(throwable);
}
int sleepMillis = (1 << attempt) * initialDelayMillis;
int jitter =
ThreadLocalRandom.current().nextInt((int) (sleepMillis * jitterRatio))
- (int) (sleepMillis * jitterRatio / 2);
sleeper.sleepUninterruptibly(Duration.millis(sleepMillis + jitter));
jpaTm().transact(() -> jpaTm().put(ofyEntity));
counter.inc();
} catch (RuntimeException e) {
throw new RuntimeException(toOfyKey(ofyEntity).toString(), e);
}
}
}

View File

@@ -20,7 +20,7 @@ import static google.registry.beam.BeamUtils.getQueryFromFile;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.auto.value.AutoValue;
import com.google.common.collect.ImmutableSet;
import google.registry.backup.AppEngineEnvironment;
import google.registry.beam.initsql.Transforms;
import google.registry.beam.initsql.Transforms.SerializableSupplier;
import google.registry.beam.spec11.SafeBrowsingTransforms.EvaluateSafeBrowsingFn;
import google.registry.config.CredentialModule.LocalCredential;
@@ -43,7 +43,6 @@ import org.apache.beam.sdk.options.Description;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.apache.beam.sdk.options.ValueProvider;
import org.apache.beam.sdk.options.ValueProvider.NestedValueProvider;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.GroupByKey;
import org.apache.beam.sdk.transforms.MapElements;
import org.apache.beam.sdk.transforms.ParDo;
@@ -191,34 +190,27 @@ public class Spec11Pipeline implements Serializable {
PCollection<Subdomain> domains,
EvaluateSafeBrowsingFn evaluateSafeBrowsingFn,
ValueProvider<String> dateProvider) {
PCollection<KV<Subdomain, ThreatMatch>> subdomainsSql =
domains.apply("Run through SafeBrowsing API", ParDo.of(evaluateSafeBrowsingFn));
/* Store ThreatMatch objects in SQL. */
TypeDescriptor<KV<Subdomain, ThreatMatch>> descriptor =
new TypeDescriptor<KV<Subdomain, ThreatMatch>>() {};
subdomainsSql.apply(
ParDo.of(
new DoFn<KV<Subdomain, ThreatMatch>, Void>() {
@ProcessElement
public void processElement(ProcessContext context) {
// create the Spec11ThreatMatch from Subdomain and ThreatMatch
try (AppEngineEnvironment env = new AppEngineEnvironment()) {
Subdomain subdomain = context.element().getKey();
Spec11ThreatMatch threatMatch =
new Spec11ThreatMatch.Builder()
.setThreatTypes(
ImmutableSet.of(
ThreatType.valueOf(context.element().getValue().threatType())))
.setCheckDate(
LocalDate.parse(dateProvider.get(), ISODateTimeFormat.date()))
.setDomainName(subdomain.domainName())
.setDomainRepoId(subdomain.domainRepoId())
.setRegistrarId(subdomain.registrarId())
.build();
JpaTransactionManager jpaTransactionManager = jpaSupplierFactory.get();
jpaTransactionManager.transact(() -> jpaTransactionManager.insert(threatMatch));
}
}
}));
Transforms.writeToSql(
"Spec11ThreatMatch",
4,
4,
jpaSupplierFactory,
(kv) -> {
Subdomain subdomain = kv.getKey();
return new Spec11ThreatMatch.Builder()
.setThreatTypes(ImmutableSet.of(ThreatType.valueOf(kv.getValue().threatType())))
.setCheckDate(LocalDate.parse(dateProvider.get(), ISODateTimeFormat.date()))
.setDomainName(subdomain.domainName())
.setDomainRepoId(subdomain.domainRepoId())
.setRegistrarId(subdomain.registrarId())
.build();
},
descriptor));
/* Store ThreatMatch objects in JSON. */
PCollection<KV<Subdomain, ThreatMatch>> subdomainsJson =

View File

@@ -415,6 +415,14 @@ public final class RegistryConfig {
return config.cloudSql.instanceConnectionName;
}
@Provides
@Config("cloudSqlDbInstanceName")
public static String providesCloudSqlDbInstance(RegistryConfigSettings config) {
// Format of instanceConnectionName: project-id:region:instance-name
int lastColonIndex = config.cloudSql.instanceConnectionName.lastIndexOf(':');
return config.cloudSql.instanceConnectionName.substring(lastColonIndex + 1);
}
@Provides
@Config("cloudDnsRootUrl")
public static Optional<String> getCloudDnsRootUrl(RegistryConfigSettings config) {
@@ -1377,6 +1385,12 @@ public final class RegistryConfig {
public static int provideMinimumRsaKeyLength(RegistryConfigSettings config) {
return config.sslCertificateValidation.minimumRsaKeyLength;
}
@Provides
@Config("allowedEcdsaCurves")
public static ImmutableSet<String> provideAllowedEcdsaCurves(RegistryConfigSettings config) {
return ImmutableSet.copyOf(config.sslCertificateValidation.allowedEcdsaCurves);
}
}
/** Returns the App Engine project ID, which is based off the environment name. */
@@ -1586,6 +1600,22 @@ public final class RegistryConfig {
CONFIG_SETTINGS.get().cloudSql.replicateTransactions = replicateTransactions;
}
/**
* Returns whether or not to replay commit logs to the SQL database after export to GCS.
*
* <p>If true, we will trigger the {@link google.registry.backup.ReplayCommitLogsToSqlAction}
* after the {@link google.registry.backup.ExportCommitLogDiffAction} to load the commit logs and
* replay them to SQL.
*/
public static boolean getCloudSqlReplayCommitLogs() {
return CONFIG_SETTINGS.get().cloudSql.replayCommitLogs;
}
@VisibleForTesting
public static void overrideCloudSqlReplayCommitLogs(boolean replayCommitLogs) {
CONFIG_SETTINGS.get().cloudSql.replayCommitLogs = replayCommitLogs;
}
/** Returns the roid suffix to be used for the roids of all contacts and hosts. */
public static String getContactAndHostRoidSuffix() {
return CONFIG_SETTINGS.get().registryPolicy.contactAndHostRoidSuffix;

View File

@@ -16,6 +16,7 @@ package google.registry.config;
import java.util.List;
import java.util.Map;
import java.util.Set;
/** The POJO that YAML config files are deserialized into. */
public class RegistryConfigSettings {
@@ -125,6 +126,7 @@ public class RegistryConfigSettings {
public String username;
public String instanceConnectionName;
public boolean replicateTransactions;
public boolean replayCommitLogs;
}
/** Configuration for Apache Beam (Cloud Dataflow). */
@@ -226,5 +228,6 @@ public class RegistryConfigSettings {
public Map<String, Integer> maxValidityDaysSchedule;
public int expirationWarningDays;
public int minimumRsaKeyLength;
public Set<String> allowedEcdsaCurves;
}
}

View File

@@ -233,6 +233,8 @@ cloudSql:
# Set this to true to replicate cloud SQL transactions to datastore in the
# background.
replicateTransactions: false
# Set this to true to enable replay of commit logs to SQL
replayCommitLogs: false
cloudDns:
# Set both properties to null in Production.
@@ -458,5 +460,9 @@ sslCertificateValidation:
# The number of days before a certificate expires that indicates the
# certificate is nearing expiration and warnings should be sent.
expirationWarningDays: 30
# The minimum number of bits an RSA key must contain
# The minimum number of bits an RSA key must contain.
minimumRsaKeyLength: 2048
# The ECDSA curves that are allowed for public keys.
allowedEcdsaCurves:
- secp256r1
- secp384r1

View File

@@ -80,7 +80,7 @@
</cron>
<cron>
<url><![CDATA[/_dr/task/resaveAllEppResources]]></url>
<url><![CDATA[/_dr/task/resaveAllEppResources?fast=true]]></url>
<description>
This job resaves all our resources, projected in time to "now".
It is needed for "deleteOldCommitLogs" to work correctly.

View File

@@ -208,6 +208,12 @@
<max-concurrent-requests>5</max-concurrent-requests>
</queue>
<!-- Queue for replaying commit logs to SQL during the transition from Datastore -> SQL. -->
<queue>
<name>replay-commit-logs-to-sql</name>
<rate>1/s</rate>
</queue>
<!-- The load[0-9] queues are used for load-testing, and can be safely deleted
in any environment that doesn't require load-testing. -->
<queue>

View File

@@ -103,7 +103,7 @@
</cron>
<cron>
<url><![CDATA[/_dr/task/resaveAllEppResources]]></url>
<url><![CDATA[/_dr/task/resaveAllEppResources?fast=true]]></url>
<description>
This job resaves all our resources, projected in time to "now".
It is needed for "deleteOldCommitLogs" to work correctly.

View File

@@ -21,7 +21,7 @@
</cron>
<cron>
<url><![CDATA[/_dr/task/resaveAllEppResources]]></url>
<url><![CDATA[/_dr/task/resaveAllEppResources?fast=true]]></url>
<description>
This job resaves all our resources, projected in time to "now".
It is needed for "deleteOldCommitLogs" to work correctly.

View File

@@ -87,7 +87,7 @@
</cron>
<cron>
<url><![CDATA[/_dr/task/resaveAllEppResources]]></url>
<url><![CDATA[/_dr/task/resaveAllEppResources?fast=true]]></url>
<description>
This job resaves all our resources, projected in time to "now".
It is needed for "deleteOldCommitLogs" to work correctly.

View File

@@ -74,6 +74,14 @@ public class EppRequestHandler {
&& eppOutput.getResponse().getResult().getCode() == SUCCESS_AND_CLOSE) {
response.setHeader("Epp-Session", "close");
}
// If a login request returns a success, a logged-in header is added to the response to inform
// the proxy that it is no longer necessary to send the full client certificate to the backend
// for this connection.
if (eppOutput.isResponse()
&& eppOutput.getResponse().isLoginResponse()
&& eppOutput.isSuccess()) {
response.setHeader("Logged-In", "true");
}
} catch (Exception e) {
logger.atWarning().withCause(e).log("handleEppCommand general exception");
response.setStatus(SC_BAD_REQUEST);

View File

@@ -15,8 +15,8 @@
package google.registry.flows;
import static com.google.common.collect.Sets.intersection;
import static google.registry.model.EppResourceUtils.getLinkedDomainKeys;
import static google.registry.model.EppResourceUtils.loadByForeignKey;
import static google.registry.model.EppResourceUtils.queryForLinkedDomains;
import static google.registry.model.index.ForeignKeyIndex.loadAndGetKey;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
@@ -94,14 +94,13 @@ public final class ResourceFlowUtils {
* actual reference then we can reliably fail. If we don't find any, we can't
* trust the query and need to do the full mapreduce.
*/
Iterable<Key<DomainBase>> keys =
queryForLinkedDomains(fki.getResourceKey().getOfyKey(), now)
.limit(FAILFAST_CHECK_COUNT)
.keys();
Iterable<VKey<DomainBase>> keys =
getLinkedDomainKeys(fki.getResourceKey(), now, FAILFAST_CHECK_COUNT);
VKey<R> resourceVKey = fki.getResourceKey();
Predicate<DomainBase> predicate =
domain -> getPotentialReferences.apply(domain).contains(resourceVKey);
return ofy().load().keys(keys).values().stream().anyMatch(predicate)
return tm().loadByKeys(keys).values().stream().anyMatch(predicate)
? new ResourceToDeleteIsReferencedException()
: null;
});
@@ -139,7 +138,7 @@ public final class ResourceFlowUtils {
Class<R> clazz, String targetId, DateTime now, String clientId) throws EppException {
VKey<R> key = loadAndGetKey(clazz, targetId, now);
if (key != null) {
R resource = tm().load(key);
R resource = tm().loadByKey(key);
// These are similar exceptions, but we can track them internally as log-based metrics.
if (Objects.equals(clientId, resource.getPersistedCurrentSponsorClientId())) {
throw new ResourceAlreadyExistsForThisClientException(targetId);

View File

@@ -15,9 +15,7 @@
package google.registry.flows;
import static com.google.common.base.MoreObjects.toStringHelper;
import static com.google.common.base.Strings.isNullOrEmpty;
import static google.registry.request.RequestParameters.extractOptionalHeader;
import static google.registry.request.RequestParameters.extractRequiredHeader;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableList;
@@ -27,7 +25,10 @@ import com.google.common.net.InetAddresses;
import dagger.Module;
import dagger.Provides;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryEnvironment;
import google.registry.flows.EppException.AuthenticationErrorException;
import google.registry.flows.certs.CertificateChecker;
import google.registry.flows.certs.CertificateChecker.InsecureCertificateException;
import google.registry.model.registrar.Registrar;
import google.registry.request.Header;
import google.registry.util.CidrAddressBlock;
@@ -44,6 +45,9 @@ import javax.servlet.http.HttpServletRequest;
* <dl>
* <dt>X-SSL-Certificate
* <dd>This field should contain a base64 encoded digest of the client's TLS certificate. It is
* used only if the validation of the full certificate fails.
* <dt>X-SSL-Full-Certificate
* <dd>This field should contain a base64 encoding of the client's TLS certificate. It is
* validated during an EPP login command against a known good value that is transmitted out of
* band.
* <dt>X-Forwarded-For
@@ -56,17 +60,23 @@ public class TlsCredentials implements TransportCredentials {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final boolean requireSslCertificates;
private final String clientCertificateHash;
private final InetAddress clientInetAddr;
private final Optional<String> clientCertificateHash;
private final Optional<String> clientCertificate;
private final Optional<InetAddress> clientInetAddr;
private final CertificateChecker certificateChecker;
@Inject
public TlsCredentials(
@Config("requireSslCertificates") boolean requireSslCertificates,
@Header("X-SSL-Certificate") String clientCertificateHash,
@Header("X-Forwarded-For") Optional<String> clientAddress) {
@Header("X-SSL-Certificate") Optional<String> clientCertificateHash,
@Header("X-SSL-Full-Certificate") Optional<String> clientCertificate,
@Header("X-Forwarded-For") Optional<String> clientAddress,
CertificateChecker certificateChecker) {
this.requireSslCertificates = requireSslCertificates;
this.clientCertificateHash = clientCertificateHash;
this.clientInetAddr = clientAddress.isPresent() ? parseInetAddress(clientAddress.get()) : null;
this.clientCertificate = clientCertificate;
this.clientInetAddr = clientAddress.map(TlsCredentials::parseInetAddress);
this.certificateChecker = certificateChecker;
}
static InetAddress parseInetAddress(String asciiAddr) {
@@ -93,14 +103,18 @@ public class TlsCredentials implements TransportCredentials {
ImmutableList<CidrAddressBlock> ipAddressAllowList = registrar.getIpAddressAllowList();
if (ipAddressAllowList.isEmpty()) {
logger.atInfo().log(
"Skipping IP allow list check because %s doesn't have an IP allow list",
"Skipping IP allow list check because %s doesn't have an IP allow list.",
registrar.getClientId());
return;
}
for (CidrAddressBlock cidrAddressBlock : ipAddressAllowList) {
if (cidrAddressBlock.contains(clientInetAddr)) {
// IP address is in allow list; return early.
return;
// In the rare unexpected case that the client inet address wasn't passed along at all, then
// by default deny access.
if (clientInetAddr.isPresent()) {
for (CidrAddressBlock cidrAddressBlock : ipAddressAllowList) {
if (cidrAddressBlock.contains(clientInetAddr.get())) {
// IP address is in allow list; return early.
return;
}
}
}
logger.atInfo().log(
@@ -113,13 +127,65 @@ public class TlsCredentials implements TransportCredentials {
/**
* Verifies client SSL certificate is permitted to issue commands as {@code registrar}.
*
* @throws MissingRegistrarCertificateException if frontend didn't send certificate hash header
* @throws MissingRegistrarCertificateException if frontend didn't send certificate header
* @throws BadRegistrarCertificateException if registrar requires certificate and it didn't match
*/
@VisibleForTesting
void validateCertificate(Registrar registrar) throws AuthenticationErrorException {
if (isNullOrEmpty(registrar.getClientCertificateHash())
&& isNullOrEmpty(registrar.getFailoverClientCertificateHash())) {
// Check that certificate is present in registrar object
if (!registrar.getClientCertificate().isPresent()
&& !registrar.getFailoverClientCertificate().isPresent()) {
// Log an error and validate using certificate hash instead
// TODO(sarahbot): throw a RegistrarCertificateNotConfiguredException once hash is no longer
// used as failover
logger.atWarning().log(
"There is no certificate configured for registrar %s.", registrar.getClientId());
} else if (!clientCertificate.isPresent()) {
// Check that the request included the full certificate
// Log an error and validate using certificate hash instead
// TODO(sarahbot): throw a MissingRegistrarCertificateException once hash is no longer used as
// failover
logger.atWarning().log(
"Request from registrar %s did not include X-SSL-Full-Certificate.",
registrar.getClientId());
} else {
// Check if the certificate is equal to the one on file for the registrar.
if (clientCertificate.equals(registrar.getClientCertificate())
|| clientCertificate.equals(registrar.getFailoverClientCertificate())) {
// Check certificate for any requirement violations
// TODO(Sarahbot@): Throw exceptions instead of just logging once requirement enforcement
// begins
try {
certificateChecker.validateCertificate(clientCertificate.get());
} catch (InsecureCertificateException e) {
// throw exception in unit tests and Sandbox
if (RegistryEnvironment.get().equals(RegistryEnvironment.UNITTEST)
|| RegistryEnvironment.get().equals(RegistryEnvironment.SANDBOX)) {
throw new CertificateContainsSecurityViolationsException(e);
}
logger.atWarning().log(
"Registrar certificate used for %s does not meet certificate requirements: %s",
registrar.getClientId(), e.getMessage());
} catch (Exception e) {
logger.atWarning().log(
"Error validating certificate for %s: %s", registrar.getClientId(), e);
}
// successfully validated, return here since hash validation is not necessary
return;
}
// Log an error and validate using certificate hash instead
// TODO(sarahbot): throw a BadRegistrarCertificateException once hash is no longer used as
// failover
logger.atWarning().log("Non-matching certificate for registrar %s.", registrar.getClientId());
}
validateCertificateHash(registrar);
}
private void validateCertificateHash(Registrar registrar) throws AuthenticationErrorException {
// Check the certificate hash as a failover
// TODO(sarahbot): Remove hash checks once certificate checks are working.
if (!registrar.getClientCertificateHash().isPresent()
&& !registrar.getFailoverClientCertificateHash().isPresent()) {
if (requireSslCertificates) {
throw new RegistrarCertificateNotConfiguredException();
} else {
@@ -128,14 +194,17 @@ public class TlsCredentials implements TransportCredentials {
return;
}
}
if (isNullOrEmpty(clientCertificateHash)) {
logger.atInfo().log("Request did not include X-SSL-Certificate");
// Check that the request included the certificate hash
if (!clientCertificateHash.isPresent()) {
logger.atInfo().log(
"Request from registrar %s did not include X-SSL-Certificate.", registrar.getClientId());
throw new MissingRegistrarCertificateException();
}
// Check if the certificate hash is equal to the one on file for the registrar.
if (!clientCertificateHash.equals(registrar.getClientCertificateHash())
&& !clientCertificateHash.equals(registrar.getFailoverClientCertificateHash())) {
logger.atWarning().log(
"bad certificate hash (%s) for %s, wanted either %s or %s",
"Non-matching certificate hash (%s) for %s, wanted either %s or %s.",
clientCertificateHash,
registrar.getClientId(),
registrar.getClientCertificateHash(),
@@ -154,21 +223,35 @@ public class TlsCredentials implements TransportCredentials {
@Override
public String toString() {
return toStringHelper(getClass())
.add("clientCertificateHash", clientCertificateHash)
.add("clientAddress", clientInetAddr)
.add("clientCertificateHash", clientCertificateHash.orElse(null))
.add("clientAddress", clientInetAddr.orElse(null))
.toString();
}
/** Registrar certificate does not match stored certificate. */
public static class BadRegistrarCertificateException extends AuthenticationErrorException {
public BadRegistrarCertificateException() {
BadRegistrarCertificateException() {
super("Registrar certificate does not match stored certificate");
}
}
/** Registrar certificate contains the following security violations: ... */
public static class CertificateContainsSecurityViolationsException
extends AuthenticationErrorException {
InsecureCertificateException exception;
CertificateContainsSecurityViolationsException(InsecureCertificateException exception) {
super(
String.format(
"Registrar certificate contains the following security violations:\n%s",
exception.getMessage()));
this.exception = exception;
}
}
/** Registrar certificate not present. */
public static class MissingRegistrarCertificateException extends AuthenticationErrorException {
public MissingRegistrarCertificateException() {
MissingRegistrarCertificateException() {
super("Registrar certificate not present");
}
}
@@ -176,14 +259,14 @@ public class TlsCredentials implements TransportCredentials {
/** Registrar certificate is not configured. */
public static class RegistrarCertificateNotConfiguredException
extends AuthenticationErrorException {
public RegistrarCertificateNotConfiguredException() {
RegistrarCertificateNotConfiguredException() {
super("Registrar certificate is not configured");
}
}
/** Registrar IP address is not in stored allow list. */
public static class BadRegistrarIpAddressException extends AuthenticationErrorException {
public BadRegistrarIpAddressException() {
BadRegistrarIpAddressException() {
super("Registrar IP address is not in stored allow list");
}
}
@@ -191,10 +274,21 @@ public class TlsCredentials implements TransportCredentials {
/** Dagger module for the EPP TLS endpoint. */
@Module
public static final class EppTlsModule {
@Provides
@Header("X-SSL-Certificate")
static String provideClientCertificateHash(HttpServletRequest req) {
return extractRequiredHeader(req, "X-SSL-Certificate");
static Optional<String> provideClientCertificateHash(HttpServletRequest req) {
// Note: This header is actually required, we just want to handle its absence explicitly
// by throwing an EPP exception rather than a generic Bad Request exception.
return extractOptionalHeader(req, "X-SSL-Certificate");
}
@Provides
@Header("X-SSL-Full-Certificate")
static Optional<String> provideClientCertificate(HttpServletRequest req) {
// Note: This header is actually required, we just want to handle its absence explicitly
// by throwing an EPP exception rather than a generic Bad Request exception.
return extractOptionalHeader(req, "X-SSL-Full_Certificate");
}
@Provides

View File

@@ -28,10 +28,14 @@ import java.security.PublicKey;
import java.security.cert.CertificateException;
import java.security.cert.CertificateFactory;
import java.security.cert.X509Certificate;
import java.security.interfaces.ECPublicKey;
import java.security.interfaces.RSAPublicKey;
import java.util.Date;
import java.util.stream.Collectors;
import javax.inject.Inject;
import org.bouncycastle.jcajce.provider.asymmetric.util.EC5Util;
import org.bouncycastle.jce.ECNamedCurveTable;
import org.bouncycastle.jce.spec.ECNamedCurveParameterSpec;
import org.joda.time.DateTime;
import org.joda.time.Days;
@@ -42,6 +46,7 @@ public class CertificateChecker {
private final int daysToExpiration;
private final int minimumRsaKeyLength;
private final Clock clock;
private final ImmutableSet<String> allowedEcdsaCurves;
/**
* Constructs a CertificateChecker instance with the specified configuration parameters.
@@ -65,16 +70,18 @@ public class CertificateChecker {
@Inject
public CertificateChecker(
@Config("maxValidityDaysSchedule")
ImmutableSortedMap<DateTime, Integer> maxValidityLengthSchedule,
@Config("expirationWarningDays") int daysToExpiration,
ImmutableSortedMap<DateTime, Integer> maxValidityDaysSchedule,
@Config("expirationWarningDays") int expirationWarningDays,
@Config("minimumRsaKeyLength") int minimumRsaKeyLength,
@Config("allowedEcdsaCurves") ImmutableSet<String> allowedEcdsaCurves,
Clock clock) {
checkArgument(
maxValidityLengthSchedule.containsKey(START_OF_TIME),
maxValidityDaysSchedule.containsKey(START_OF_TIME),
"Max validity length schedule must contain an entry for START_OF_TIME");
this.maxValidityLengthSchedule = maxValidityLengthSchedule;
this.daysToExpiration = daysToExpiration;
this.maxValidityLengthSchedule = maxValidityDaysSchedule;
this.daysToExpiration = expirationWarningDays;
this.minimumRsaKeyLength = minimumRsaKeyLength;
this.allowedEcdsaCurves = allowedEcdsaCurves;
this.clock = clock;
}
@@ -82,14 +89,14 @@ public class CertificateChecker {
* Checks the given certificate string for violations and throws an exception if any violations
* exist.
*/
public void validateCertificate(String certificateString) {
public void validateCertificate(String certificateString) throws InsecureCertificateException {
ImmutableSet<CertificateViolation> violations = checkCertificate(certificateString);
if (!violations.isEmpty()) {
String displayMessages =
violations.stream()
.map(violation -> getViolationDisplayMessage(violation))
.collect(Collectors.joining("\n"));
throw new IllegalArgumentException(displayMessages);
throw new InsecureCertificateException(violations, displayMessages);
}
}
@@ -123,7 +130,9 @@ public class CertificateChecker {
violations.add(CertificateViolation.RSA_KEY_LENGTH_TOO_SHORT);
}
} else if (key.getAlgorithm().equals("EC")) {
// TODO(sarahbot): Add verification of ECDSA curves
if (!checkCurveName(key, allowedEcdsaCurves)) {
violations.add(CertificateViolation.INVALID_ECDSA_CURVE);
}
} else {
violations.add(CertificateViolation.ALGORITHM_CONSTRAINED);
}
@@ -171,6 +180,33 @@ public class CertificateChecker {
return Days.daysBetween(start.withTimeAtStartOfDay(), end.withTimeAtStartOfDay()).getDays();
}
/** Checks if the curve used for a public key is in the list of acceptable curves. */
private static boolean checkCurveName(PublicKey key, ImmutableSet<String> allowedEcdsaCurves) {
org.bouncycastle.jce.spec.ECParameterSpec params;
// These 2 different instances of PublicKey need to be handled separately since their OIDs are
// encoded differently. More details on this can be found at
// https://stackoverflow.com/questions/49895713/how-to-find-the-matching-curve-name-from-an-ecpublickey.
if (key instanceof ECPublicKey) {
ECPublicKey ecKey = (ECPublicKey) key;
params = EC5Util.convertSpec(ecKey.getParams(), false);
} else if (key instanceof org.bouncycastle.jce.interfaces.ECPublicKey) {
org.bouncycastle.jce.interfaces.ECPublicKey ecKey =
(org.bouncycastle.jce.interfaces.ECPublicKey) key;
params = ecKey.getParameters();
} else {
throw new IllegalArgumentException("Unrecognized instance of PublicKey.");
}
return allowedEcdsaCurves.stream()
.anyMatch(
curve -> {
ECNamedCurveParameterSpec cParams = ECNamedCurveTable.getParameterSpec(curve);
return cParams.getN().equals(params.getN())
&& cParams.getH().equals(params.getH())
&& cParams.getCurve().equals(params.getCurve())
&& cParams.getG().equals(params.getG());
});
}
private String getViolationDisplayMessage(CertificateViolation certificateViolation) {
// Yes, we'd rather do this as an instance method on the CertificateViolation enum itself, but
// we can't because we need access to configuration (injected as instance variables) which you
@@ -190,6 +226,9 @@ public class CertificateChecker {
return String.format(
"Certificate validity period is too long; it must be less than or equal to %d days.",
this.maxValidityLengthSchedule.lastEntry().getValue());
case INVALID_ECDSA_CURVE:
return String.format(
"The ECDSA key must use one of these algorithms: %s", allowedEcdsaCurves);
default:
throw new IllegalArgumentException(
String.format(
@@ -206,7 +245,8 @@ public class CertificateChecker {
NOT_YET_VALID,
VALIDITY_LENGTH_TOO_LONG,
RSA_KEY_LENGTH_TOO_SHORT,
ALGORITHM_CONSTRAINED;
ALGORITHM_CONSTRAINED,
INVALID_ECDSA_CURVE;
/**
* Gets a suitable end-user-facing display message for this particular certificate violation.
@@ -218,4 +258,14 @@ public class CertificateChecker {
return certificateChecker.getViolationDisplayMessage(this);
}
}
/** Exception to throw when a certificate has security violations. */
public static class InsecureCertificateException extends Exception {
ImmutableSet<CertificateViolation> violations;
InsecureCertificateException(ImmutableSet<CertificateViolation> violations, String message) {
super(message);
this.violations = violations;
}
}
}

View File

@@ -19,9 +19,9 @@ import static google.registry.flows.ResourceFlowUtils.verifyResourceDoesNotExist
import static google.registry.flows.contact.ContactFlowUtils.validateAsciiPostalInfo;
import static google.registry.flows.contact.ContactFlowUtils.validateContactAgainstPolicy;
import static google.registry.model.EppResourceUtils.createRepoId;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.config.RegistryConfig.Config;
import google.registry.flows.EppException;
@@ -95,11 +95,12 @@ public final class ContactCreateFlow implements TransactionalFlow {
.setModificationTime(now)
.setXmlBytes(null) // We don't want to store contact details in the history entry.
.setParent(Key.create(newContact));
ofy().save().entities(
newContact,
historyBuilder.build(),
ForeignKeyIndex.create(newContact, newContact.getDeletionTime()),
EppResourceIndex.create(Key.create(newContact)));
tm().insertAll(
ImmutableSet.of(
newContact,
historyBuilder.build().toChildHistoryEntity(),
ForeignKeyIndex.create(newContact, newContact.getDeletionTime()),
EppResourceIndex.create(Key.create(newContact))));
return responseBuilder
.setResData(ContactCreateData.create(newContact.getContactId(), now))
.build();

View File

@@ -21,7 +21,6 @@ import static google.registry.flows.ResourceFlowUtils.verifyNoDisallowedStatuses
import static google.registry.flows.ResourceFlowUtils.verifyOptionalAuthInfo;
import static google.registry.flows.ResourceFlowUtils.verifyResourceOwnership;
import static google.registry.model.eppoutput.Result.Code.SUCCESS_WITH_ACTION_PENDING;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
@@ -101,7 +100,8 @@ public final class ContactDeleteFlow implements TransactionalFlow {
.setType(HistoryEntry.Type.CONTACT_PENDING_DELETE)
.setModificationTime(now)
.setParent(Key.create(existingContact));
ofy().save().<Object>entities(newContact, historyBuilder.build());
tm().insert(historyBuilder.build().toChildHistoryEntity());
tm().update(newContact);
return responseBuilder.setResultFromCode(SUCCESS_WITH_ACTION_PENDING).build();
}
}

View File

@@ -20,7 +20,6 @@ import static google.registry.flows.ResourceFlowUtils.verifyResourceOwnership;
import static google.registry.model.EppResourceUtils.isLinked;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.Flow;
@@ -77,7 +76,7 @@ public final class ContactInfoFlow implements Flow {
clientId.equals(contact.getCurrentSponsorClientId()) || authInfo.isPresent();
ImmutableSet.Builder<StatusValue> statusValues = new ImmutableSet.Builder<>();
statusValues.addAll(contact.getStatusValues());
if (isLinked(Key.create(contact), now)) {
if (isLinked(contact.createVKey(), now)) {
statusValues.add(StatusValue.LINKED);
}
return responseBuilder

View File

@@ -22,9 +22,9 @@ import static google.registry.flows.ResourceFlowUtils.verifyResourceOwnership;
import static google.registry.flows.contact.ContactFlowUtils.createGainingTransferPollMessage;
import static google.registry.flows.contact.ContactFlowUtils.createTransferResponse;
import static google.registry.model.ResourceTransferUtils.approvePendingTransfer;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
@@ -94,7 +94,8 @@ public final class ContactTransferApproveFlow implements TransactionalFlow {
// Create a poll message for the gaining client.
PollMessage gainingPollMessage =
createGainingTransferPollMessage(targetId, newContact.getTransferData(), historyEntry);
ofy().save().<Object>entities(newContact, historyEntry, gainingPollMessage);
tm().insertAll(ImmutableSet.of(historyEntry.toChildHistoryEntity(), gainingPollMessage));
tm().update(newContact);
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
tm().delete(existingContact.getTransferData().getServerApproveEntities());

View File

@@ -22,9 +22,9 @@ import static google.registry.flows.ResourceFlowUtils.verifyTransferInitiator;
import static google.registry.flows.contact.ContactFlowUtils.createLosingTransferPollMessage;
import static google.registry.flows.contact.ContactFlowUtils.createTransferResponse;
import static google.registry.model.ResourceTransferUtils.denyPendingTransfer;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
@@ -90,7 +90,8 @@ public final class ContactTransferCancelFlow implements TransactionalFlow {
// Create a poll message for the losing client.
PollMessage losingPollMessage =
createLosingTransferPollMessage(targetId, newContact.getTransferData(), historyEntry);
ofy().save().<Object>entities(newContact, historyEntry, losingPollMessage);
tm().insertAll(ImmutableSet.of(historyEntry.toChildHistoryEntity(), losingPollMessage));
tm().update(newContact);
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
tm().delete(existingContact.getTransferData().getServerApproveEntities());

View File

@@ -22,9 +22,9 @@ import static google.registry.flows.ResourceFlowUtils.verifyResourceOwnership;
import static google.registry.flows.contact.ContactFlowUtils.createGainingTransferPollMessage;
import static google.registry.flows.contact.ContactFlowUtils.createTransferResponse;
import static google.registry.model.ResourceTransferUtils.denyPendingTransfer;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
@@ -87,7 +87,8 @@ public final class ContactTransferRejectFlow implements TransactionalFlow {
.build();
PollMessage gainingPollMessage =
createGainingTransferPollMessage(targetId, newContact.getTransferData(), historyEntry);
ofy().save().<Object>entities(newContact, historyEntry, gainingPollMessage);
tm().insertAll(ImmutableSet.of(historyEntry.toChildHistoryEntity(), gainingPollMessage));
tm().update(newContact);
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
tm().delete(existingContact.getTransferData().getServerApproveEntities());

View File

@@ -23,7 +23,6 @@ import static google.registry.flows.contact.ContactFlowUtils.createGainingTransf
import static google.registry.flows.contact.ContactFlowUtils.createLosingTransferPollMessage;
import static google.registry.flows.contact.ContactFlowUtils.createTransferResponse;
import static google.registry.model.eppoutput.Result.Code.SUCCESS_WITH_ACTION_PENDING;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
@@ -145,12 +144,13 @@ public final class ContactTransferRequestFlow implements TransactionalFlow {
.setTransferData(pendingTransferData)
.addStatusValue(StatusValue.PENDING_TRANSFER)
.build();
ofy().save().<Object>entities(
newContact,
historyEntry,
requestPollMessage,
serverApproveGainingPollMessage,
serverApproveLosingPollMessage);
tm().update(newContact);
tm().insertAll(
ImmutableSet.of(
historyEntry.toChildHistoryEntity(),
requestPollMessage,
serverApproveGainingPollMessage,
serverApproveLosingPollMessage));
return responseBuilder
.setResultFromCode(SUCCESS_WITH_ACTION_PENDING)
.setResData(createTransferResponse(targetId, newContact.getTransferData()))

View File

@@ -24,7 +24,6 @@ import static google.registry.flows.ResourceFlowUtils.verifyOptionalAuthInfo;
import static google.registry.flows.ResourceFlowUtils.verifyResourceOwnership;
import static google.registry.flows.contact.ContactFlowUtils.validateAsciiPostalInfo;
import static google.registry.flows.contact.ContactFlowUtils.validateContactAgainstPolicy;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
@@ -151,7 +150,8 @@ public final class ContactUpdateFlow implements TransactionalFlow {
}
validateAsciiPostalInfo(newContact.getInternationalizedPostalInfo());
validateContactAgainstPolicy(newContact);
ofy().save().<Object>entities(newContact, historyBuilder.build());
tm().insert(historyBuilder.build().toChildHistoryEntity());
tm().update(newContact);
return responseBuilder.build();
}

View File

@@ -225,7 +225,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
if (gracePeriod.getOneTimeBillingEvent() != null) {
// Take the amount of amount of registration time being refunded off the expiration time.
// This can be either add grace periods or renew grace periods.
BillingEvent.OneTime oneTime = tm().load(gracePeriod.getOneTimeBillingEvent());
BillingEvent.OneTime oneTime = tm().loadByKey(gracePeriod.getOneTimeBillingEvent());
newExpirationTime = newExpirationTime.minusYears(oneTime.getPeriodYears());
} else if (gracePeriod.getRecurringBillingEvent() != null) {
// Take 1 year off the registration if in the autorenew grace period (no need to load the
@@ -372,12 +372,12 @@ public final class DomainDeleteFlow implements TransactionalFlow {
private Money getGracePeriodCost(GracePeriod gracePeriod, DateTime now) {
if (gracePeriod.getType() == GracePeriodStatus.AUTO_RENEW) {
DateTime autoRenewTime =
tm().load(checkNotNull(gracePeriod.getRecurringBillingEvent()))
tm().loadByKey(checkNotNull(gracePeriod.getRecurringBillingEvent()))
.getRecurrenceTimeOfYear()
.getLastInstanceBeforeOrAt(now);
return getDomainRenewCost(targetId, autoRenewTime, 1);
}
return tm().load(checkNotNull(gracePeriod.getOneTimeBillingEvent())).getCost();
return tm().loadByKey(checkNotNull(gracePeriod.getOneTimeBillingEvent())).getCost();
}
@Nullable

View File

@@ -517,7 +517,7 @@ public class DomainFlowUtils {
*/
public static void updateAutorenewRecurrenceEndTime(DomainBase domain, DateTime newEndTime) {
Optional<PollMessage.Autorenew> autorenewPollMessage =
tm().maybeLoad(domain.getAutorenewPollMessage());
tm().loadByKeyIfPresent(domain.getAutorenewPollMessage());
// Construct an updated autorenew poll message. If the autorenew poll message no longer exists,
// create a new one at the same id. This can happen if a transfer was requested on a domain
@@ -542,7 +542,7 @@ public class DomainFlowUtils {
ofy().save().entity(updatedAutorenewPollMessage);
}
Recurring recurring = tm().load(domain.getAutorenewBillingEvent());
Recurring recurring = tm().loadByKey(domain.getAutorenewBillingEvent());
ofy().save().entity(recurring.asBuilder().setRecurrenceEndTime(newEndTime).build());
}
@@ -1022,7 +1022,7 @@ public class DomainFlowUtils {
for (DesignatedContact contact : contacts) {
builder.add(
ForeignKeyedDesignatedContact.create(
contact.getType(), tm().load(contact.getContactKey()).getContactId()));
contact.getType(), tm().loadByKey(contact.getContactKey()).getContactId()));
}
return builder.build();
}

View File

@@ -101,8 +101,8 @@ public final class DomainInfoFlow implements Flow {
flowCustomLogic.afterValidation(
AfterValidationParameters.newBuilder().setDomain(domain).build());
// Prefetch all referenced resources. Calling values() blocks until loading is done.
tm().load(domain.getNameservers());
tm().load(domain.getReferencedContacts());
tm().loadByKeys(domain.getNameservers());
tm().loadByKeys(domain.getReferencedContacts());
// Registrars can only see a few fields on unauthorized domains.
// This is a policy decision that is left up to us by the rfcs.
DomainInfoData.Builder infoBuilder =
@@ -110,7 +110,7 @@ public final class DomainInfoFlow implements Flow {
.setFullyQualifiedDomainName(domain.getDomainName())
.setRepoId(domain.getRepoId())
.setCurrentSponsorClientId(domain.getCurrentSponsorClientId())
.setRegistrant(tm().load(domain.getRegistrant()).getContactId());
.setRegistrant(tm().loadByKey(domain.getRegistrant()).getContactId());
// If authInfo is non-null, then the caller is authorized to see the full information since we
// will have already verified the authInfo is valid.
if (clientId.equals(domain.getCurrentSponsorClientId()) || authInfo.isPresent()) {

View File

@@ -15,14 +15,13 @@
package google.registry.flows.domain.token;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.Maps;
import com.google.common.net.InternetDomainName;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.EppException.AssociationProhibitsOperationException;
import google.registry.flows.EppException.AuthorizationErrorException;
@@ -108,7 +107,7 @@ public class AllocationTokenFlowUtils {
/** Redeems a SINGLE_USE {@link AllocationToken}, returning the redeemed copy. */
public AllocationToken redeemToken(
AllocationToken token, VKey<HistoryEntry> redemptionHistoryEntry) {
AllocationToken token, VKey<? extends HistoryEntry> redemptionHistoryEntry) {
checkArgument(
TokenType.SINGLE_USE.equals(token.getTokenType()),
"Only SINGLE_USE tokens can be marked as redeemed");
@@ -153,14 +152,15 @@ public class AllocationTokenFlowUtils {
// See https://tools.ietf.org/html/draft-ietf-regext-allocation-token-04#section-2.1
throw new InvalidAllocationTokenException();
}
AllocationToken tokenEntity = ofy().load().key(Key.create(AllocationToken.class, token)).now();
if (tokenEntity == null) {
Optional<AllocationToken> maybeTokenEntity =
tm().loadByKeyIfPresent(VKey.create(AllocationToken.class, token));
if (!maybeTokenEntity.isPresent()) {
throw new InvalidAllocationTokenException();
}
if (tokenEntity.isRedeemed()) {
if (maybeTokenEntity.get().isRedeemed()) {
throw new AlreadyRedeemedAllocationTokenException();
}
return tokenEntity;
return maybeTokenEntity.get();
}
// Note: exception messages should be <= 32 characters long for domain check results

View File

@@ -21,10 +21,8 @@ import static google.registry.flows.host.HostFlowUtils.validateHostName;
import static google.registry.flows.host.HostFlowUtils.verifySuperordinateDomainNotInPendingDelete;
import static google.registry.flows.host.HostFlowUtils.verifySuperordinateDomainOwnership;
import static google.registry.model.EppResourceUtils.createRepoId;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.isNullOrEmpty;
import static google.registry.util.CollectionUtils.union;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
@@ -137,13 +135,11 @@ public final class HostCreateFlow implements TransactionalFlow {
ImmutableSet<ImmutableObject> entitiesToSave =
ImmutableSet.of(
newHost,
historyBuilder.build(),
historyBuilder.build().toChildHistoryEntity(),
ForeignKeyIndex.create(newHost, newHost.getDeletionTime()),
EppResourceIndex.create(Key.create(newHost)));
if (superordinateDomain.isPresent()) {
entitiesToSave =
union(
entitiesToSave,
tm().update(
superordinateDomain
.get()
.asBuilder()
@@ -153,7 +149,7 @@ public final class HostCreateFlow implements TransactionalFlow {
// they are only written as NS records from the referencing domain.
dnsQueue.addHostRefreshTask(targetId);
}
ofy().save().entities(entitiesToSave);
tm().insertAll(entitiesToSave);
return responseBuilder.setResData(HostCreateData.create(targetId, now)).build();
}

View File

@@ -21,7 +21,6 @@ import static google.registry.flows.ResourceFlowUtils.verifyNoDisallowedStatuses
import static google.registry.flows.ResourceFlowUtils.verifyResourceOwnership;
import static google.registry.flows.host.HostFlowUtils.validateHostName;
import static google.registry.model.eppoutput.Result.Code.SUCCESS_WITH_ACTION_PENDING;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
@@ -96,7 +95,7 @@ public final class HostDeleteFlow implements TransactionalFlow {
// the client id, needs to be read off of it.
EppResource owningResource =
existingHost.isSubordinate()
? tm().load(existingHost.getSuperordinateDomain()).cloneProjectedAtTime(now)
? tm().loadByKey(existingHost.getSuperordinateDomain()).cloneProjectedAtTime(now)
: existingHost;
verifyResourceOwnership(clientId, owningResource);
}
@@ -108,7 +107,8 @@ public final class HostDeleteFlow implements TransactionalFlow {
.setType(HistoryEntry.Type.HOST_PENDING_DELETE)
.setModificationTime(now)
.setParent(Key.create(existingHost));
ofy().save().<Object>entities(newHost, historyBuilder.build());
tm().insert(historyBuilder.build().toChildHistoryEntity());
tm().update(newHost);
return responseBuilder.setResultFromCode(SUCCESS_WITH_ACTION_PENDING).build();
}
}

View File

@@ -21,7 +21,6 @@ import static google.registry.model.EppResourceUtils.isLinked;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
import google.registry.flows.ExtensionManager;
import google.registry.flows.Flow;
@@ -68,7 +67,7 @@ public final class HostInfoFlow implements Flow {
HostResource host = loadAndVerifyExistence(HostResource.class, targetId, now);
ImmutableSet.Builder<StatusValue> statusValues = new ImmutableSet.Builder<>();
statusValues.addAll(host.getStatusValues());
if (isLinked(Key.create(host), now)) {
if (isLinked(host.createVKey(), now)) {
statusValues.add(StatusValue.LINKED);
}
HostInfoData.Builder hostInfoDataBuilder = HostInfoData.newBuilder();
@@ -77,7 +76,7 @@ public final class HostInfoFlow implements Flow {
// there is no superordinate domain, the host's own values for these fields will be correct.
if (host.isSubordinate()) {
DomainBase superordinateDomain =
tm().load(host.getSuperordinateDomain()).cloneProjectedAtTime(now);
tm().loadByKey(host.getSuperordinateDomain()).cloneProjectedAtTime(now);
hostInfoDataBuilder
.setCurrentSponsorClientId(superordinateDomain.getCurrentSponsorClientId())
.setLastTransferTime(host.computeLastTransferTime(superordinateDomain));

View File

@@ -27,7 +27,6 @@ import static google.registry.flows.host.HostFlowUtils.validateHostName;
import static google.registry.flows.host.HostFlowUtils.verifySuperordinateDomainNotInPendingDelete;
import static google.registry.flows.host.HostFlowUtils.verifySuperordinateDomainOwnership;
import static google.registry.model.index.ForeignKeyIndex.loadAndGetKey;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.isNullOrEmpty;
@@ -139,7 +138,7 @@ public final class HostUpdateFlow implements TransactionalFlow {
String newHostName = firstNonNull(suppliedNewHostName, oldHostName);
DomainBase oldSuperordinateDomain =
existingHost.isSubordinate()
? tm().load(existingHost.getSuperordinateDomain()).cloneProjectedAtTime(now)
? tm().loadByKey(existingHost.getSuperordinateDomain()).cloneProjectedAtTime(now)
: null;
// Note that lookupSuperordinateDomain calls cloneProjectedAtTime on the domain for us.
Optional<DomainBase> newSuperordinateDomain =
@@ -191,23 +190,26 @@ public final class HostUpdateFlow implements TransactionalFlow {
.setPersistedCurrentSponsorClientId(newPersistedClientId)
.build();
verifyHasIpsIffIsExternal(command, existingHost, newHost);
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
entitiesToSave.add(newHost);
ImmutableSet.Builder<ImmutableObject> entitiesToInsert = new ImmutableSet.Builder<>();
ImmutableSet.Builder<ImmutableObject> entitiesToUpdate = new ImmutableSet.Builder<>();
entitiesToUpdate.add(newHost);
// Keep the {@link ForeignKeyIndex} for this host up to date.
if (isHostRename) {
// Update the foreign key for the old host name and save one for the new host name.
entitiesToSave.add(
ForeignKeyIndex.create(existingHost, now),
ForeignKeyIndex.create(newHost, newHost.getDeletionTime()));
entitiesToUpdate.add(ForeignKeyIndex.create(existingHost, now));
entitiesToUpdate.add(ForeignKeyIndex.create(newHost, newHost.getDeletionTime()));
updateSuperordinateDomains(existingHost, newHost);
}
enqueueTasks(existingHost, newHost);
entitiesToSave.add(historyBuilder
.setType(HistoryEntry.Type.HOST_UPDATE)
.setModificationTime(now)
.setParent(Key.create(existingHost))
.build());
ofy().save().entities(entitiesToSave.build());
entitiesToInsert.add(
historyBuilder
.setType(HistoryEntry.Type.HOST_UPDATE)
.setModificationTime(now)
.setParent(Key.create(existingHost))
.build()
.toChildHistoryEntity());
tm().updateAll(entitiesToUpdate.build());
tm().insertAll(entitiesToInsert.build());
return responseBuilder.build();
}
@@ -286,7 +288,7 @@ public final class HostUpdateFlow implements TransactionalFlow {
&& Objects.equals(
existingHost.getSuperordinateDomain(), newHost.getSuperordinateDomain())) {
tm().put(
tm().load(existingHost.getSuperordinateDomain())
tm().loadByKey(existingHost.getSuperordinateDomain())
.asBuilder()
.removeSubordinateHost(existingHost.getHostName())
.addSubordinateHost(newHost.getHostName())
@@ -295,14 +297,14 @@ public final class HostUpdateFlow implements TransactionalFlow {
}
if (existingHost.isSubordinate()) {
tm().put(
tm().load(existingHost.getSuperordinateDomain())
tm().loadByKey(existingHost.getSuperordinateDomain())
.asBuilder()
.removeSubordinateHost(existingHost.getHostName())
.build());
}
if (newHost.isSubordinate()) {
tm().put(
tm().load(newHost.getSuperordinateDomain())
tm().loadByKey(newHost.getSuperordinateDomain())
.asBuilder()
.addSubordinateHost(newHost.getHostName())
.build());

View File

@@ -141,7 +141,7 @@ public class LoginFlow implements Flow {
sessionMetadata.resetFailedLoginAttempts();
sessionMetadata.setClientId(login.getClientId());
sessionMetadata.setServiceExtensionUris(serviceExtensionUrisBuilder.build());
return responseBuilder.build();
return responseBuilder.setIsLoginResponse().build();
}
/** Registrar with this client ID could not be found. */

View File

@@ -15,6 +15,7 @@
package google.registry.mapreduce;
import static google.registry.mapreduce.MapreduceRunner.PARAM_DRY_RUN;
import static google.registry.mapreduce.MapreduceRunner.PARAM_FAST;
import static google.registry.mapreduce.MapreduceRunner.PARAM_MAP_SHARDS;
import static google.registry.mapreduce.MapreduceRunner.PARAM_REDUCE_SHARDS;
import static google.registry.request.RequestParameters.extractBooleanParameter;
@@ -36,6 +37,12 @@ public final class MapreduceModule {
return extractBooleanParameter(req, PARAM_DRY_RUN);
}
@Provides
@Parameter(PARAM_FAST)
static boolean provideIsFast(HttpServletRequest req) {
return extractBooleanParameter(req, PARAM_FAST);
}
@Provides
@Parameter(PARAM_MAP_SHARDS)
static Optional<Integer> provideMapShards(HttpServletRequest req) {

View File

@@ -55,6 +55,7 @@ public class MapreduceRunner {
public static final String PARAM_DRY_RUN = "dryRun";
public static final String PARAM_MAP_SHARDS = "mapShards";
public static final String PARAM_REDUCE_SHARDS = "reduceShards";
public static final String PARAM_FAST = "fast";
private static final String BASE_URL = "/_dr/mapreduce/";
private static final String QUEUE_NAME = "mapreduce";

View File

@@ -0,0 +1,38 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryEnvironment;
/** Utility methods related to migrating dual-read/dual-write entities. */
public class DatabaseMigrationUtils {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
/** Throws exceptions only in unit tests, otherwise only logs exceptions. */
public static void suppressExceptionUnlessInTest(Runnable work, String message) {
try {
work.run();
} catch (Exception e) {
if (RegistryEnvironment.get().equals(RegistryEnvironment.UNITTEST)) {
throw e;
}
logger.atWarning().withCause(e).log(message);
}
}
private DatabaseMigrationUtils() {}
}

View File

@@ -148,7 +148,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
*
* @see google.registry.model.translators.CommitLogRevisionsTranslatorFactory
*/
@Transient
@Transient @DoNotCompare
ImmutableSortedMap<DateTime, Key<CommitLogManifest>> revisions = ImmutableSortedMap.of();
public String getRepoId() {
@@ -360,13 +360,13 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
@Override
public EppResource load(VKey<? extends EppResource> key) {
return tm().doTransactionless(() -> tm().load(key));
return tm().doTransactionless(() -> tm().loadByKey(key));
}
@Override
public Map<VKey<? extends EppResource>, EppResource> loadAll(
Iterable<? extends VKey<? extends EppResource>> keys) {
return tm().doTransactionless(() -> tm().load(keys));
return tm().doTransactionless(() -> tm().loadByKeys(keys));
}
};
@@ -406,7 +406,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
public static ImmutableMap<VKey<? extends EppResource>, EppResource> loadCached(
Iterable<VKey<? extends EppResource>> keys) {
if (!RegistryConfig.isEppResourceCachingEnabled()) {
return tm().load(keys);
return tm().loadByKeys(keys);
}
try {
return cacheEppResources.getAll(keys);
@@ -423,7 +423,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
*/
public static <T extends EppResource> T loadCached(VKey<T> key) {
if (!RegistryConfig.isEppResourceCachingEnabled()) {
return tm().load(key);
return tm().loadByKey(key);
}
try {
// Safe to cast because loading a Key<T> returns an entity of type T.

View File

@@ -17,7 +17,9 @@ package google.registry.model;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.persistence.transaction.TransactionManagerUtil.transactIfJpaTm;
import static google.registry.util.DateTimeUtils.isAtOrAfter;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import static google.registry.util.DateTimeUtils.latestOf;
@@ -27,7 +29,6 @@ import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.Result;
import com.googlecode.objectify.cmd.Query;
import com.googlecode.objectify.util.ResultNow;
import google.registry.config.RegistryConfig;
import google.registry.model.EppResource.BuilderWithTransferData;
@@ -36,6 +37,7 @@ import google.registry.model.EppResource.ResourceWithTransferData;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.DomainBase;
import google.registry.model.eppcommon.StatusValue;
import google.registry.model.host.HostResource;
import google.registry.model.index.ForeignKeyIndex;
import google.registry.model.ofy.CommitLogManifest;
import google.registry.model.ofy.CommitLogMutation;
@@ -43,11 +45,13 @@ import google.registry.model.registry.Registry;
import google.registry.model.transfer.DomainTransferData;
import google.registry.model.transfer.TransferData;
import google.registry.model.transfer.TransferStatus;
import google.registry.persistence.VKey;
import java.util.List;
import java.util.Map.Entry;
import java.util.Optional;
import java.util.function.Function;
import javax.annotation.Nullable;
import javax.persistence.Query;
import org.joda.time.DateTime;
import org.joda.time.Interval;
@@ -56,6 +60,22 @@ public final class EppResourceUtils {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final String CONTACT_LINKED_DOMAIN_QUERY =
"SELECT repoId FROM Domain "
+ "WHERE (adminContact = :fkRepoId "
+ "OR billingContact = :fkRepoId "
+ "OR techContact = :fkRepoId "
+ "OR registrantContact = :fkRepoId) "
+ "AND deletionTime > :now";
// We have to use the native SQL query here because DomainHost table doesn't have its entity
// class so we cannot reference its property like domainHost.hostRepoId in a JPQL query.
private static final String HOST_LINKED_DOMAIN_QUERY =
"SELECT d.repo_id FROM \"Domain\" d "
+ "JOIN \"DomainHost\" dh ON dh.domain_repo_id = d.repo_id "
+ "WHERE d.deletion_time > :now "
+ "AND dh.host_repo_id = :fkRepoId";
/** Returns the full domain repoId in the format HEX-TLD for the specified long id and tld. */
public static String createDomainRepoId(long repoId, String tld) {
return createRepoId(repoId, Registry.get(tld).getRoidSuffix());
@@ -135,7 +155,7 @@ public final class EppResourceUtils {
useCache
? ForeignKeyIndex.loadCached(clazz, ImmutableList.of(foreignKey), now)
.getOrDefault(foreignKey, null)
: ofy().load().type(ForeignKeyIndex.mapToFkiClass(clazz)).id(foreignKey).now();
: ForeignKeyIndex.load(clazz, foreignKey, now);
// The value of fki.getResourceKey() might be null for hard-deleted prober data.
if (fki == null || isAtOrAfter(now, fki.getDeletionTime()) || fki.getResourceKey() == null) {
return Optional.empty();
@@ -143,7 +163,7 @@ public final class EppResourceUtils {
T resource =
useCache
? EppResource.loadCached(fki.getResourceKey())
: tm().maybeLoad(fki.getResourceKey()).orElse(null);
: transactIfJpaTm(() -> tm().loadByKeyIfPresent(fki.getResourceKey()).orElse(null));
if (resource == null || isAtOrAfter(now, resource.getDeletionTime())) {
return Optional.empty();
}
@@ -364,21 +384,63 @@ public final class EppResourceUtils {
}
/**
* Returns a query for domains or applications that reference a specified contact or host.
* Returns a set of {@link VKey} for domains that reference a specified contact or host.
*
* <p>This is an eventually consistent query.
* <p>This is an eventually consistent query if used for Datastore.
*
* @param key the referent key
* @param now the logical time of the check
* @param limit the maximum number of returned keys
*/
public static Query<DomainBase> queryForLinkedDomains(
Key<? extends EppResource> key, DateTime now) {
boolean isContactKey = key.getKind().equals(Key.getKind(ContactResource.class));
return ofy()
.load()
.type(DomainBase.class)
.filter(isContactKey ? "allContacts.contact" : "nsHosts", key)
.filter("deletionTime >", now);
public static ImmutableSet<VKey<DomainBase>> getLinkedDomainKeys(
VKey<? extends EppResource> key, DateTime now, int limit) {
checkArgument(
key.getKind().equals(ContactResource.class) || key.getKind().equals(HostResource.class),
"key must be either VKey<ContactResource> or VKey<HostResource>, but it is %s",
key);
boolean isContactKey = key.getKind().equals(ContactResource.class);
if (tm().isOfy()) {
return ofy()
.load()
.type(DomainBase.class)
.filter(isContactKey ? "allContacts.contact" : "nsHosts", key.getOfyKey())
.filter("deletionTime >", now)
.limit(limit)
.keys()
.list()
.stream()
.map(DomainBase::createVKey)
.collect(toImmutableSet());
} else {
return tm().transact(
() -> {
Query query;
if (isContactKey) {
query =
jpaTm()
.getEntityManager()
.createQuery(CONTACT_LINKED_DOMAIN_QUERY, String.class)
.setParameter("fkRepoId", key)
.setParameter("now", now);
} else {
query =
jpaTm()
.getEntityManager()
.createNativeQuery(HOST_LINKED_DOMAIN_QUERY)
.setParameter("fkRepoId", key.getSqlKey())
.setParameter("now", now.toDate());
}
return (ImmutableSet<VKey<DomainBase>>)
query
.setMaxResults(limit)
.getResultStream()
.map(
repoId ->
DomainBase.createVKey(
Key.create(DomainBase.class, (String) repoId)))
.collect(toImmutableSet());
});
}
}
/**
@@ -389,8 +451,8 @@ public final class EppResourceUtils {
* @param key the referent key
* @param now the logical time of the check
*/
public static boolean isLinked(Key<? extends EppResource> key, DateTime now) {
return queryForLinkedDomains(key, now).limit(1).count() > 0;
public static boolean isLinked(VKey<? extends EppResource> key, DateTime now) {
return getLinkedDomainKeys(key, now, 1).size() > 0;
}
private EppResourceUtils() {}

View File

@@ -54,9 +54,37 @@ public abstract class ImmutableObject implements Cloneable {
@Target(FIELD)
public @interface DoNotHydrate {}
@Ignore
@XmlTransient
Integer hashCode;
/**
* Indicates that the field should be ignored when comparing an object in the datastore to the
* corresponding object in Cloud SQL.
*/
@Documented
@Retention(RUNTIME)
@Target(FIELD)
public @interface DoNotCompare {}
/**
* Indicates that the field stores a null value to indicate an empty set. This is also used in
* object comparison.
*/
@Documented
@Retention(RUNTIME)
@Target(FIELD)
public @interface EmptySetToNull {}
/**
* Indicates that the field does not take part in the immutability contract.
*
* <p>Certain fields currently get modified by hibernate and there is nothing we can do about it.
* As well as violating immutability, this breaks hashing and equality comparisons, so we mark
* these fields with this annotation to exclude them from most operations.
*/
@Documented
@Retention(RUNTIME)
@Target(FIELD)
public @interface Insignificant {}
@Ignore @XmlTransient protected Integer hashCode;
private boolean equalsImmutableObject(ImmutableObject other) {
return getClass().equals(other.getClass())
@@ -71,7 +99,14 @@ public abstract class ImmutableObject implements Cloneable {
* <p>Isolated into a method so that derived classes can override it.
*/
protected Map<Field, Object> getSignificantFields() {
return ModelUtils.getFieldValues(this);
// Can't use streams or ImmutableMap because we can have null values.
Map<Field, Object> result = new LinkedHashMap();
for (Map.Entry<Field, Object> entry : ModelUtils.getFieldValues(this).entrySet()) {
if (!entry.getKey().isAnnotationPresent(Insignificant.class)) {
result.put(entry.getKey(), entry.getValue());
}
}
return result;
}
@Override

View File

@@ -87,7 +87,7 @@ public class ModelUtils {
});
/** Lists all instance fields on an object, including non-public and inherited fields. */
static Map<String, Field> getAllFields(Class<?> clazz) {
public static Map<String, Field> getAllFields(Class<?> clazz) {
return ALL_FIELDS_CACHE.getUnchecked(clazz);
}

View File

@@ -211,11 +211,6 @@ public final class OteAccountBuilder {
return transformRegistrars(builder -> builder.setPassword(password));
}
/** Sets the client certificate hash to all the OT&amp;E Registrars. */
public OteAccountBuilder setCertificateHash(String certHash) {
return transformRegistrars(builder -> builder.setClientCertificateHash(certHash));
}
/** Sets the client certificate to all the OT&amp;E Registrars. */
public OteAccountBuilder setCertificate(String asciiCert, DateTime now) {
return transformRegistrars(builder -> builder.setClientCertificate(asciiCert, now));

View File

@@ -22,12 +22,17 @@ import javax.annotation.Nullable;
import org.joda.time.DateTime;
/**
* A timestamp that auto-updates on each save to Datastore.
* A timestamp that auto-updates on each save to Datastore/Cloud SQL.
*
* @see UpdateAutoTimestampTranslatorFactory
*/
public class UpdateAutoTimestamp extends ImmutableObject {
// When set to true, database converters/translators should do tha auto update. When set to
// false, auto update should be suspended (this exists to allow us to preserve the original value
// during a replay).
private static ThreadLocal<Boolean> autoUpdateEnabled = ThreadLocal.withInitial(() -> true);
DateTime timestamp;
/** Returns the timestamp, or {@code START_OF_TIME} if it's null. */
@@ -40,4 +45,30 @@ public class UpdateAutoTimestamp extends ImmutableObject {
instance.timestamp = timestamp;
return instance;
}
// TODO(b/175610935): Remove the auto-update disabling code below after migration.
/** Class to allow us to safely disable auto-update in a try-with-resources block. */
public static class DisableAutoUpdateResource implements AutoCloseable {
DisableAutoUpdateResource() {
autoUpdateEnabled.set(false);
}
@Override
public void close() {
autoUpdateEnabled.set(true);
}
}
/**
* Resturns a resource that disables auto-updates on all {@link UpdateAutoTimestamp}s in the
* current thread, suitable for use with in a try-with-resources block.
*/
public static DisableAutoUpdateResource disableAutoUpdate() {
return new DisableAutoUpdateResource();
}
public static boolean autoUpdateEnabled() {
return autoUpdateEnabled.get();
}
}

View File

@@ -24,7 +24,6 @@ import static google.registry.util.CollectionUtils.nullToEmptyImmutableCopy;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Sets;
@@ -47,11 +46,12 @@ import google.registry.model.domain.rgp.GracePeriodStatus;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
import google.registry.persistence.BillingVKey.BillingEventVKey;
import google.registry.persistence.BillingVKey.BillingRecurrenceVKey;
import google.registry.persistence.VKey;
import google.registry.persistence.WithLongVKey;
import google.registry.schema.replay.DatastoreAndSqlEntity;
import google.registry.schema.replay.DatastoreEntity;
import google.registry.schema.replay.SqlEntity;
import google.registry.schema.replay.DatastoreOnlyEntity;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
@@ -121,13 +121,11 @@ public abstract class BillingEvent extends ImmutableObject
String clientId;
/** Revision id of the entry in DomainHistory table that ths bill belongs to. */
// TODO(shicong): Add foreign key constraint when DomainHistory table is generated
@Ignore
@Column(nullable = false)
Long domainHistoryRevisionId;
/** ID of the EPP resource that the bill is for. */
// TODO(shicong): Add foreign key constraint when we expand DatastoreHelp for Postgresql
@Ignore
@Column(nullable = false)
String domainRepoId;
@@ -291,10 +289,10 @@ public abstract class BillingEvent extends ImmutableObject
@javax.persistence.Index(columnList = "eventTime"),
@javax.persistence.Index(columnList = "billingTime"),
@javax.persistence.Index(columnList = "syntheticCreationTime"),
@javax.persistence.Index(columnList = "allocation_token_id")
@javax.persistence.Index(columnList = "allocationToken")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_event_id"))
@WithLongVKey
@WithLongVKey(compositeKey = true)
public static class OneTime extends BillingEvent implements DatastoreAndSqlEntity {
/** The billable value. */
@@ -334,10 +332,7 @@ public abstract class BillingEvent extends ImmutableObject
/**
* The {@link AllocationToken} used in the creation of this event, or null if one was not used.
*
* <p>TODO(shicong): Add foreign key constraint when AllocationToken schema is generated
*/
@Column(name = "allocation_token_id")
@Index
@Nullable
VKey<AllocationToken> allocationToken;
@@ -471,7 +466,7 @@ public abstract class BillingEvent extends ImmutableObject
@javax.persistence.Index(columnList = "recurrence_time_of_year")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_recurrence_id"))
@WithLongVKey
@WithLongVKey(compositeKey = true)
public static class Recurring extends BillingEvent implements DatastoreAndSqlEntity {
/**
@@ -566,7 +561,7 @@ public abstract class BillingEvent extends ImmutableObject
@javax.persistence.Index(columnList = "billingTime")
})
@AttributeOverride(name = "id", column = @Column(name = "billing_cancellation_id"))
@WithLongVKey
@WithLongVKey(compositeKey = true)
public static class Cancellation extends BillingEvent implements DatastoreAndSqlEntity {
/** The billing time of the charge that is being cancelled. */
@@ -579,8 +574,7 @@ public abstract class BillingEvent extends ImmutableObject
* <p>Although the type is {@link Key} the name "ref" is preserved for historical reasons.
*/
@IgnoreSave(IfNull.class)
@Column(name = "billing_event_id")
VKey<BillingEvent.OneTime> refOneTime = null;
BillingEventVKey refOneTime = null;
/**
* The recurring billing event to cancel, or null for non-autorenew cancellations.
@@ -588,15 +582,14 @@ public abstract class BillingEvent extends ImmutableObject
* <p>Although the type is {@link Key} the name "ref" is preserved for historical reasons.
*/
@IgnoreSave(IfNull.class)
@Column(name = "billing_recurrence_id")
VKey<BillingEvent.Recurring> refRecurring = null;
BillingRecurrenceVKey refRecurring = null;
public DateTime getBillingTime() {
return billingTime;
}
public VKey<? extends BillingEvent> getEventKey() {
return firstNonNull(refOneTime, refRecurring);
return firstNonNull(refOneTime, refRecurring).createVKey();
}
/** The mapping from billable grace period types to originating billing event reasons. */
@@ -663,12 +656,12 @@ public abstract class BillingEvent extends ImmutableObject
}
public Builder setOneTimeEventKey(VKey<BillingEvent.OneTime> eventKey) {
getInstance().refOneTime = eventKey;
getInstance().refOneTime = BillingEventVKey.create(eventKey);
return this;
}
public Builder setRecurringEventKey(VKey<BillingEvent.Recurring> eventKey) {
getInstance().refRecurring = eventKey;
getInstance().refRecurring = BillingRecurrenceVKey.create(eventKey);
return this;
}
@@ -687,8 +680,8 @@ public abstract class BillingEvent extends ImmutableObject
/** An event representing a modification of an existing one-time billing event. */
@ReportedOn
@Entity
@WithLongVKey
public static class Modification extends BillingEvent implements DatastoreEntity {
@WithLongVKey(compositeKey = true)
public static class Modification extends BillingEvent implements DatastoreOnlyEntity {
/** The change in cost that should be applied to the original billing event. */
Money cost;
@@ -750,11 +743,6 @@ public abstract class BillingEvent extends ImmutableObject
.build();
}
@Override
public ImmutableList<SqlEntity> toSqlEntities() {
return ImmutableList.of(); // not persisted in SQL
}
/** A builder for {@link Modification} since it is immutable. */
public static class Builder extends BillingEvent.Builder<Modification, Builder> {

View File

@@ -20,15 +20,16 @@ import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.Id;
import com.googlecode.objectify.annotation.Parent;
import google.registry.model.ImmutableObject;
import javax.persistence.MappedSuperclass;
import javax.persistence.Transient;
/** A singleton entity in Datastore. */
@MappedSuperclass
public abstract class CrossTldSingleton extends ImmutableObject {
public static final long SINGLETON_ID = 1; // There is always exactly one of these.
public static final long SINGLETON_ID = 1; // There is always exactly one of these.
@Id
long id = SINGLETON_ID;
@Id @Transient long id = SINGLETON_ID;
@Parent
Key<EntityGroupRoot> parent = getCrossTldKey();
@Transient @Parent Key<EntityGroupRoot> parent = getCrossTldKey();
}

View File

@@ -21,7 +21,6 @@ import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableList;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Id;
@@ -29,8 +28,7 @@ import com.googlecode.objectify.annotation.Parent;
import google.registry.model.ImmutableObject;
import google.registry.model.UpdateAutoTimestamp;
import google.registry.model.registry.Registry;
import google.registry.schema.replay.DatastoreEntity;
import google.registry.schema.replay.SqlEntity;
import google.registry.schema.replay.DatastoreOnlyEntity;
import java.util.List;
import org.joda.time.DateTime;
@@ -40,7 +38,7 @@ import org.joda.time.DateTime;
* scoped on {@link EntityGroupRoot}.
*/
@Entity
public class Cursor extends ImmutableObject implements DatastoreEntity {
public class Cursor extends ImmutableObject implements DatastoreOnlyEntity {
/** The types of cursors, used as the string id field for each cursor in Datastore. */
public enum CursorType {
@@ -137,11 +135,6 @@ public class Cursor extends ImmutableObject implements DatastoreEntity {
return CursorType.valueOf(String.join("_", id.subList(1, id.size())));
}
@Override
public ImmutableList<SqlEntity> toSqlEntities() {
return ImmutableList.of(); // Cursors are not converted since they are ephemeral
}
/**
* Checks that the type of the scoped object (or null) matches the required type for the specified
* cursor (or null, if the cursor is a global cursor).

View File

@@ -14,13 +14,11 @@
package google.registry.model.common;
import com.google.common.collect.ImmutableList;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Id;
import google.registry.model.BackupGroupRoot;
import google.registry.schema.replay.DatastoreEntity;
import google.registry.schema.replay.SqlEntity;
import google.registry.schema.replay.DatastoreOnlyEntity;
/**
* The root key for the entity group which is known as the cross-tld entity group for historical
@@ -37,7 +35,7 @@ import google.registry.schema.replay.SqlEntity;
* entity group for the single namespace where global data applicable for all TLDs lived.
*/
@Entity
public class EntityGroupRoot extends BackupGroupRoot implements DatastoreEntity {
public class EntityGroupRoot extends BackupGroupRoot implements DatastoreOnlyEntity {
@SuppressWarnings("unused")
@Id
@@ -47,9 +45,4 @@ public class EntityGroupRoot extends BackupGroupRoot implements DatastoreEntity
public static Key<EntityGroupRoot> getCrossTldKey() {
return Key.create(EntityGroupRoot.class, "cross-tld");
}
@Override
public ImmutableList<SqlEntity> toSqlEntities() {
return ImmutableList.of(); // not persisted in SQL
}
}

View File

@@ -14,7 +14,6 @@
package google.registry.model.contact;
import com.google.common.collect.ImmutableList;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.EntitySubclass;
import google.registry.model.ImmutableObject;
@@ -95,9 +94,9 @@ public class ContactHistory extends HistoryEntry implements SqlEntity {
}
/** Creates a {@link VKey} instance for this entity. */
@SuppressWarnings("unchecked")
public VKey<ContactHistory> createVKey() {
return VKey.create(
ContactHistory.class, new ContactHistoryId(getContactRepoId(), getId()), Key.create(this));
return (VKey<ContactHistory>) createVKey(Key.create(this));
}
@PostLoad
@@ -111,12 +110,12 @@ public class ContactHistory extends HistoryEntry implements SqlEntity {
// In Datastore, save as a HistoryEntry object regardless of this object's type
@Override
public ImmutableList<DatastoreEntity> toDatastoreEntities() {
return ImmutableList.of(asHistoryEntry());
public Optional<DatastoreEntity> toDatastoreEntity() {
return Optional.of(asHistoryEntry());
}
/** Class to represent the composite primary key of {@link ContactHistory} entity. */
static class ContactHistoryId extends ImmutableObject implements Serializable {
public static class ContactHistoryId extends ImmutableObject implements Serializable {
private String contactRepoId;
@@ -125,7 +124,7 @@ public class ContactHistory extends HistoryEntry implements SqlEntity {
/** Hibernate requires this default constructor. */
private ContactHistoryId() {}
ContactHistoryId(String contactRepoId, long id) {
public ContactHistoryId(String contactRepoId, long id) {
this.contactRepoId = contactRepoId;
this.id = id;
}

View File

@@ -75,7 +75,9 @@ public class DomainBase extends DomainContent
}
@ElementCollection
@JoinTable(name = "DomainHost")
@JoinTable(
name = "DomainHost",
indexes = {@Index(columnList = "domain_repo_id,host_repo_id", unique = true)})
@Access(AccessType.PROPERTY)
@Column(name = "host_repo_id")
public Set<VKey<HostResource>> getNsHosts() {

Some files were not shown because too many files have changed in this diff Show More