1
0
mirror of https://github.com/google/nomulus synced 2026-01-27 16:12:19 +00:00

Compare commits

..

127 Commits

Author SHA1 Message Date
Lai Jiang
9304e2f421 Enable Java 11 features (#737)
* Enable Java 11 features

As of this commit Java 11 must be used to build. The generated bytecode
is still at Java 8 due to App Engine task queue limit.

Also fixed a bug where the included google-java-format jar file is not
used, requiring the user to install it separately.

See: https://cloud.google.com/appengine/docs/standard/java/taskqueue/push
2020-08-03 16:08:29 -04:00
Lai Jiang
5f2be914a1 Use Java 11 in GCB to build release candidates (#736) 2020-08-03 13:13:08 -04:00
sarahcaseybot
4ad7f9734d Add IntervalDescriptor and change DurationConverter to use Interval datatype (#653)
* Add use of interval data type

* Add support for Millis

* Use Java-object type

* Change column type for relock_duration

* add years and months

* Add tests for hours, minutes, and seconds

* Add javadoc describing how joda duration is stored

* Add test for lots of days
2020-08-03 12:42:34 -04:00
Lai Jiang
90d080d42f Delete screenshots folder in fragile test first (#735)
It is not clear to me why this folder is deleted in standardTest first,
presumably this is done when the webdriver tests were part of
standardTest?

Also not clear to me is why screenshots are only deleted in presubmits
but not locally. At least on my workstation standardTest runs after
fragileTest, just like in kokoro, so it should have deleted the
screenshots
folder, but it didn't.

Regardless, the correct thing to do is to delete this folder first in
fragileTest so that it doesn't interfere with the results.
2020-08-03 11:46:34 -04:00
Ben McIlwain
e95ce30fa6 Upgrade prober & proxy packages from JUnit 4 to JUnit 5 (#734)
* Upgrade prober & proxy packages from JUnit 4 to JUnit 5
2020-08-01 11:34:58 -04:00
Lai Jiang
090c233592 Some more changes to make it possible to use Java 11 to compile (#732)
Add the class paths of the source files generated by annotation processors to
the javadoc task's class path so that it doesn't complain about missing
Dagger classes.

Also remove empty <p> tags in all generated source files, because jaxb
genrerates files in multiple locations.

Lastly, for unkown reasons when the source level is set to > 8, the core
subproject throws a warning about a Gradle internal annotator processor
that only supports up to Java 8 and cause the Java compliation to fail because
we set -Werror on all java compliation tasks. I don't think there is a
strong reason that we set -Werror anyway, so this commit removes it.
2020-07-31 17:21:28 -04:00
Ben McIlwain
16a31e460c Get rid of all remaining JUnit 4 usages except in prober & proxy (#731)
* Get rid of all remaining JUnit 4 usages except in prober & proxy subprojects

Caveat: Test suites aren't yet implemented in JUnit 5 so we still use the ones
from JUnit 5 in the core subproject.

* Fix some build errors
2020-07-30 20:29:00 -04:00
Lai Jiang
a02b67caf5 Migrate the documentation package to Java 11 (#729)
* Migrate the documentation package to Java 11

The old Doclet API is deprected and removed in Java 12. This commit
changes the documentation package to use the new recommended API.
However it is not a drop-in replacement and there are non-idiomatic
usages all over the place. I think it is eaiser to keep the current code
logic and kind of shoehorn in the new API than starting afresh as the
return on investment of a do-over is not great.

Also note that the docs package is disabled as of this commit because we
are still using Java 8 to compile which lacks the new API. Once we
switch our toolchains to Java 11 (but still compiling Java 8 bytecode)
we can re-enable this package.

TESTED=ran `./gradlew :docs:test` locally with the documentation package
enabled.
2020-07-30 17:12:33 -04:00
Lai Jiang
bf20a8ef96 Remove empty <p> tag from generated javadoc (#730) 2020-07-30 16:31:25 -04:00
Lai Jiang
8750c07fef Clean up generated sources configuration (#724)
1. The Gradle apt plugin is no longer needed to process annotations.
2. Without the apt plugin, Gralde puts the source files generated by
   annotation processors in build/generated/sources/annotationProcessor.
3. Change the location of custom generated files to be consistent.
4. Fix a javadoc formatting error.
2020-07-30 15:43:42 -04:00
Weimin Yu
7821de67f8 Run InitSqlPipeline (#727)
* Run InitSqlPipeline

Added the main() method to InitSqlPipeline.

Added a Gradle task to run InitSqlPipeline from command line. This
task is meant for testing and experiments.

Corrected the file name prefix of Datastore export files. Should
be 'output-', defined as 'input-'.
2020-07-30 14:28:53 -04:00
Shicong Huang
1022817384 Fix broken RegistryCli (#728) 2020-07-30 14:10:09 -04:00
Ben McIlwain
8c04bf2599 Rename InjectRule and remove some JUnit4-only dependencies (#726)
* Rename InjectRule and remove some deps

* Merge remote-tracking branch 'upstream/master' into final-core-tests
2020-07-29 20:27:47 -04:00
Ben McIlwain
34116e3811 Clarify un-avail reason on allocation-token-reserved domains (#725)
Apparently, in domain check responses, `avail=false, reason=Allocation token
required` was not sufficiently understood by all registrars. This changes it to
`avail=false, reason=Reserved; alloc. token required` to hopefully make it
crystal clear that the domain in question is reserved, i.e. if you were supposed
to be able to register this domain you'd already know it because we'd have
already given you the requisite allocation token.
2020-07-29 17:13:38 -04:00
Lai Jiang
d180ef43ac Move the documentation package to its own subproject (#722)
This makes it easier to later migrate the package to Java 11. If we move
and migrate in a single PR, because of the portion of the contents that
s changed, git will have trouble recognizing that some files are
renamed *and* modified and treat them as distinct files, making code
review difficult.
2020-07-29 13:41:02 -04:00
Michael Muller
f55270c46f Integrate transaction persistence into JpaTM (#717)
* Integrate transaction persistence into JpaTM

Store the serialized transaction whenever we commit from the JPA transaction
manager.  This change also adds:

-   The Transaction table.
-   The TransactionEntity which is stored in it.
-   Changes to the test infrastructure to register the TransactionEntity for
    tests where we don't load the nomulus schema.
-   A new configuration variable to allow us to turn the transaction
    persistence functionality on and off (default is "off").

* Changes for review.

* Incremented sequence number of flyway file
2020-07-28 19:23:44 -04:00
Ben McIlwain
d6d9874da1 Upgrade App Engine and webserver tests from JUnit 4 to 5 (#720)
* Upgrade App Engine and webserver tests from JUnit 4 to 5

* Fix most errors

* Merge branch 'master' into junit5ification

* Fix test server by extracting non-test setup/tear-down

* Merge branch 'master' into junit5ification

* Fix backup tests

* Don't createFile(); asCharSink does it

* Increase the timeout for all WebDriver tests to 60s (helps w/ flakiness)
2020-07-28 14:18:16 -04:00
gbrodman
e0d04cec4f Set up deployment of the Spec11 pipeline with JPA TM (#716)
* Set up deployment of the Spec11 pipeline with JPA TM

* Remove unnecessarily pipeline options setting

* Use enviroment name in BeamJpaModuleTest

* Fix checkstyle error
2020-07-27 21:04:52 -04:00
Michael Muller
0ce431212e Add the :nom:generate_golden_schema pseudo-task (#718)
Add a "pseudo-task" in nom_build to do the three step process of generating
the golden schema.  In the course of this, add support for pseudo-tasks in
general, improve the database directory readme and make nom_build not call
gradlew if there are no tasks.
2020-07-27 18:33:16 -04:00
gbrodman
32868b3ab8 Run the (Un)lockDomainCommand in an outer JPA txn (#688)
* Run the (Un)lockDomainCommand in an outer JPA txn

There are a couple things going on here in this commit.

First, we add an external JPA transaction in the
LockOrUnlockDomainCommand class. This doesn't appear to do much, but it
avoids a situation similar to deadlock if an error occurs in Datastore
when saving the domain object. Specifically, DomainLockUtils relies on
the fact that any error in Datastore will be re-thrown in the JPA
transaction, meaning that any Datastore error will back out of the SQL
transaction as well. However, this is no longer true if we are already
in a Datastore transaction when calling DomainLockUtils (unless, again,
we are also in a JPA transaction). Basically, we require that the outer
transaction is the JPA one.

Secondly, this just allows for more breakglass operations in the lock or
unlock domain commands -- in a situation where things possibly go
haywire, we should allow admins to make sure with certainty that a
domain is locked or unlocked.

* Add more robustness and tests for admins locking locked domains

* Fix expected exception message in tests
2020-07-27 18:16:24 -04:00
Shicong Huang
0ecc20b48c Rename a V40 flyway file to V41 to resolve conflict (#719) 2020-07-27 15:16:01 -04:00
Shicong Huang
c65af4b480 Add remaining columns to Domain's SQL schema (#702) 2020-07-27 13:32:39 -04:00
Legina Chen
3a15a8bdc7 Drop foreign key constraint for Registrar table (#715) 2020-07-27 09:05:30 -07:00
Weimin Yu
9806fab880 Use rearranged sql credentials in flyway task (#712)
* Use rearranged sql credentials in flyway task

Let the flyway tasks use the sql credential files set up for BEAM
pipelines.

Credential files have been created for each environment in GCS
at gs://${project}-beam/cloudsql/admin_credential.enc. All
project editors have access to this file, including the Dataflow
control service account.

Alpha and crash use the 'nomulus-tools-key' in their own project to
decrypt the credential file.

Sandbox and production use the 'nomulus-tools-key' in
domain-registry-dev to decrypt the credential file.

Note that this setup is temporary. It will become obsolete once
we migrate to Cloud Secret Manager for secret storage.
2020-07-24 15:32:01 -04:00
Weimin Yu
6591e0672a End-to-end Datastore to SQL pipeline (#707)
* End-to-end Datastore to SQL pipeline

Defined InitSqlPipeline that performs end-to-end migration from
a Datastore backup to a SQL database.

Also fixed/refined multiple tests related to this migration.
2020-07-24 09:57:43 -04:00
Ben McIlwain
91b7d92cf8 Upgrade TestPipeline extension from JUnit 4 to 5 2020-07-23 21:21:58 -04:00
Ben McIlwain
33910613da Get presubmits passing
This involves Guava -> Java 8 util migrations and fixing the license header.
2020-07-23 21:21:58 -04:00
Ben McIlwain
1fde678250 Copy TestPipeline rule from Apache Beam project into our codebase
This is copied in here with the absolute minimum # of modifications required
(just a rename to JUnit 5 format and some small fixes required to enable
compilation to be successful).

This is in preparation for the next commit where I'll convert this Rule into a
JUnit 5 extension, which is the entire goal here. But I wanted to get the code
from Apache Beam in with the maximum possible fidelity so that my changes will
be in a separate commit and will thus be obvious.

Note that we do unfortunately need to modify/rewrite the Rule itself; merely
wrapping it in some manner isn't possible.
2020-07-23 21:21:58 -04:00
gbrodman
8d56577653 Don't run presubmits over the .git folder (#711) 2020-07-23 18:12:34 -04:00
Ben McIlwain
3891d411de Upgrade most of remaining tests from JUnit 4 to JUnit 5 (#708) 2020-07-23 15:43:59 -04:00
gbrodman
cadecb15d8 Rename the email field in UI and include rlock email if it exists (#697)
* Rename the email field in UI and include rlock email if it exists

* Change the capitalization of fields and titles and add a description
2020-07-23 14:30:12 -04:00
gbrodman
9b7f6ce500 Fix some SQL credential issues identified when deploying Beam pipelines (#706)
* Fix some SQL credential issues identified when deploying Beam pipelines

There are two issues fixed here.
1. Without calling `FileSystems.setDefaultPipelineOptions(PipelineOptionsFactory.create()), the Nomulus tool doesn't know how to handle gs:// scheme files. Thus, if you try to deploy (for instance) the Spec11 pipeline using a GCS credential file, it fails.
2. There was a misunderstanding before about what the credential file
actually refers to -- there is a credential file in JSON format that is
used for gcloud authorization, and there is a space-delimited SQL access
info file that has the instance name, username, and password. These are
separate options and should have separate command-line params.

* Actually we don't need this for remote deployment
2020-07-22 16:52:31 -04:00
Ben McIlwain
cd23748fe8 Upgrade rest of tools test classes to JUnit 5 (#705) 2020-07-22 11:09:21 -04:00
Ben McIlwain
cf41f5d354 Upgrade all remaining flows tests to JUnit 5 (#704) 2020-07-21 19:52:33 -04:00
Ben McIlwain
9a5ba249db Upgrade converters/TMCH/RDAP to JUnit 5 (#703)
Also renames some existing Rules to Extensions (and removes JUnit 4 features
from them entirely if no longer being used).
2020-07-21 18:48:41 -04:00
Shicong Huang
f5186f8476 Merge two PremiumList entities (#690) 2020-07-21 18:18:52 -04:00
Lai Jiang
4e0ca19d2e Remove IDN elements from BRDA (#670)
Also added unit tests for RdeStagingReducer.
2020-07-21 15:29:32 -04:00
Ben McIlwain
c812807ab3 Upgrade mapreduce and DNS tests from JUnit 4 to JUnit 5 (#701)
* Upgrade mapreduce and DNS tests from JUnit 4 to JUnit 5

* Merge branch 'master' into junit5-batch-and-dns
2020-07-20 21:33:24 -04:00
Ben McIlwain
9edb43f3e4 Upgrade command test classes from JUnit 4 to JUnit 5 (#700)
* Convert first batch of command tests to JUnit 5

* Upgrade rest of command tests to JUnit 5

* Migrate the last few test classes
2020-07-20 20:45:52 -04:00
gbrodman
b721533759 Create an ImmutableObjectSubject for comparing SQL objects (#695)
* Create an ImmutableObjectSubject for comparing SQL objects

Many times, when comparing objects that are loaded in from / saved to
SQL in tests, there are some fields we don't care about. Specifically,
we might not care about the last update time, revision ID, or other
things like that that are autoassigned by the DB. If we use this, we can
ignore those fields while still comparing the other ones.

* Create an ImmutableObject Correspondence for more flexible usage
2020-07-20 13:14:09 -04:00
gbrodman
ce35f6bc93 Include the user's registry lock email in the lock/unlock modal (#696)
* Include the user's registry lock email in the lock/unlock modal
2020-07-20 12:01:34 -04:00
gbrodman
f7a67b7676 Add a 'Host' parameter to the relock action enqueuer (#699)
* Add a 'Host' parameter to the relock action enqueuer

I believe this is why we are seeing 404s currently -- we should be
specifying the backend host as the target like we do for the
resave-entity async action.
2020-07-17 15:35:44 -04:00
gbrodman
4438944900 Validate potentially-invalid domain names when (un)locking domains (#698)
* Validate potentially-invalid domain names when (un)locking domains
2020-07-17 12:05:19 -04:00
Legina Chen
a22998e1bc Change clientId to registrarId to resolve the bug wrt the mismatch of the variable names (#692) 2020-07-15 11:03:52 -07:00
Weimin Yu
03d02ab299 Fix JpaIntegrationRule in JUnit4 (#687)
* Fix JpaIntegrationRule in JUnit4

Made DatastoreExtension a JUnit4 Rule.

Nomulus model objects need Datastore API when manipulating Ofy keys.
As a result, JpaIntegrationTestRule must be used with AppEngineRule
or DatastoreExtension.

Also fixed WriteToSqlTest, which is the only JUnit4 test that uses
JpaIntegrationTestRule.
2020-07-15 10:42:33 -04:00
Lai Jiang
47f65f70ab Fix a typo (#689) 2020-07-15 10:39:11 -04:00
Weimin Yu
1aa1f351bf Run rdeStaging twice daily in Sandbox (#684)
* Run rdeStaging twice daily in Sandbox

This will allow the cursor to catch up to current date if
it somehow falls behind.
2020-07-14 14:54:34 -04:00
Weimin Yu
94c8c6b9f3 Add lastUpdateTime column to epp resources (#683)
* Add lastUpdateTime column to epp resources

Property was inadvertently left out.

Renamed getter and setter to match the property name.

Added a test helper to compare EppResources while ignoring
lastUpdateTime, which changes every time an instance is persisted.
2020-07-14 14:53:05 -04:00
gbrodman
e74a9e6f02 Allow overrides of ContactBase methods (#681)
Hibernate might (will?) need to override these, so they shouldn't be
final.
2020-07-14 14:47:47 -04:00
gbrodman
37d3cc44b4 Fix small naming issue in a test (#685) 2020-07-14 13:57:44 -04:00
Lai Jiang
c844c8e9b1 Add the ability to parse PKCS#8 private key in PEM file (#682) 2020-07-14 11:20:00 -04:00
gbrodman
f747610533 Include the relock action in the web.xml routing file (#680) 2020-07-13 21:57:35 -04:00
Shicong Huang
e1db357fc3 Merge two reserved list entities (#616)
* Merge reserved list

* Replace INSTANCE with getInstance()

* Fix broken test

* Rebase on master

* Simplify class
2020-07-13 13:40:34 -04:00
Weimin Yu
ba1915e271 Write one PCollection to SQL (#664)
* Write one PCollection to SQL

Defined a transform that writes a PCollection of entities to SQL using
JPA. Allows configuring parallelism level and batch size.
2020-07-13 13:34:01 -04:00
Shicong Huang
58618a274e Add two folders of auto-generated Java classes to .gitignore (#679) 2020-07-13 10:09:56 -04:00
Lai Jiang
e4d0571125 Increase the maximum number of nodes in a nood pool to 15 (#672) 2020-07-10 21:54:18 -04:00
Ben McIlwain
4cb88ab6e7 Convert RDE tests (and some test rules) from JUnit 4 to JUnit 5 (#677)
* Add JUnit Params and start using it

* Convert rest of RDE tests

* Don't check headers for generated tests

* Expand visibility to fix build breakage

* Bump JUnit versions to 5.6.2
2020-07-10 21:32:36 -04:00
gbrodman
987f390ff7 Run 'npm audit fix' to fix low-severity vulnerabilities in packages (#676) 2020-07-10 15:57:59 -04:00
Ben McIlwain
ca756e14e6 Migrate all model tests from JUnit 4 to JUnit 5 (#675)
* Make first handful of tests JUnit 5

* Migrate rest of model package to JUnit 5
2020-07-10 14:56:28 -04:00
Ben McIlwain
caa0cd9d61 Add a "coreDev" gradle target (#667)
* Add a "buildFmt" gradle target

This does the same thing as the automatic Java build target, except instead of
failing if the code formatting isn't correct, it just automatically reformats as
necessary and continues on.

* Remove unnecessary mustRunAfters

* Make it run tests too, and add :taskTree task

* Rename task to coreDev and remove run afters

* Add task tree dependency

* Actually that may not be necessary
2020-07-10 10:03:59 -04:00
Legina Chen
7806cc7edb Add domainRepoId to Subdomain class (#674)
* Change Subdomain class to contain domainRepoId

* Remove jpaTm from Spec11PipelineTest and change clientId -> registrarId

* Remove 'client' from a comment

* Include changes to Spec11Pipeline

* add SafeBrowsingTransforms

* Run style
2020-07-09 16:26:35 -07:00
Lai Jiang
0964fdf1dc Upgrade to Gradle 6.5.1 (#673) 2020-07-09 14:04:22 -04:00
gbrodman
d17ec1fcb1 Use an enum instead of boolean in EntityTestCase constructor (#669)
* Use an enum instead of boolean in EntityTestCase constructor

It's more clear to use an enum rather than just a simple boolean

* Add Javadoc and make the enum name more verbose
2020-07-09 12:54:32 -04:00
Ben McIlwain
fac5987c13 Double the # of pubapi instances to better handle traffic spikes (#671)
* Double the # of pubapi instances to better handle traffic spikes

We may also consider switching to an automatic scaling mode soon, on the hope
that it's working better than the last time we tried it (it would help to keep
resource costs down at least).
2020-07-09 11:52:15 -04:00
Ben McIlwain
a3319e0026 Upgrade flow test classes to JUnit 5 (#666)
Most of the diffs are visibility changes.

Also deletes ShardableTestCase, which was only necessary because of Blaze (and
possible Bazel) limitations.
2020-07-08 14:08:05 -04:00
Weimin Yu
5578464e06 Make sure uncommitted txn is rolled back (#665)
* Make sure uncommit txn is rolled back

The try block around commit that catches RuntimeException should also
catch Error, which is also unchecked.
2020-07-06 17:39:13 -04:00
gbrodman
c24a61f813 Refactor ContactResource into ContactBase and create ContactHistory (#634)
* Create ContactHistory class + table

This is similar to #587, but with contacts instead of hosts.

This also includes a couple cleanups for HostHistoryTest and RegistryLockDaoTest, just making code more proper (we shouldn't be referencing constant revision IDs when using a sequence that is used by multiple classes, and RLDT can extend EntityTest)

Note as well that we set ContactHistory to use the same revision ID sequence as HostHistory.

* Move ContactResource -> ContactBase

* Alter ContactBase and ContactResource
2020-07-06 12:52:16 -04:00
gbrodman
806f3b2456 Verify that the RegistryLock input has the correct registrar ID (#661)
* Verify that the RegistryLock input has the correct registrar ID

We already verify (correctly) that the user has access to the registrar
they specify, but nowhere did we verify that the registrar ID they used
is actually the current sponsor ID for the domain in question. This is
an oversight caused by the fact that our testing framework only uses
admin accounts, which by the nature of things have access to all
registrars and domains.

In addition, rename "clientId" to "registrarId" in the RLPA object

* Change the wording on the incorrect-registrar message
2020-07-05 22:31:14 -04:00
gbrodman
333170a724 Allow users the option of seeing their registry lock password (#663)
* Allow users the option of seeing their registry lock password

Only when entering it for the first time, of course.
2020-07-05 20:08:22 -04:00
Lai Jiang
47eeb8c4e4 Output PO number in detailed report (#659)
* Output PO number in detailed report

The PO number header was added during the beam migration but we forgot
to print the actual data in the corresponding column. This resulted in a
misalignment of columns in the detailed report.

This PR fixes it. Note that we cannot drop PO number from the header (as is not
useful in the detailed report) because the header represents all fields
that are to be parsed from the SQL query results, and PO number *is*
needed when generating the invoice itself. By dual-purposing the header
(both as the required fields in the parser and the first line in the
detailed report) we have to include the value of PO number in the
detailed report CSV as well.
2020-07-01 19:09:05 -04:00
Shicong Huang
391929b518 Expand AckPollMessagesCommand to ack PollMessage.Autorenew (#647)
* Expand AckPollMessagesCommand to ack PollMessage.Autorenew

* Rebase on master and address comment

* Resolve comments
2020-07-01 15:06:35 -04:00
gbrodman
7f62b7a89c Include the registry lock email in the JS object as a sensitive field (#658)
* Include the registry lock email in the JS object as a sensitive field

* Change wording of exceptions to be more consistent
2020-07-01 13:05:21 -04:00
gbrodman
a1da32bfde Disambiguate injected Cloud SQL parameter names (#657)
* Disambiguate injected Cloud SQL parameter names

This allows us to also inject the BeamJpaModule into RegistryTool, which
allows us to use the SocketJpaTransactionManager in Beam pipelines.

Some side effects of this include:
- duplication of KMS connections -- one standard, one Beam
- duplication of the creation of the partial Hibernate SQL configs
- removal of ambiguity between credentialFileName, credentialFilename,
and credentialFilePath -- we now use the latter.
- Performing the credential null check when instantiating the SQL
connection rather than when instantiating the module object. See the code
comments for more details on this.

I verified that this compiles and the tests run successfully when
injecting a @SocketFactoryJpaTm into a Beam pipeline.

* Remove two unnecessary config points and change the name of two params

* Use @Config instead of @Named and change the pool size

* Replace non-visible link with code
2020-07-01 11:55:21 -04:00
Weimin Yu
1961a5759d Load Datastore snapshot from backup files (#660)
* Load Datastore snapshot from backup files

Defined a composite transform that loads from a Datastore export and
concurrent CommitLog files, identify entities that still exist at the
end of the time window, and resolve their latest states in the window.
2020-07-01 09:58:42 -04:00
Weimin Yu
d065ff63fc Exclude Test/Monitoring Registrars from escrow (#655)
* Exclude Test/Monitoring Registrars from escrow

Registrars used for testing and monitoring should not be included
in Data escrow. They also lack the required ianaIdentifier property
and would fail ICANN data validation.

Note that since alpha and crash environments have bad data that
break the RDE process, we need to verify this change in Sandbox.
2020-06-26 19:11:22 -04:00
Michael Muller
07ff6279bb Make EppResource.loadCached() use batched fetch (#652)
* Make EppResource.loadCached() use batched fetch

Use a batched fetch (ofy().load().keys(...)) from datastore in
EppResource.loadCached().

To support this, convert TransactionManager.load(Iterable<VKey>) to accept the
more flexible generic parameters and return a map.

* Simplify datastore key streaming

* Changes requested in review.
2020-06-26 13:50:02 -04:00
Legina Chen
5c5b6b20ce Allow multiple threat types in the Spec11ThreatMatch table (#650)
* Update to generic Spec11ThreatMatch table

* Fix SQL syntax

* Make changes to the schema and add a test for null and empty threatTypes

* Fix a small typo

* Change the exception thrown with illegal arguments

Change the import for isNullOrEmpty

* Fix import for checkArgument

* Added a threat to test multiple threat types
2020-06-26 10:35:00 -07:00
Ben McIlwain
74b2de5c35 Make ImmutableMap Stream collect()ion nicer (#654)
This adds an entriesToImmutableMap() collector that can be used in place of
toImmutableMap(Map.Entry::getkey, Map.Entry::getValue()).

It also fixes up some existing calls that use toImmutableMap() when terser
alternatives exist.
2020-06-26 11:57:26 -04:00
Ben McIlwain
fba6804d3b Properly handle restore fees on domain checks (#646)
* Properly handle restore command fees for domain checks

* Get tests working and handle fee classes better

* Remove unused ImmutableSet imports

* Fix code review concerns, mostly surrounding immutability

* Rename more targetIds and make them immutable too

* Merge remote-tracking branch 'upstream/master' into domain-check-restore-fees

* Fix Javadoc formatting
2020-06-26 10:59:46 -04:00
Lai Jiang
db5311075d Patch terraform changes made internally (#651)
There were several LSC that made some formatting changes to our .tf
files. Export these changes externally for consistency.
2020-06-25 13:59:37 -04:00
Ben McIlwain
6e26dacdff Make nomulus compatible with Java 11 (#649)
* Make nomulus compatible with Java 11

This fixes the double-spacing bug with logged EPP XML on App Engine that started
appearing after App Engine switching from using Java 8 to Java 11. Java 9 made
some changes to XML Transformer classes that needed a little bit of work to
accommodate.

This also fixes the unit tests that were failing in Java 11 (all of which were
related to said XML Transformer changes).

* Make code review changes
2020-06-25 13:17:22 -04:00
Weimin Yu
2e5466f32f Fix flaky tests that depends on order of rules
SystemPropertyRule in some cases should be applied last:
when multiple rules exist and and modified property is checked
in cleanups.

ConsoleOteSetupActionTest and ConsoleRegistrarCreatorActionTest
are two such classes, and can be flaky in JUnit 4. This PR
migrates them to JUnit5 and applies ordering to extensions in
them.

Added a mockito dependency, and upgraded mockito-core to 3.3.3.

Meaningful changes: SystemPropertyRule.java and
ConsoleOteSetupActionTest.java, and
ConsoleRegistrarCreatorActionTest.java
2020-06-24 23:14:47 -04:00
Lai Jiang
a65a3fd8b7 Add a property in Registry to disable invoicing (#644)
The added property will by default be null for existing entities and false for new entities. We could write a temporary MapReduce to set it to false for all existing ones, but it seems easier to just modify the query to account for the nullness of the column. The need to check for null won't exist once we migrate the Registry entity to Cloud SQL.

TESTED=deployed to alpha, set the property for tld zombo to true and triggered an export, verified that the column is null for all rows other than the row for zombo.
2020-06-24 16:21:38 -04:00
Weimin Yu
071800ae50 Delay SQL availability check during Tool start (#643)
* Delay SQL availability check during Tool start

During Cloud SQL migration, Cloud SQL password setup in Registry
server requires a few tool commands. Such commands must be able
to run before Cloud SQL is configured.

In this change, we deplay SQL credential loading until access is
needed.
2020-06-24 14:00:36 -04:00
Shicong Huang
75659671db Fix a warning from Hibernate (#645)
Hibernate complained that ContactResource.getTransferData should not be
final:
WARN: HHH000305: Could not create proxy factory
for:google.registry.model.contact.ContactResource
org.hibernate.HibernateException: Getter methods of lazy classes cannot
be final: google.registry.model.contact.ContactResource#getTransferData
2020-06-24 10:02:42 -04:00
Michael Muller
c9d4ffb233 Convert all VKeys to symmetric VKeys (#641)
* Convert hosts to symmetric VKey

* Convert ContactResource to symmetric VKeys

* Convert BillingEvents to symmetric VKeys

* Converted PollMessage to symmetric VKeys

* Convert AllocationToken to symmetric VKeys

* Remove static methods, get everything working

* Changes requested in review.

* Removed newly introduced createOfy() calls
2020-06-24 08:02:11 -04:00
Weimin Yu
11ec4d64f2 Set up JpaTransactionManager in BEAM pipelines (#639)
* Set up JpaTransactionManager in BEAM pipelines

Added modules and utilities to create JpaTransactionManager in BEAM
pipelines.

Not wanting to set up AppEngine Remote API to access Keyring in the
Datastore, we instead use the credential files in GCS, which are
used by Spinnaker/Cloud Build and desktop access. Added utility
to download, decrypt, and parse the file. Also added/modified dagger
modules.
2020-06-23 11:04:52 -04:00
Ben McIlwain
e9ad1b6f72 Don't expect a renewal fee on restores when one isn't due (#637)
* Don't expect a renewal fee on restores when one isn't due

This is a fix on top of #632 so that domain restore commands don't require
acking an illusory renewal fee for 1 year when that isn't actually happening
(i.e. if the domain isn't yet past its original expiration).

Unfortunately, there's still a problem remaining wherein the restore fee on a
domain check will always include the additional year even if it's not
necessary. We don't have a good solution to that. Also note that in versions of
the fee extension more recent than 0.6, the fee extension cannot be passed on a
domain info command at all, so the domain check command is the only way you have
to determine what the restore fee should be. So we definitely do want to get
that right so that the apparent restore fee on a check is the same as the actual
restore fee when running the restore command. We're not quite there yet though
and it's hard to say how we will get there, since we don't load domains during a
domain check command for performance reasons yet we would need to do so in order
to know the expiration date and thus whether the additional year of renewal
should be charged.

A problem for a future PR.
2020-06-22 15:24:36 -04:00
Shicong Huang
e7db9b3c1a Add a test to verify JPA supports polimorphic VKey (#642) 2020-06-22 15:13:38 -04:00
Legina Chen
ec09226baa Create a Java entity to store ThreatMatch objects in SQL (#617)
* Squash everything together
Create SafeBrowsing_Threats table

Create LocalDateConverter and add indexes to SafeBrowsingThreats

Add indexes to SafeBrowsingThreats and make small style changes

Pass in DateTimeFormatter

Delete LocalDateConverterTest.java

Rebase

Make changes to ThreatType comments

Create LocalDateConverterTest

Add review changes

Add SafeBrowsingThreatTest

Rename repoId, refactor LocalDateConverterTest/SafeBrowsingThreatTest, add foreign keys

Change imports

Add foreign keys and rename version number

Add new generated db-schema file

Clean up null test cases

Add changes

Add foreign keys into SafeBrowsingThreatTeat and apply style checks

Add SafeBrowsingThreatTest into SqlIntegrationTestSuite and change golden file

Make small changes to SafeBrowsingThreatTest

Add tests for ForeignKeyViolations and remove setId in SafeBrowsingThreat

* Change V35 -> V36

* Add a foreign key test for a reference to Registrar

* Move some variables around
2020-06-22 12:07:59 -07:00
Weimin Yu
2c243a7d5f Add testcontainers' Junit5 support dependency (#640)
* Add testcontainers' Junit5 support dependency

Also updated guava, dagger, hibernate, postgresql, and cloud socket factory
to latest version.

Migrated PersistenceModuleTest as an example.

Real changes:
- dependencies.gradle
- core/build.gradle
- PersistenceModuleTest.java
2020-06-22 13:12:58 -04:00
Ben McIlwain
df8ce38796 Rename whitelist -> allow list (#635)
* Rename whitelist -> allow list

* Merge branch 'master' into allowlist-denylist
2020-06-18 18:36:05 -04:00
Weimin Yu
382c8014de Fix flakiness caused by ofy entity name conflict (#636)
* Fix flakiness caused by ofy entity name conflict

Overrode the 'kind' of two test entities to prevent name conflicts.
Tests are flaky because of this.

Added a check in AppEngineRuleTest for conflicting kinds.
2020-06-18 13:14:10 -04:00
Ben McIlwain
57113b4746 Show price of reserved domains when using matching allocation token (#632)
* Show price of reserved domains when using matching allocation token

When the registrar passes the fee extension, this shows the price of the domain
on a check command for reserved domains if the provided allocation token is a
match. Of course, the price is already always displayed on non-reserved names
(regardless of whether the specific provided token is a match or not).

This affects domain checks only; the price is already always displayed on domain
creates because you already by definition have access to register the domain in
question.
2020-06-18 11:57:22 -04:00
Weimin Yu
d43564172f Refactor pipline for Datastore backup loading (#628)
* Refactor pipline for Datastore backup loading

Refactored pipeline transforms.

Added testing utilities that handles assertions better.

Investigated and documented challenges in serializing Ofy entities
without side effects.
2020-06-17 22:10:14 -04:00
gbrodman
69a1d04c18 Remove 'fullyQualified' from host and domain names (#631)
* Remove 'fullyQualified' from host and domain names

We don't actually enforce that these are properly fully-qualified
(there's no dot at the end) and we specifically use the term "label
name" when talking about labels.

Note: this doesn't convert FQDN -> DN (et al) in at least two types of
cases:
1. When the term is part of the XML schema
2. When the term is used by some external system, e.g. SafeBrowsing API

* Add TODO to rename fields
2020-06-17 16:19:26 -04:00
Michael Muller
19395def5b Implement a persistable Transaction object (#614)
* Implement a persistable Transaction object

Implement Transaction, which encapsulates a sequence of datastore mutations
that can be serialized and written to the Cloud SQL Transaction table and
subsequently replayed to Datastore from a backend cron job.

* Changes requested in review

* Add a mujtation count to the persisted format
2020-06-17 14:16:48 -04:00
Shicong Huang
2f600e3e69 Implement remaining methods in JpaTransactionManager (#633) 2020-06-17 12:39:34 -04:00
Michael Muller
31841ccc55 Fix cookie processing for RDAP URL update (#630)
* Fix cookie processing for RDAP URL update

The existing code only does cookie processing on the _first_ Set-Cookie
header.  Therefore, if the "id" cookie used for authentication is defined in
anything other than the first Set-Cookie header (as it now is), we don't find
it.

Replace the cookie processing stanza with a line that processes all cookies in
all Set-Cookie headers.
2020-06-16 15:07:13 -04:00
gbrodman
4f37c65af5 Fix versioning semantic merge conflict (#629) 2020-06-16 12:51:28 -04:00
gbrodman
47178d4fb5 Add HostBase and HostHistory classes (#587)
* Add proof of concept for HostBase and HostHistory classes

* Use a PROPERTY accessor for @Ids

* Add an unused setter method for Hibernate's sake

* Refactor HostHistory

* Some responses to CR

* Fix relationship and test

* Manually manage the foreign keys for HostHistory

* Protect HostBase's builder and use text for the enum type

* Add responses to CR

- Add javadocs
- Create an ID sequence for host history objects

* Don't try to set the ID

* Use a Long and remove the setter

* Add some comments and rename a couple fields

* Don't change Datastore schema

* Use Long in the Datastore schema

* Add new createVKey method

* Add comments and rename fields

* Rename v27->v31 and regenerate the golden

* Fix superordinateDomain and inetAddresses in HostHistory

* V31 -> V32

* Fix SQL files that got messed up in the merge

* Configure and use a manually-created history ID sequence

* Add three more indices to HostHistory
2020-06-16 11:47:17 -04:00
Shicong Huang
26e2a51180 Refactor TransferData to remove unused fields in Contact table (#623)
* Add DomainTransferData and ContactTransferData

* Refactor TransferData to remove unused fields in Contact table

* Add scope for TransferData's type parameter
2020-06-16 10:42:57 -04:00
Michael Muller
21f2f38ad1 Allow class-specific creation of symmetric VKeys (#625)
* Allow class-specific creation of symmetrict VKeys

When translating from a datastore Key to a VKey, see if the "kind" class
contains a createVKey(com.googlecode.objectify.Key) static method and if it
does, use it to construct a symmetric VKey instead of simply creating an
objectify-sided asymmetric VKey.

As a test case for this, implement the createVKey() static function for
DomainBase.  Also, create unit tests for VKeyTranslatorFactory, which
continues to house the functionality.
2020-06-15 11:35:03 -04:00
Lai Jiang
0e04f6ca5b Upgrade to Gradle 6.5 (#627) 2020-06-15 10:57:29 -04:00
Ben McIlwain
4be70c8509 Refactor Fee handling so that each fee knows if it's premium (#626)
* Refactor Fee handling so that each fee knows if it's premium

This is a noop for now, as the new isPremium boolean isn't yet used by anything,
but it will be used in follow-up PRs to add additional fee information using the
fee extension (see: b/157621273).  Specifically what we're trying to do here is
return <fee:command name="create" standard="1"> (using the finalized version of
the fee extension) when an entire command has no premium fee associated with
it. And in the current earlier versions of the fee extension that we support,
we'll want to display the correct fee amount and class for creates/checks on
reserved domains when a valid allocation token is passed. This also needs the
isPremium information.

There are no testing implications yet because isPremium isn't exposed anywhere,
but there will definitely be lots of test changes once it's feeding into EPP-
visible changes.

* Rename things, add method Javadoc

* Apply formatting
2020-06-12 16:43:02 -04:00
Ben McIlwain
cf1448bca8 Restore the original expiration time on domain restore (#601)
* Restore the original expiration time on domain restore

Except if that time is now in the past, then add a year to it.

* Apply auto-formatter changes to fix my local build

* Merge branch 'master' into restore-expiry-date

* Fix reversed comments
2020-06-12 14:33:49 -04:00
gbrodman
f62473542f Use the requested server host when creating the registry lock verification URL (#624)
* Use the server host when creating the registry lock verification URL

The app doesn't know about any external configuration that may point to
this app, so there's no way of finding out that, for instance,
registry.google points to the app. Thus, we have to use what the user
gives us so that, in our case, the registry-lock verification
emails can point to https://registry.google/registry-lock-verify instead
of https://domain-registry.appspot.com/registry-lock-verify. The former
is used by clients / users to authenticate, and unfortunately
authenticating on registry.google does not give authentication to
domain-registry.apspot.com.

Tested using the RDAP code that uses getServerName() -- in that case, if
you access registry.google/rdap/<>, it uses registry.google in the URLs
but if you use domain-registry.appspot.com/rdap/<>, it uses
domain-registry.appspot.com in the URLs.

Relatedly, frontend_config_prod-appengine.asciiproto in Piper
is what configures registry.google to point to
domain-registry.appspot.com
2020-06-12 10:11:53 -04:00
Lai Jiang
484173b659 Log client certificate type and length (#622)
* Log client certificate type and length

It appears that most client certificates are RSA certs, but we should
make sure that is indeed the case. Print out the strength of the cert
if it is RSA.

Also adds supports for TLS 1.3 and print out the supported cipher suites.

* Add a comment about zero length certificate

* Make length of non-RSA keys -1

* Don't use TLS 1.3 if JDK SSL provider is used
2020-06-11 17:11:40 -04:00
Weimin Yu
d3fd826dc1 Load CommitLog into BEAM pipeline (#618)
* Load CommitLog into BEAM pipeline

Created tools that can generate CommitLogs on a test
datastore.

Defined BEAM transforms that load from CommitLog files and
added a few simple tests.

This is work-in-progress. Next step is to build a consistent
backup by merging Datastore exports and CommitLogs.
2020-06-11 11:38:53 -04:00
gbrodman
1c62728886 Rename V30 -> V31 to avoid duplicates (#621) 2020-06-10 16:08:31 -04:00
Lai Jiang
b5d3186e67 Update Netty to the latest version (#620)
* Upgrade to the latest version of Netty

* Update lock files
2020-06-10 16:08:11 -04:00
gbrodman
b4dfec5fd5 Rename client_id to registrar_id in SQL (#619)
We'll eventually want to shift everything over to using registrar_id and
registrarId rather than client_id and clientId but for the sake of the
Datastore schema and existing code, we won't change the Java identifier
for now. Once we're completely and only on SQL, we can rename the Java
field easily.
2020-06-10 15:11:27 -04:00
gbrodman
40b14fb695 Create a converter for sets of InetAddresses and use it in HostResource (#612)
* Create a converter for sets of inetAddresses and use it in HostResource

This can just be a set of strings where each string represents an
address;  there's no need for it to be a separate table. This allows
for simplification of the SQL schema.

* Regenerate golden SQL file after renaming v28 -> v29

* Add more tests and rename a typo in the file

* Refactor common test code and use tm methods

* Use JUnit5 API

* Rename test entity
2020-06-10 13:04:20 -04:00
Shicong Huang
fdac686250 Add columns for TransferData in Domain and Contact (#577)
* Add columns for TransferData in Domain and Contact

* Rename flyway file and foreign key

* Rebase on master and address comment

* Compileable commit

* Fix unit test

* Refactor TransferServerApproveEntity

* Use tm().delete(vkeys)

* Rename transfer_period fields

* Rename client_id to registrar_id

* Rebase on master

* Resolve comment

* Rebase on master
2020-06-09 16:39:55 -04:00
Shicong Huang
f0765dc893 Make JpaUnitTestRule not depend on the nomulus schema (#613) 2020-06-09 14:51:13 -04:00
Lai Jiang
2995bb03fd Update Javadoc URL (#615) 2020-06-09 10:25:56 -04:00
gbrodman
0f415f78a6 Use the correct text VKey for HostResource's superordinateDomain (#608)
* Store the superordinateDomain reference as a VKey rather than Key

This is a reference to a Domain object, so we should store it as a VKey
in reference to the Domain table. This should not affect any business
logic, but rather will allow us to set up the SQL tables for
HostResource et al. properly.
2020-06-08 12:21:51 -04:00
gbrodman
b324fb98d3 Only use asymmetric VKeys for EPP resources (#611)
Given that we currently have no way of reconstituting a symmetric key
from the asymmetric key (or at least, we don't have a 100% reliable way
of doing so) it's best to keep keys as asymmetrical, referring to the
correct database. That way, we don't get situations where we cannot
compare equality of two keys due to one being asymmetrical and one being
symmetrical.
2020-06-05 16:55:12 -04:00
Lai Jiang
d27fe8ead5 Fix a typo (#610) 2020-06-05 15:53:17 -04:00
Lai Jiang
da65a38782 Add a GCB job to build and publish javadoc (#609) 2020-06-05 13:00:15 -04:00
Legina Chen
5a1f3d0376 Remove platformType and threatEntryMetaData fields from ThreatMatch (#607)
* Remove platformType and threatEntryMetaData fields from ThreatMatch

* Run google-java-format on both files

* Add test for removal of unnecessary fields

* Removed unnecessary fields from Spec11PipelineTest.testEndToEndPipeline_generatesExpectedFiles

* Added style check

* Fix typo
2020-06-05 09:00:07 -07:00
Shicong Huang
b1241b98b2 Generate sql schema for PollMessage (#582)
* Generate sql schema for PollMessage

* Rework columns and resolve comments

* Fix datastore schema
2020-06-04 18:24:59 -04:00
Shicong Huang
b42ded9451 Add test to verify the behavior of @DualDatabaseTest (#606) 2020-06-03 14:55:37 -04:00
Shicong Huang
472503541b Add deleteAll method to TransactionManager (#604)
* Add deleteAll method to TransactionManager

* Rename deleteAll to delete

* Add bucket.getLastWrittenTime() before second mutation
2020-06-03 10:02:48 -04:00
Weimin Yu
ed64dd3548 Load raw records from Datastore export (#605)
* Load raw records from Datastore export

Created a tool that can export from a test instance of Datastore.

Defined Beam pipeline transforms for loading raw records back from
the export.

This is the first part of the effort to create a consistent snapshot
of Datastore. The next step is to load entity records from CommitLog
files.
2020-06-02 18:55:03 -04:00
Michael Muller
6a96b1a9cd Use TransactionManager for hosts and contacts (#603)
* Use TransactionManager for hosts and contacts

Replace Ofy calls with TransactionManager for most interactions involving
hosts and contacts.  In the course of this, also convert ForeignKeyIndex and
the EppResourceCache.

* Minor formatting fix
2020-06-02 13:17:16 -04:00
Michael Muller
c23d4f3ba5 Add createVKey() at the EppResource level (#600)
* Add createVKey() at the EppResource level

Also convert createKey() to createVKey() to normalize with what we've settled
on.
2020-05-29 08:36:57 -04:00
1186 changed files with 33992 additions and 20935 deletions

7
.gitignore vendored
View File

@@ -79,6 +79,10 @@ nomulus.iml
nomulus.ipr
nomulus.iws
# Auto-generated java classes by Intellij
*/src/main/generated/
*/src/test/generated_tests/
# VScode
.vscode
@@ -93,8 +97,9 @@ nomulus.iws
######################################################################
# Gradle Ignores
# We don't want to ignore the gradle jar files
# We don't want to ignore the gradle or google-java-format jar files
!/gradle/wrapper/**/*.jar
!/java-format/*.jar
.gradle/
**/build
cloudbuild-caches/

View File

@@ -2,3 +2,5 @@ extraction:
java:
prepare:
packages: "npm"
index:
java_version: "11"

View File

@@ -33,7 +33,7 @@ running system:
* View the source code for the [GAE app](https://github.com/google/nomulus/tree/master/core/src/main/java/google/registry)
and for the [GKE proxy](https://github.com/google/nomulus/tree/master/proxy/src/main/java/google/registry)
* [Other docs](https://github.com/google/nomulus/tree/master/docs)
* [Javadoc](https://nomulus.foo/javadoc/latest/)
* [Javadoc](https://javadoc.nomulus.foo/)
* [Nomulus discussion
group](https://groups.google.com/forum/#!forum/nomulus-discuss), for any
other questions

View File

@@ -25,7 +25,7 @@ buildscript {
dependencies {
classpath 'com.google.cloud.tools:appengine-gradle-plugin:2.0.1'
classpath "net.ltgt.gradle:gradle-errorprone-plugin:0.6.1"
classpath 'net.ltgt.gradle:gradle-errorprone-plugin:0.6.1'
classpath 'org.sonatype.aether:aether-api:1.13.1'
classpath 'org.sonatype.aether:aether-impl:1.13.1'
}
@@ -35,9 +35,6 @@ plugins {
// Java static analysis plugins. Keep versions consistent with
// ./buildSrc/build.gradle
id 'nebula.lint' version '16.0.2'
// TODO(weiminyu): consider remove net.ltgt.apt. Gradle 5.2+
// has similar functionalities.
id 'net.ltgt.apt' version '0.19' apply false
id 'net.ltgt.errorprone' version '0.6.1'
id 'checkstyle'
id 'com.github.johnrengelman.shadow' version '5.1.0'
@@ -49,6 +46,7 @@ plugins {
id 'com.diffplug.gradle.spotless' version '3.25.0'
id 'jacoco'
id 'com.dorongold.task-tree' version '1.5'
}
wrapper {
@@ -289,12 +287,6 @@ subprojects {
project.tasks.test.dependsOn runPresubmits
// Path to code generated with annotation processors. Note that this path is
// chosen by the 'net.ltgt.apt' plugin, and may change if IDE-specific plugins
// are applied, e.g., 'idea' or 'eclipse'
def aptGeneratedDir = "${project.buildDir}/generated/source/apt/main"
def aptGeneratedTestDir = "${project.buildDir}/generated/source/apt/test"
def commonlyExcludedResources = ['**/*.java', '**/BUILD']
project.ext.javaDir = "${project.projectDir}/src/main/java"
@@ -303,18 +295,12 @@ subprojects {
sourceSets {
main {
java {
srcDirs += aptGeneratedDir
}
resources {
srcDirs += project.ext.javaDir
exclude commonlyExcludedResources
}
}
test {
java {
srcDirs += aptGeneratedTestDir
}
resources {
srcDirs += project.ext.javaTestDir
exclude commonlyExcludedResources
@@ -322,9 +308,14 @@ subprojects {
}
}
javadocSource << project.sourceSets.main.allJava
javadocClasspath << project.sourceSets.main.compileClasspath
javadocDependentTasks << project.tasks.compileJava
// No need to produce javadoc for the docs subproject, which has no APIs to
// expose to users.
if (project.name != 'docs') {
javadocSource << project.sourceSets.main.allJava
javadocClasspath << project.sourceSets.main.compileClasspath
javadocClasspath << "${buildDir}/generated/sources/annotationProcessor/java/main"
javadocDependentTasks << project.tasks.compileJava
}
}
// If "-P verboseTestOutput=true" is passed in, configure all subprojects to dump all of their
@@ -443,6 +434,7 @@ task javaIncrementalFormatDryRun {
println("${invokeJavaDiffFormatScript("show")}")
}
}
tasks.build.dependsOn(tasks.javaIncrementalFormatCheck)
// Checks if modified lines in Java source files need reformatting.
// Note that this task processes modified Java files in the entire repository.
@@ -452,12 +444,11 @@ task javaIncrementalFormatApply {
}
}
tasks.build.dependsOn(tasks.javaIncrementalFormatCheck)
task javadoc(type: Javadoc) {
source javadocSource
classpath = files(javadocClasspath)
destinationDir = file("${buildDir}/docs/javadoc")
options.encoding = "UTF-8"
// In a lot of places we don't write @return so suppress warnings about that.
options.addBooleanOption('Xdoclint:all,-missing', true)
options.addBooleanOption("-allow-script-in-comments",true)
@@ -467,4 +458,16 @@ task javadoc(type: Javadoc) {
tasks.build.dependsOn(tasks.javadoc)
javadocDependentTasks.each { tasks.javadoc.shouldRunAfter(it) }
// Task for doing development on core Nomulus.
// This fixes code formatting automatically as necessary, builds and tests the
// core Nomulus codebase, and runs all presubmits.
task coreDev {
dependsOn 'javaIncrementalFormatApply'
dependsOn 'javadoc'
dependsOn 'checkDependenciesDotGradle'
dependsOn 'checkLicense'
dependsOn ':core:check'
dependsOn 'assemble'
}
javadocDependentTasks.each { tasks.javadoc.dependsOn(it) }

View File

@@ -26,9 +26,6 @@ buildscript {
plugins {
// Java static analysis plugins. Keep versions consistent with ../build.gradle
id 'nebula.lint' version '16.0.2'
// Config helper for annotation processors such as AutoValue and Dagger.
// Ensures that source code is generated at an appropriate location.
id 'net.ltgt.apt' version '0.19' apply false
id 'net.ltgt.errorprone' version '0.6.1'
id 'checkstyle'
id 'com.diffplug.gradle.spotless' version '3.25.0'
@@ -83,10 +80,8 @@ dependencies {
annotationProcessor deps['com.google.auto.value:auto-value']
testCompile deps['com.google.truth:truth']
testCompile deps['com.google.truth.extensions:truth-java8-extension']
testCompile deps['junit:junit']
testCompile deps['org.junit.jupiter:junit-jupiter-api']
testCompile deps['org.junit.jupiter:junit-jupiter-engine']
testCompile deps['org.junit.vintage:junit-vintage-engine']
testCompile deps['org.mockito:mockito-core']
}

View File

@@ -19,9 +19,7 @@ commons-lang:commons-lang:2.6
javax.inject:javax.inject:1
junit:junit:4.12
nebula.lint:nebula.lint.gradle.plugin:16.0.2
net.ltgt.apt:net.ltgt.apt.gradle.plugin:0.19
net.ltgt.errorprone:net.ltgt.errorprone.gradle.plugin:0.6.1
net.ltgt.gradle:gradle-apt-plugin:0.19
net.ltgt.gradle:gradle-errorprone-plugin:0.6.1
org.apache.commons:commons-lang3:3.8.1
org.apache.maven:maven-artifact:3.6.2

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -50,7 +50,7 @@ org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -50,7 +50,7 @@ org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -50,7 +50,7 @@ org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0

View File

@@ -1,4 +0,0 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.errorprone:javac:9+181-r4173-1

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -50,7 +50,7 @@ org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -50,7 +50,7 @@ org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.json:json:20160212
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -49,25 +49,24 @@ javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.validation:validation-api:1.0.0.GA
joda-time:joda-time:2.9.2
junit:junit:4.13
net.bytebuddy:byte-buddy-agent:1.9.7
net.bytebuddy:byte-buddy:1.9.7
junit:junit:4.12
net.bytebuddy:byte-buddy-agent:1.10.5
net.bytebuddy:byte-buddy:1.10.5
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.hamcrest:hamcrest-core:1.3
org.json:json:20160212
org.junit.jupiter:junit-jupiter-api:5.6.1
org.junit.jupiter:junit-jupiter-engine:5.6.1
org.junit.platform:junit-platform-commons:1.6.1
org.junit.platform:junit-platform-engine:1.6.1
org.junit.vintage:junit-vintage-engine:5.6.1
org.junit:junit-bom:5.6.1
org.mockito:mockito-core:2.25.0
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.mockito:mockito-core:3.3.3
org.objenesis:objenesis:2.6
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:6.0

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -49,25 +49,24 @@ javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.validation:validation-api:1.0.0.GA
joda-time:joda-time:2.9.2
junit:junit:4.13
net.bytebuddy:byte-buddy-agent:1.9.7
net.bytebuddy:byte-buddy:1.9.7
junit:junit:4.12
net.bytebuddy:byte-buddy-agent:1.10.5
net.bytebuddy:byte-buddy:1.10.5
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.hamcrest:hamcrest-core:1.3
org.json:json:20160212
org.junit.jupiter:junit-jupiter-api:5.6.1
org.junit.jupiter:junit-jupiter-engine:5.6.1
org.junit.platform:junit-platform-commons:1.6.1
org.junit.platform:junit-platform-engine:1.6.1
org.junit.vintage:junit-vintage-engine:5.6.1
org.junit:junit-bom:5.6.1
org.mockito:mockito-core:2.25.0
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.mockito:mockito-core:3.3.3
org.objenesis:objenesis:2.6
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:6.0

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -49,25 +49,24 @@ javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.validation:validation-api:1.0.0.GA
joda-time:joda-time:2.9.2
junit:junit:4.13
net.bytebuddy:byte-buddy-agent:1.9.7
net.bytebuddy:byte-buddy:1.9.7
junit:junit:4.12
net.bytebuddy:byte-buddy-agent:1.10.5
net.bytebuddy:byte-buddy:1.10.5
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.hamcrest:hamcrest-core:1.3
org.json:json:20160212
org.junit.jupiter:junit-jupiter-api:5.6.1
org.junit.jupiter:junit-jupiter-engine:5.6.1
org.junit.platform:junit-platform-commons:1.6.1
org.junit.platform:junit-platform-engine:1.6.1
org.junit.vintage:junit-vintage-engine:5.6.1
org.junit:junit-bom:5.6.1
org.mockito:mockito-core:2.25.0
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.mockito:mockito-core:3.3.3
org.objenesis:objenesis:2.6
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:6.0

View File

@@ -22,7 +22,7 @@ com.google.code.gson:gson:2.7
com.google.common.html.types:types:1.0.4
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.0-beta1
com.google.http-client:google-http-client-appengine:1.27.0
@@ -49,25 +49,24 @@ javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.validation:validation-api:1.0.0.GA
joda-time:joda-time:2.9.2
junit:junit:4.13
net.bytebuddy:byte-buddy-agent:1.9.7
net.bytebuddy:byte-buddy:1.9.7
junit:junit:4.12
net.bytebuddy:byte-buddy-agent:1.10.5
net.bytebuddy:byte-buddy:1.10.5
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.hamcrest:hamcrest-core:1.3
org.json:json:20160212
org.junit.jupiter:junit-jupiter-api:5.6.1
org.junit.jupiter:junit-jupiter-engine:5.6.1
org.junit.platform:junit-platform-commons:1.6.1
org.junit.platform:junit-platform-engine:1.6.1
org.junit.vintage:junit-vintage-engine:5.6.1
org.junit:junit-bom:5.6.1
org.mockito:mockito-core:2.25.0
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.mockito:mockito-core:3.3.3
org.objenesis:objenesis:2.6
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:6.0

View File

@@ -29,6 +29,7 @@ import google.registry.gradle.plugin.ProjectData.TaskData;
import java.io.File;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Map;
import java.util.function.Supplier;
import java.util.stream.Collectors;
@@ -162,7 +163,7 @@ final class CoverPageGenerator {
task.reports().entrySet().stream()
.collect(
toImmutableMap(
entry -> entry.getKey(),
Map.Entry::getKey,
entry ->
entry.getValue().files().isEmpty()
? ""

View File

@@ -57,6 +57,10 @@ final class GcsPluginUtils {
return file.toPath().toAbsolutePath().normalize();
}
static Path toNormalizedPath(Path file) {
return file.toAbsolutePath().normalize();
}
static String getContentType(String fileName) {
return EXTENSION_TO_CONTENT_TYPE.getOrDefault(
Files.getFileExtension(fileName), DEFAULT_CONTENT_TYPE);
@@ -168,7 +172,7 @@ final class GcsPluginUtils {
.collect(
toImmutableMap(
file -> rootDir.relativize(toNormalizedPath(file)),
file -> toByteArraySupplier(file)));
GcsPluginUtils::toByteArraySupplier));
if (files.isEmpty()) {
// The directory exists, but is empty. Return empty FilesWithEntryPoint

View File

@@ -26,13 +26,10 @@ import com.google.common.collect.ImmutableSet;
import google.registry.gradle.plugin.ProjectData.TaskData;
import java.io.File;
import java.nio.file.Paths;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
import org.junit.jupiter.api.Test;
/** Tests for {@link CoverPageGenerator} */
@RunWith(JUnit4.class)
public final class CoverPageGeneratorTest {
final class CoverPageGeneratorTest {
private static final ProjectData EMPTY_PROJECT =
ProjectData.builder()
@@ -88,14 +85,14 @@ public final class CoverPageGeneratorTest {
}
@Test
public void testGetFilesToUpload_entryPoint_isIndexHtml() {
void testGetFilesToUpload_entryPoint_isIndexHtml() {
CoverPageGenerator coverPageGenerator = new CoverPageGenerator(EMPTY_PROJECT);
assertThat(coverPageGenerator.getFilesToUpload().entryPoint())
.isEqualTo(Paths.get("index.html"));
}
@Test
public void testGetFilesToUpload_containsEntryFile() {
void testGetFilesToUpload_containsEntryFile() {
String content = getContentOfGeneratedFile(EMPTY_PROJECT, "index.html");
assertThat(content)
.contains(
@@ -103,7 +100,7 @@ public final class CoverPageGeneratorTest {
}
@Test
public void testCoverPage_showsFailedTask() {
void testCoverPage_showsFailedTask() {
String content = getCoverPage(EMPTY_PROJECT.toBuilder().addTask(EMPTY_TASK_FAILURE).build());
assertThat(content).contains("task-failure");
assertThat(content).contains("<p>FAILURE</p>");
@@ -112,7 +109,7 @@ public final class CoverPageGeneratorTest {
}
@Test
public void testCoverPage_showsSuccessfulTask() {
void testCoverPage_showsSuccessfulTask() {
String content = getCoverPage(EMPTY_PROJECT.toBuilder().addTask(EMPTY_TASK_SUCCESS).build());
assertThat(content).contains("task-success");
assertThat(content).doesNotContain("<p>FAILURE</p>");
@@ -121,7 +118,7 @@ public final class CoverPageGeneratorTest {
}
@Test
public void testCoverPage_showsUpToDateTask() {
void testCoverPage_showsUpToDateTask() {
String content = getCoverPage(EMPTY_PROJECT.toBuilder().addTask(EMPTY_TASK_UP_TO_DATE).build());
assertThat(content).contains("task-up-to-date");
assertThat(content).doesNotContain("<p>FAILURE</p>");
@@ -130,11 +127,10 @@ public final class CoverPageGeneratorTest {
}
@Test
public void testCoverPage_failedAreFirst() {
void testCoverPage_failedAreFirst() {
String content =
getCoverPage(
EMPTY_PROJECT
.toBuilder()
EMPTY_PROJECT.toBuilder()
.addTask(EMPTY_TASK_UP_TO_DATE)
.addTask(EMPTY_TASK_FAILURE)
.addTask(EMPTY_TASK_SUCCESS)
@@ -149,11 +145,10 @@ public final class CoverPageGeneratorTest {
}
@Test
public void testCoverPage_failingTask_statusIsFailure() {
void testCoverPage_failingTask_statusIsFailure() {
String content =
getCoverPage(
EMPTY_PROJECT
.toBuilder()
EMPTY_PROJECT.toBuilder()
.addTask(EMPTY_TASK_UP_TO_DATE)
.addTask(EMPTY_TASK_FAILURE)
.addTask(EMPTY_TASK_SUCCESS)
@@ -162,11 +157,10 @@ public final class CoverPageGeneratorTest {
}
@Test
public void testCoverPage_noFailingTask_statusIsSuccess() {
void testCoverPage_noFailingTask_statusIsSuccess() {
String content =
getCoverPage(
EMPTY_PROJECT
.toBuilder()
EMPTY_PROJECT.toBuilder()
.addTask(EMPTY_TASK_UP_TO_DATE)
.addTask(EMPTY_TASK_SUCCESS)
.build());
@@ -174,7 +168,7 @@ public final class CoverPageGeneratorTest {
}
@Test
public void testGetFilesToUpload_containsCssFile() {
void testGetFilesToUpload_containsCssFile() {
ImmutableMap<String, String> files = getGeneratedFiles(EMPTY_PROJECT);
assertThat(files).containsKey(filenameJoiner.join("css", "style.css"));
assertThat(files.get(filenameJoiner.join("css", "style.css"))).contains("body {");
@@ -183,14 +177,12 @@ public final class CoverPageGeneratorTest {
}
@Test
public void testCreateReportFiles_taskWithLog() {
void testCreateReportFiles_taskWithLog() {
ImmutableMap<String, String> files =
getGeneratedFiles(
EMPTY_PROJECT
.toBuilder()
EMPTY_PROJECT.toBuilder()
.addTask(
EMPTY_TASK_SUCCESS
.toBuilder()
EMPTY_TASK_SUCCESS.toBuilder()
.setUniqueName("my:name")
.setLog(toByteArraySupplier("my log data"))
.build())
@@ -200,11 +192,10 @@ public final class CoverPageGeneratorTest {
}
@Test
public void testCreateReportFiles_taskWithoutLog() {
void testCreateReportFiles_taskWithoutLog() {
ImmutableMap<String, String> files =
getGeneratedFiles(
EMPTY_PROJECT
.toBuilder()
EMPTY_PROJECT.toBuilder()
.addTask(EMPTY_TASK_SUCCESS.toBuilder().setUniqueName("my:name").build())
.build());
assertThat(files).doesNotContainKey("logs/my-name.log");
@@ -212,14 +203,12 @@ public final class CoverPageGeneratorTest {
}
@Test
public void testCreateReportFiles_taskWithFilledReport() {
void testCreateReportFiles_taskWithFilledReport() {
ImmutableMap<String, String> files =
getGeneratedFiles(
EMPTY_PROJECT
.toBuilder()
EMPTY_PROJECT.toBuilder()
.addTask(
EMPTY_TASK_SUCCESS
.toBuilder()
EMPTY_TASK_SUCCESS.toBuilder()
.putReport(
"someReport",
FilesWithEntryPoint.create(
@@ -234,14 +223,12 @@ public final class CoverPageGeneratorTest {
}
@Test
public void testCreateReportFiles_taskWithEmptyReport() {
void testCreateReportFiles_taskWithEmptyReport() {
ImmutableMap<String, String> files =
getGeneratedFiles(
EMPTY_PROJECT
.toBuilder()
EMPTY_PROJECT.toBuilder()
.addTask(
EMPTY_TASK_SUCCESS
.toBuilder()
EMPTY_TASK_SUCCESS.toBuilder()
.putReport(
"someReport",
FilesWithEntryPoint.create(
@@ -254,14 +241,12 @@ public final class CoverPageGeneratorTest {
}
@Test
public void testCreateReportFiles_taskWithLogAndMultipleReports() {
void testCreateReportFiles_taskWithLogAndMultipleReports() {
ImmutableMap<String, String> files =
getGeneratedFiles(
EMPTY_PROJECT
.toBuilder()
EMPTY_PROJECT.toBuilder()
.addTask(
EMPTY_TASK_SUCCESS
.toBuilder()
EMPTY_TASK_SUCCESS.toBuilder()
.setUniqueName("my:name")
.setLog(toByteArraySupplier("log data"))
.putReport(

View File

@@ -23,6 +23,9 @@ import static google.registry.gradle.plugin.GcsPluginUtils.toNormalizedPath;
import static google.registry.gradle.plugin.GcsPluginUtils.uploadFileToGcs;
import static google.registry.gradle.plugin.GcsPluginUtils.uploadFilesToGcsMultithread;
import static java.nio.charset.StandardCharsets.UTF_8;
import static java.nio.file.Files.createDirectories;
import static java.nio.file.Files.createDirectory;
import static java.nio.file.Files.createFile;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.verifyNoMoreInteractions;
@@ -36,22 +39,20 @@ import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Optional;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
/** Tests for {@link GcsPluginUtilsTest} */
@RunWith(JUnit4.class)
public final class GcsPluginUtilsTest {
final class GcsPluginUtilsTest {
private static final Joiner filenameJoiner = Joiner.on(File.separator);
@Rule public final TemporaryFolder folder = new TemporaryFolder();
@SuppressWarnings("WeakerAccess")
@TempDir
Path tmpDir;
@Test
public void testGetContentType_knownTypes() {
void testGetContentType_knownTypes() {
assertThat(getContentType("path/to/file.html")).isEqualTo("text/html");
assertThat(getContentType("path/to/file.htm")).isEqualTo("text/html");
assertThat(getContentType("path/to/file.log")).isEqualTo("text/plain");
@@ -63,12 +64,12 @@ public final class GcsPluginUtilsTest {
}
@Test
public void testGetContentType_unknownTypes() {
void testGetContentType_unknownTypes() {
assertThat(getContentType("path/to/file.unknown")).isEqualTo("application/octet-stream");
}
@Test
public void testUploadFileToGcs() {
void testUploadFileToGcs() {
Storage storage = mock(Storage.class);
uploadFileToGcs(
storage, "my-bucket", Paths.get("my", "filename.txt"), toByteArraySupplier("my data"));
@@ -82,7 +83,7 @@ public final class GcsPluginUtilsTest {
}
@Test
public void testUploadFilesToGcsMultithread() {
void testUploadFilesToGcsMultithread() {
Storage storage = mock(Storage.class);
uploadFilesToGcsMultithread(
storage,
@@ -121,21 +122,21 @@ public final class GcsPluginUtilsTest {
}
@Test
public void testToByteArraySupplier_string() {
void testToByteArraySupplier_string() {
assertThat(toByteArraySupplier("my string").get()).isEqualTo("my string".getBytes(UTF_8));
}
@Test
public void testToByteArraySupplier_stringSupplier() {
void testToByteArraySupplier_stringSupplier() {
assertThat(toByteArraySupplier(() -> "my string").get()).isEqualTo("my string".getBytes(UTF_8));
}
@Test
public void testToByteArraySupplier_file() throws Exception {
folder.newFolder("arbitrary");
File file = folder.newFile("arbitrary/file.txt");
Files.write(file.toPath(), "some data".getBytes(UTF_8));
assertThat(toByteArraySupplier(file).get()).isEqualTo("some data".getBytes(UTF_8));
void testToByteArraySupplier_file() throws Exception {
Path dir = createDirectory(tmpDir.resolve("arbitrary"));
Path file = createFile(dir.resolve("file.txt"));
Files.write(file, "some data".getBytes(UTF_8));
assertThat(toByteArraySupplier(file.toFile()).get()).isEqualTo("some data".getBytes(UTF_8));
}
private ImmutableMap<String, String> readAllFiles(FilesWithEntryPoint reportFiles) {
@@ -147,16 +148,16 @@ public final class GcsPluginUtilsTest {
}
@Test
public void testCreateReportFiles_destinationIsFile() throws Exception {
Path root = toNormalizedPath(folder.newFolder("my", "root"));
folder.newFolder("my", "root", "some", "path");
File destination = folder.newFile("my/root/some/path/file.txt");
Files.write(destination.toPath(), "some data".getBytes(UTF_8));
void testCreateReportFiles_destinationIsFile() throws Exception {
Path root = toNormalizedPath(createDirectories(tmpDir.resolve("my/root")).toAbsolutePath());
Path somePath = createDirectories(root.resolve("some/path"));
Path destination = createFile(somePath.resolve("file.txt"));
Files.write(destination, "some data".getBytes(UTF_8));
// Since the entry point is obvious here - any hint given is just ignored.
File ignoredHint = folder.newFile("my/root/ignored.txt");
File ignoredHint = createFile(root.resolve("ignored.txt")).toFile();
FilesWithEntryPoint files =
readFilesWithEntryPoint(destination, Optional.of(ignoredHint), root);
readFilesWithEntryPoint(destination.toFile(), Optional.of(ignoredHint), root);
assertThat(files.entryPoint().toString())
.isEqualTo(filenameJoiner.join("some", "path", "file.txt"));
@@ -165,13 +166,13 @@ public final class GcsPluginUtilsTest {
}
@Test
public void testCreateReportFiles_destinationDoesntExist() throws Exception {
Path root = toNormalizedPath(folder.newFolder("my", "root"));
void testCreateReportFiles_destinationDoesntExist() throws Exception {
Path root = toNormalizedPath(createDirectories(tmpDir.resolve("my/root")).toAbsolutePath());
File destination = root.resolve("non/existing.txt").toFile();
assertThat(destination.isFile()).isFalse();
assertThat(destination.isDirectory()).isFalse();
// Since there are not files, any hint given is obvioulsy wrong and will be ignored.
File ignoredHint = folder.newFile("my/root/ignored.txt");
// Since there are no files, any hint given is obviously wrong and will be ignored.
File ignoredHint = createFile(root.resolve("ignored.txt")).toFile();
FilesWithEntryPoint files =
readFilesWithEntryPoint(destination, Optional.of(ignoredHint), root);
@@ -181,34 +182,33 @@ public final class GcsPluginUtilsTest {
}
@Test
public void testCreateReportFiles_noFiles() throws Exception {
Path root = toNormalizedPath(folder.newFolder("my", "root"));
File destination = folder.newFolder("my", "root", "some", "path");
folder.newFolder("my", "root", "some", "path", "a", "b");
folder.newFolder("my", "root", "some", "path", "c");
// Since there are not files, any hint given is obvioulsy wrong and will be ignored.
File ignoredHint = folder.newFile("my/root/ignored.txt");
void testCreateReportFiles_noFiles() throws Exception {
Path root = toNormalizedPath(createDirectories(tmpDir.resolve("my/root")).toAbsolutePath());
Path destination = createDirectories(root.resolve("some/path"));
createDirectories(destination.resolve("a/b"));
createDirectory(destination.resolve("c"));
// Since there are not files, any hint given is obviously wrong and will be ignored.
File ignoredHint = createFile(root.resolve("ignored.txt")).toFile();
FilesWithEntryPoint files =
readFilesWithEntryPoint(destination, Optional.of(ignoredHint), root);
readFilesWithEntryPoint(destination.toFile(), Optional.of(ignoredHint), root);
assertThat(files.entryPoint().toString()).isEqualTo(filenameJoiner.join("some", "path"));
assertThat(files.files()).isEmpty();
}
@Test
public void testCreateReportFiles_oneFile() throws Exception {
Path root = toNormalizedPath(folder.newFolder("my", "root"));
File destination = folder.newFolder("my", "root", "some", "path");
folder.newFolder("my", "root", "some", "path", "a", "b");
folder.newFolder("my", "root", "some", "path", "c");
Files.write(
folder.newFile("my/root/some/path/a/file.txt").toPath(), "some data".getBytes(UTF_8));
void testCreateReportFiles_oneFile() throws Exception {
Path root = toNormalizedPath(createDirectories(tmpDir.resolve("my/root")).toAbsolutePath());
Path destination = createDirectories(root.resolve("some/path"));
createDirectories(destination.resolve("a/b"));
createDirectory(destination.resolve("c"));
Files.write(createFile(destination.resolve("a/file.txt")), "some data".getBytes(UTF_8));
// Since the entry point is obvious here - any hint given is just ignored.
File ignoredHint = folder.newFile("my/root/ignored.txt");
File ignoredHint = createFile(root.resolve("ignored.txt")).toFile();
FilesWithEntryPoint files =
readFilesWithEntryPoint(destination, Optional.of(ignoredHint), root);
readFilesWithEntryPoint(destination.toFile(), Optional.of(ignoredHint), root);
assertThat(files.entryPoint().toString())
.isEqualTo(filenameJoiner.join("some", "path", "a", "file.txt"));
@@ -222,22 +222,19 @@ public final class GcsPluginUtilsTest {
* <p>TODO(guyben): switch to checking zip file instead.
*/
@Test
public void testCreateReportFiles_multipleFiles_noHint() throws Exception {
Path root = toNormalizedPath(folder.newFolder("my", "root"));
File destination = folder.newFolder("my", "root", "some", "path");
folder.newFolder("my", "root", "some", "path", "a", "b");
folder.newFolder("my", "root", "some", "path", "c");
void testCreateReportFiles_multipleFiles_noHint() throws Exception {
Path root = toNormalizedPath(createDirectories(tmpDir.resolve("my/root")).toAbsolutePath());
Path destination = createDirectories(root.resolve("some/path"));
createDirectories(destination.resolve("a/b"));
createDirectory(destination.resolve("c"));
Files.write(
folder.newFile("my/root/some/path/index.html").toPath(), "some data".getBytes(UTF_8));
Files.write(
folder.newFile("my/root/some/path/a/index.html").toPath(), "wrong index".getBytes(UTF_8));
Files.write(
folder.newFile("my/root/some/path/c/style.css").toPath(), "css file".getBytes(UTF_8));
Files.write(
folder.newFile("my/root/some/path/my_image.png").toPath(), "images".getBytes(UTF_8));
Files.write(createFile(destination.resolve("index.html")), "some data".getBytes(UTF_8));
Files.write(createFile(destination.resolve("a/index.html")), "wrong index".getBytes(UTF_8));
Files.write(createFile(destination.resolve("c/style.css")), "css file".getBytes(UTF_8));
Files.write(createFile(destination.resolve("my_image.png")), "images".getBytes(UTF_8));
FilesWithEntryPoint files = readFilesWithEntryPoint(destination, Optional.empty(), root);
FilesWithEntryPoint files =
readFilesWithEntryPoint(destination.toFile(), Optional.empty(), root);
assertThat(files.entryPoint().toString())
.isEqualTo(filenameJoiner.join("some", "path", "path.zip"));
@@ -251,24 +248,20 @@ public final class GcsPluginUtilsTest {
* <p>TODO(guyben): switch to checking zip file instead.
*/
@Test
public void testCreateReportFiles_multipleFiles_withBadHint() throws Exception {
Path root = toNormalizedPath(folder.newFolder("my", "root"));
File destination = folder.newFolder("my", "root", "some", "path");
void testCreateReportFiles_multipleFiles_withBadHint() throws Exception {
Path root = toNormalizedPath(createDirectories(tmpDir.resolve("my/root")).toAbsolutePath());
Path destination = createDirectories(root.resolve("some/path"));
// This entry point points to a directory, which isn't an appropriate entry point
File badEntryPoint = folder.newFolder("my", "root", "some", "path", "a", "b");
folder.newFolder("my", "root", "some", "path", "c");
File badEntryPoint = createDirectories(destination.resolve("a/b")).toFile();
createDirectory(destination.resolve("c"));
Files.write(
folder.newFile("my/root/some/path/index.html").toPath(), "some data".getBytes(UTF_8));
Files.write(
folder.newFile("my/root/some/path/a/index.html").toPath(), "wrong index".getBytes(UTF_8));
Files.write(
folder.newFile("my/root/some/path/c/style.css").toPath(), "css file".getBytes(UTF_8));
Files.write(
folder.newFile("my/root/some/path/my_image.png").toPath(), "images".getBytes(UTF_8));
Files.write(createFile(destination.resolve("index.html")), "some data".getBytes(UTF_8));
Files.write(createFile(destination.resolve("a/index.html")), "wrong index".getBytes(UTF_8));
Files.write(createFile(destination.resolve("c/style.css")), "css file".getBytes(UTF_8));
Files.write(createFile(destination.resolve("my_image.png")), "images".getBytes(UTF_8));
FilesWithEntryPoint files =
readFilesWithEntryPoint(destination, Optional.of(badEntryPoint), root);
readFilesWithEntryPoint(destination.toFile(), Optional.of(badEntryPoint), root);
assertThat(files.entryPoint().toString())
.isEqualTo(filenameJoiner.join("some", "path", "path.zip"));
@@ -277,24 +270,21 @@ public final class GcsPluginUtilsTest {
}
@Test
public void testCreateReportFiles_multipleFiles_withGoodHint() throws Exception {
Path root = toNormalizedPath(folder.newFolder("my", "root"));
File destination = folder.newFolder("my", "root", "some", "path");
folder.newFolder("my", "root", "some", "path", "a", "b");
folder.newFolder("my", "root", "some", "path", "c");
void testCreateReportFiles_multipleFiles_withGoodHint() throws Exception {
Path root = toNormalizedPath(createDirectories(tmpDir.resolve("my/root")).toAbsolutePath());
Path destination = createDirectories(root.resolve("some/path"));
createDirectories(destination.resolve("a/b"));
createDirectory(destination.resolve("c"));
// The hint is an actual file nested in the destination directory!
File goodEntryPoint = folder.newFile("my/root/some/path/index.html");
Path goodEntryPoint = createFile(destination.resolve("index.html"));
Files.write(goodEntryPoint.toPath(), "some data".getBytes(UTF_8));
Files.write(
folder.newFile("my/root/some/path/a/index.html").toPath(), "wrong index".getBytes(UTF_8));
Files.write(
folder.newFile("my/root/some/path/c/style.css").toPath(), "css file".getBytes(UTF_8));
Files.write(
folder.newFile("my/root/some/path/my_image.png").toPath(), "images".getBytes(UTF_8));
Files.write(goodEntryPoint, "some data".getBytes(UTF_8));
Files.write(createFile(destination.resolve("a/index.html")), "wrong index".getBytes(UTF_8));
Files.write(createFile(destination.resolve("c/style.css")), "css file".getBytes(UTF_8));
Files.write(createFile(destination.resolve("my_image.png")), "images".getBytes(UTF_8));
FilesWithEntryPoint files =
readFilesWithEntryPoint(destination, Optional.of(goodEntryPoint), root);
readFilesWithEntryPoint(destination.toFile(), Optional.of(goodEntryPoint.toFile()), root);
assertThat(files.entryPoint().toString())
.isEqualTo(filenameJoiner.join("some", "path", "index.html"));

View File

@@ -61,8 +61,6 @@ dependencies {
testingCompile deps['com.google.truth:truth']
testingCompile deps['io.github.java-diff-utils:java-diff-utils']
testCompile deps['junit:junit']
testCompile deps['org.junit.jupiter:junit-jupiter-api']
testCompile deps['org.junit.jupiter:junit-jupiter-engine']
testCompile deps['org.junit.vintage:junit-vintage-engine']
}

View File

@@ -4,9 +4,9 @@
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1

View File

@@ -4,9 +4,9 @@
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1

View File

@@ -4,9 +4,9 @@
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1

View File

@@ -4,9 +4,9 @@
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1

View File

@@ -1,4 +1,3 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.errorprone:javac:9+181-r4173-1

View File

@@ -4,9 +4,9 @@
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1

View File

@@ -4,9 +4,9 @@
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1

View File

@@ -6,7 +6,7 @@ com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -14,16 +14,15 @@ com.googlecode.java-diff-utils:diffutils:1.3.0
io.github.java-diff-utils:java-diff-utils:4.0
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.13
junit:junit:4.12
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
org.junit.jupiter:junit-jupiter-api:5.6.1
org.junit.jupiter:junit-jupiter-engine:5.6.1
org.junit.platform:junit-platform-commons:1.6.1
org.junit.platform:junit-platform-engine:1.6.1
org.junit.vintage:junit-vintage-engine:5.6.1
org.junit:junit-bom:5.6.1
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.opentest4j:opentest4j:1.2.0

View File

@@ -6,7 +6,7 @@ com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -14,16 +14,15 @@ com.googlecode.java-diff-utils:diffutils:1.3.0
io.github.java-diff-utils:java-diff-utils:4.0
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.13
junit:junit:4.12
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
org.junit.jupiter:junit-jupiter-api:5.6.1
org.junit.jupiter:junit-jupiter-engine:5.6.1
org.junit.platform:junit-platform-commons:1.6.1
org.junit.platform:junit-platform-engine:1.6.1
org.junit.vintage:junit-vintage-engine:5.6.1
org.junit:junit-bom:5.6.1
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.opentest4j:opentest4j:1.2.0

View File

@@ -7,7 +7,7 @@ com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.1
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -15,16 +15,15 @@ com.googlecode.java-diff-utils:diffutils:1.3.0
io.github.java-diff-utils:java-diff-utils:4.0
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.13
junit:junit:4.12
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
org.junit.jupiter:junit-jupiter-api:5.6.1
org.junit.jupiter:junit-jupiter-engine:5.6.1
org.junit.platform:junit-platform-commons:1.6.1
org.junit.platform:junit-platform-engine:1.6.1
org.junit.vintage:junit-vintage-engine:5.6.1
org.junit:junit-bom:5.6.1
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.opentest4j:opentest4j:1.2.0

View File

@@ -7,7 +7,7 @@ com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.1
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -15,16 +15,15 @@ com.googlecode.java-diff-utils:diffutils:1.3.0
io.github.java-diff-utils:java-diff-utils:4.0
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.13
junit:junit:4.12
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
org.junit.jupiter:junit-jupiter-api:5.6.1
org.junit.jupiter:junit-jupiter-engine:5.6.1
org.junit.platform:junit-platform-commons:1.6.1
org.junit.platform:junit-platform-engine:1.6.1
org.junit.vintage:junit-vintage-engine:5.6.1
org.junit:junit-bom:5.6.1
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.opentest4j:opentest4j:1.2.0

View File

@@ -6,7 +6,7 @@ com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -16,6 +16,6 @@ javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3

View File

@@ -6,7 +6,7 @@ com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -16,6 +16,6 @@ javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3

View File

@@ -7,7 +7,7 @@ com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.1
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -17,6 +17,6 @@ javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3

View File

@@ -7,7 +7,7 @@ com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.1
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
@@ -17,6 +17,6 @@ javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3

View File

@@ -17,7 +17,7 @@ package google.registry.testing.truth;
import static com.google.common.io.Resources.getResource;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.testing.truth.TextDiffSubject.assertThat;
import static org.junit.Assert.assertThrows;
import static org.junit.jupiter.api.Assertions.assertThrows;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
@@ -25,13 +25,10 @@ import com.google.common.io.Resources;
import google.registry.testing.truth.TextDiffSubject.DiffFormat;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
import org.junit.jupiter.api.Test;
/** Unit tests for {@link TextDiffSubject}. */
@RunWith(JUnit4.class)
public class TextDiffSubjectTest {
class TextDiffSubjectTest {
private static final String RESOURCE_FOLDER = "google/registry/testing/truth/";
// Resources for input data.
@@ -44,21 +41,21 @@ public class TextDiffSubjectTest {
RESOURCE_FOLDER + "text-sidebyside-diff.txt";
@Test
public void unifiedDiff_equal() throws IOException {
void unifiedDiff_equal() throws IOException {
assertThat(getResource(ACTUAL_RESOURCE))
.withDiffFormat(DiffFormat.UNIFIED_DIFF)
.hasSameContentAs(getResource(ACTUAL_RESOURCE));
}
@Test
public void sideBySideDiff_equal() throws IOException {
void sideBySideDiff_equal() throws IOException {
assertThat(getResource(ACTUAL_RESOURCE))
.withDiffFormat(DiffFormat.SIDE_BY_SIDE_MARKDOWN)
.hasSameContentAs(getResource(ACTUAL_RESOURCE));
}
@Test
public void unifedDiff_notEqual() throws IOException {
void unifedDiff_notEqual() throws IOException {
assertThrows(
AssertionError.class,
() ->
@@ -68,7 +65,7 @@ public class TextDiffSubjectTest {
}
@Test
public void sideBySideDiff_notEqual() throws IOException {
void sideBySideDiff_notEqual() throws IOException {
assertThrows(
AssertionError.class,
() ->
@@ -78,13 +75,13 @@ public class TextDiffSubjectTest {
}
@Test
public void displayed_unifiedDiff_noDiff() throws IOException {
void displayed_unifiedDiff_noDiff() throws IOException {
ImmutableList<String> actual = readAllLinesFromResource(ACTUAL_RESOURCE);
assertThat(TextDiffSubject.generateUnifiedDiff(actual, actual)).isEqualTo("");
}
@Test
public void displayed_unifiedDiff_hasDiff() throws IOException {
void displayed_unifiedDiff_hasDiff() throws IOException {
ImmutableList<String> actual = readAllLinesFromResource(ACTUAL_RESOURCE);
ImmutableList<String> expected = readAllLinesFromResource(EXPECTED_RESOURCE);
String diff = Joiner.on('\n').join(readAllLinesFromResource(UNIFIED_DIFF_RESOURCE));
@@ -92,7 +89,7 @@ public class TextDiffSubjectTest {
}
@Test
public void displayed_sideBySideDiff_hasDiff() throws IOException {
void displayed_sideBySideDiff_hasDiff() throws IOException {
ImmutableList<String> actual = readAllLinesFromResource(ACTUAL_RESOURCE);
ImmutableList<String> expected = readAllLinesFromResource(EXPECTED_RESOURCE);
String diff = Joiner.on('\n').join(readAllLinesFromResource(SIDE_BY_SIDE_DIFF_RESOURCE));

View File

@@ -16,6 +16,7 @@ package google.registry.testing.truth;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.truth.Truth.assertAbout;
import static com.google.common.truth.Truth.assertWithMessage;
import static java.nio.charset.StandardCharsets.UTF_8;
import com.github.difflib.DiffUtils;
@@ -31,6 +32,7 @@ import com.google.common.collect.ImmutableList;
import com.google.common.io.Resources;
import com.google.common.truth.Fact;
import com.google.common.truth.FailureMetadata;
import com.google.common.truth.SimpleSubjectBuilder;
import com.google.common.truth.Subject;
import java.io.IOException;
import java.net.URL;
@@ -68,6 +70,15 @@ public class TextDiffSubject extends Subject {
this.actual = ImmutableList.copyOf(actual);
}
protected TextDiffSubject(FailureMetadata metadata, URL actual) {
super(metadata, actual);
try {
this.actual = ImmutableList.copyOf(Resources.asCharSource(actual, UTF_8).readLines());
} catch (IOException e) {
throw new RuntimeException(e);
}
}
public TextDiffSubject withDiffFormat(DiffFormat format) {
this.diffFormat = format;
return this;
@@ -100,6 +111,11 @@ public class TextDiffSubject extends Subject {
return assertThat(Resources.asCharSource(resourceUrl, UTF_8).readLines());
}
public static SimpleSubjectBuilder<TextDiffSubject, URL> assertWithMessageAboutUrlSource(
String format, Object... params) {
return assertWithMessage(format, params).about(urlFactory());
}
private static final Subject.Factory<TextDiffSubject, ImmutableList<String>>
TEXT_DIFF_SUBJECT_TEXT_FACTORY = TextDiffSubject::new;
@@ -107,6 +123,13 @@ public class TextDiffSubject extends Subject {
return TEXT_DIFF_SUBJECT_TEXT_FACTORY;
}
private static final Subject.Factory<TextDiffSubject, URL> TEXT_DIFF_SUBJECT_URL_FACTORY =
TextDiffSubject::new;
public static Subject.Factory<TextDiffSubject, URL> urlFactory() {
return TEXT_DIFF_SUBJECT_URL_FACTORY;
}
static String generateUnifiedDiff(
ImmutableList<String> expectedContent, ImmutableList<String> actualContent) {
Patch<String> diff;

View File

@@ -19,6 +19,7 @@ import argparse
import attr
import io
import os
import shutil
import subprocess
import sys
from typing import List, Union
@@ -49,15 +50,30 @@ PROPERTIES_HEADER = """\
# This file defines properties used by the gradle build. It must be kept in
# sync with config/nom_build.py.
#
# To regenerate, run config/nom_build.py --generate-gradle-properties
# To regenerate, run ./nom_build --generate-gradle-properties
#
# To view property descriptions (which are command line flags for
# nom_build), run config/nom_build.py --help.
# nom_build), run ./nom_build --help.
#
# DO NOT EDIT THIS FILE BY HAND
org.gradle.jvmargs=-Xmx1024m
"""
# Help text to be displayed (in addition to the synopsis and flag help, which
# are displayed automatically).
HELP_TEXT = """\
A wrapper around the gradle build that provides the following features:
- Converts properties into flags to guard against property name spelling errors
and to provide help descriptions for all properties.
- Provides pseudo-commands (with the ":nom:" prefix) that encapsulate common
actions that are difficult to implement in gradle.
Pseudo-commands:
:nom:generate_golden_file - regenerates the golden file from the current
set of flyway files.
"""
# Define all of our special gradle properties here.
PROPERTIES = [
Property('mavenUrl',
@@ -114,6 +130,11 @@ PROPERTIES = [
Property('nomulus_version',
'The version of nomulus to test against in a database '
'integration test.'),
Property('dot_path',
'The path to "dot", part of the graphviz package that converts '
'a BEAM pipeline to image. Setting this property to empty string '
'will disable image generation.',
'/usr/bin/dot'),
]
GRADLE_FLAGS = [
@@ -251,8 +272,42 @@ def get_root() -> str:
return cur_dir
def main(args):
parser = argparse.ArgumentParser('nom_build')
class Abort(Exception):
"""Raised to terminate the process with a non-zero error code.
Parameters are ignored.
"""
def do_pseudo_task(task: str) -> None:
root = get_root()
if task == ':nom:generate_golden_file':
if not subprocess.call([f'{root}/gradlew', ':db:test']):
print('\033[33mWARNING:\033[0m Golden schema appears to be '
'up-to-date. If you are making schema changes, be sure to '
'add a flyway file for them.')
return
print('\033[33mWARNING:\033[0m Ignore the above failure, it is '
'expected.')
# Copy the new schema into place.
shutil.copy(f'{root}/db/build/resources/test/testcontainer/'
'mount/dump.txt',
f'{root}/db/src/main/resources/sql/schema/'
'nomulus.golden.sql')
if subprocess.call([f'{root}/gradlew', ':db:test']):
print('\033[31mERROR:\033[0m Golden file test failed after '
'copying schema. Please check your flyway files.')
raise Abort()
else:
print(f'\033[31mERROR:\033[0m Unknown task {task}')
raise Abort()
def main(args) -> int:
parser = argparse.ArgumentParser('nom_build', description=HELP_TEXT,
formatter_class=argparse.RawTextHelpFormatter)
for prop in PROPERTIES:
parser.add_argument('--' + prop.name, default=prop.default,
help=prop.desc)
@@ -291,7 +346,7 @@ def main(args):
if args.generate_gradle_properties:
with open(f'{root}/gradle.properties', 'w') as dst:
dst.write(gradle_properties)
return
return 0
# Verify that the gradle properties file is what we expect it to be.
with open(f'{root}/gradle.properties') as src:
@@ -316,12 +371,39 @@ def main(args):
if flag.has_arg:
gradle_command.append(arg_val)
# See if there are any special ":nom:" pseudo-tasks specified.
got_non_pseudo_tasks = False
for arg in args.non_flag_args[1:]:
if arg.startswith(':nom:'):
if got_non_pseudo_tasks:
# We can't currently deal with the situation of gradle tasks
# before pseudo-tasks. This could be implemented by invoking
# gradle for only the set of gradle tasks before the pseudo
# task, but that's overkill for now.
print(f'\033[31mERROR:\033[0m Pseudo task ({arg}) must be '
'specified prior to all actual gradle tasks. Aborting.')
return 1
do_pseudo_task(arg)
else:
got_non_pseudo_tasks = True
non_flag_args = [
arg for arg in args.non_flag_args[1:] if not arg.startswith(':nom:')]
if not non_flag_args:
if not got_non_pseudo_tasks:
print('\033[33mWARNING:\033[0m No tasks specified. Not '
'doing anything')
return 0
# Add the non-flag args (we exclude the first, which is the command name
# itself) and run.
gradle_command.extend(args.non_flag_args[1:])
subprocess.call(gradle_command)
gradle_command.extend(non_flag_args)
return subprocess.call(gradle_command)
if __name__ == '__main__':
main(sys.argv)
try:
sys.exit(main(sys.argv))
except Abort as ex:
sys.exit(1)

View File

@@ -14,6 +14,7 @@
import io
import os
import shutil
import unittest
from unittest import mock
import nom_build
@@ -67,6 +68,7 @@ class MyTest(unittest.TestCase):
mock.patch.object(nom_build, 'print', self.print_fake).start())
self.call_mock = mock.patch.object(subprocess, 'call').start()
self.copy_mock = mock.patch.object(shutil, 'copy').start()
self.file_contents = {
# Prefil with the actual file contents.
@@ -92,17 +94,32 @@ class MyTest(unittest.TestCase):
def test_no_args(self):
nom_build.main(['nom_build'])
self.assertEqual(self.printed, [])
self.call_mock.assert_called_with([GRADLEW])
self.assertEqual(self.printed,
['\x1b[33mWARNING:\x1b[0m No tasks specified. Not '
'doing anything'])
def test_property_calls(self):
nom_build.main(['nom_build', '--testFilter=foo'])
self.call_mock.assert_called_with([GRADLEW, '-P', 'testFilter=foo'])
nom_build.main(['nom_build', 'task-name', '--testFilter=foo'])
self.call_mock.assert_called_with([GRADLEW, '-P', 'testFilter=foo',
'task-name'])
def test_gradle_flags(self):
nom_build.main(['nom_build', '-d', '-b', 'foo'])
nom_build.main(['nom_build', 'task-name', '-d', '-b', 'foo'])
self.call_mock.assert_called_with([GRADLEW, '--build-file', 'foo',
'--debug'])
'--debug', 'task-name'])
def test_generate_golden_file(self):
self.call_mock.side_effect = [1, 0]
nom_build.main(['nom_build', ':nom:generate_golden_file'])
self.call_mock.assert_has_calls([
mock.call([GRADLEW, ':db:test']),
mock.call([GRADLEW, ':db:test'])
])
def test_generate_golden_file_nofail(self):
self.call_mock.return_value = 0
nom_build.main(['nom_build', ':nom:generate_golden_file'])
self.call_mock.assert_has_calls([mock.call([GRADLEW, ':db:test'])])
unittest.main()

View File

@@ -22,7 +22,7 @@ import sys
import re
# We should never analyze any generated files
UNIVERSALLY_SKIPPED_PATTERNS = {"/build/", "cloudbuild-caches", "/out/"}
UNIVERSALLY_SKIPPED_PATTERNS = {"/build/", "cloudbuild-caches", "/out/", ".git/"}
# We can't rely on CI to have the Enum package installed so we do this instead.
FORBIDDEN = 1
REQUIRED = 2
@@ -77,9 +77,9 @@ PRESUBMITS = {
PresubmitCheck(
r".*Copyright 20\d{2} The Nomulus Authors\. All Rights Reserved\.",
("java", "js", "soy", "sql", "py", "sh", "gradle"), {
".git", "/build/", "/generated/", "node_modules/",
"JUnitBackports.java", "registrar_bin.", "registrar_dbg.",
"google-java-format-diff.py",
".git", "/build/", "/generated/", "/generated_tests/",
"node_modules/", "JUnitBackports.java", "registrar_bin.",
"registrar_dbg.", "google-java-format-diff.py",
"nomulus.golden.sql", "soyutils_usegoog.js"
}, REQUIRED):
"File did not include the license header.",
@@ -93,20 +93,20 @@ PRESUBMITS = {
PresubmitCheck(
r".*\bSystem\.(out|err)\.print", "java", {
"StackdriverDashboardBuilder.java", "/tools/", "/example/",
"RegistryTestServerMain.java", "TestServerRule.java",
"RegistryTestServerMain.java", "TestServerExtension.java",
"FlowDocumentationTool.java"
}):
"System.(out|err).println is only allowed in tools/ packages. Please "
"use a logger instead.",
# ObjectifyService.register is restricted to main/ or AppEngineRule.
# ObjectifyService.register is restricted to main/ or AppEngineExtension.
PresubmitCheck(
r".*\bObjectifyService\.register", "java", {
"/build/", "/generated/", "node_modules/", "src/main/",
"AppEngineRule.java"
"AppEngineExtension.java"
}):
"ObjectifyService.register is not allowed in tests. Please use "
"AppengineRule.register instead.",
"ObjectifyService.register(...) is not allowed in tests. Please use "
"AppEngineExtension.register(...) instead.",
# PostgreSQLContainer instantiation must specify docker tag
PresubmitCheck(

View File

@@ -22,7 +22,7 @@ plugins {
// Path to code generated by ad hoc tasks in this project. A separate path is
// used for easy inspection.
def generatedDir = "${project.buildDir}/generated/source/custom/main"
def generatedDir = "${project.buildDir}/generated/sources/custom/java/main"
def resourcesDir = "${project.buildDir}/resources/main"
def screenshotsDir = "${project.buildDir}/screenshots"
def screenshotsForGoldensDir = "${project.buildDir}/screenshots_for_goldens"
@@ -82,11 +82,6 @@ sourceSets {
main {
java {
srcDirs += generatedDir
// Javadoc API is deprecated in Java 11 and removed in Java 12.
// TODO(jianglai): re-enable after migrating to the new Javadoc API
if ((JavaVersion.current().majorVersion as Integer) >= 11) {
exclude 'google/registry/documentation/**'
}
}
resources {
exclude '**/*.xjb'
@@ -238,6 +233,7 @@ dependencies {
compile deps['jline:jline']
compile deps['joda-time:joda-time']
compile deps['org.apache.avro:avro']
testCompile deps['org.apache.beam:beam-runners-core-construction-java']
testCompile deps['org.apache.beam:beam-runners-direct-java']
compile deps['org.apache.beam:beam-runners-google-cloud-dataflow-java']
compile deps['org.apache.beam:beam-sdks-java-core']
@@ -256,22 +252,23 @@ dependencies {
compile deps['org.bouncycastle:bcpg-jdk15on']
testCompile deps['org.bouncycastle:bcpkix-jdk15on']
compile deps['org.bouncycastle:bcprov-jdk15on']
testCompile deps['com.fasterxml.jackson.core:jackson-databind']
runtime deps['org.glassfish.jaxb:jaxb-runtime']
compile deps['org.hibernate:hibernate-core']
compile deps['org.joda:joda-money']
compile deps['org.json:json']
testCompile deps['org.mortbay.jetty:jetty']
runtimeOnly deps['org.postgresql:postgresql']
compile deps['org.postgresql:postgresql']
testCompile deps['org.seleniumhq.selenium:selenium-api']
testCompile deps['org.seleniumhq.selenium:selenium-chrome-driver']
testCompile deps['org.seleniumhq.selenium:selenium-java']
testCompile deps['org.seleniumhq.selenium:selenium-remote-driver']
runtimeOnly deps['org.slf4j:slf4j-jdk14']
testCompile deps['org.testcontainers:jdbc']
testCompile deps['org.testcontainers:junit-jupiter']
compile deps['org.testcontainers:postgresql']
testCompile deps['org.testcontainers:selenium']
testCompile deps['org.testcontainers:testcontainers']
testCompile deps['pl.pragmatists:JUnitParams']
compile deps['xerces:xmlParserAPIs']
compile deps['xpp3:xpp3']
// This dependency must come after javax.mail:mail as it would otherwise
@@ -311,10 +308,12 @@ dependencies {
testCompile deps['org.junit.jupiter:junit-jupiter-api']
testCompile deps['org.junit.jupiter:junit-jupiter-engine']
testCompile deps['org.junit.jupiter:junit-jupiter-migrationsupport']
testCompile deps['org.junit.jupiter:junit-jupiter-params']
testCompile deps['org.junit-pioneer:junit-pioneer']
testCompile deps['org.junit.platform:junit-platform-runner']
testCompile deps['org.junit.platform:junit-platform-suite-api']
testCompile deps['org.junit.vintage:junit-vintage-engine']
testCompile deps['org.mockito:mockito-core']
testCompile deps['org.mockito:mockito-junit-jupiter']
runtime deps['org.postgresql:postgresql']
// Indirect dependency found by undeclared-dependency check. Such
@@ -405,6 +404,9 @@ task jaxbToJava {
}
}
}
execInBash(
'find . -name *.java -exec sed -i /\\*\\ \\<p\\>\\$/d {} +',
generatedDir)
}
}
@@ -703,6 +705,10 @@ task fragileTest(type: FilteringTest) {
// Run every test class in a freshly started process.
forkEvery 1
doFirst {
new File(screenshotsDir).deleteDir()
}
}
task outcastTest(type: FilteringTest) {
@@ -820,6 +826,30 @@ createToolTask(
'google.registry.tools.DevTool',
sourceSets.nonprod)
project.tasks.create('initSqlPipeline', JavaExec) {
main = 'google.registry.beam.initsql.InitSqlPipeline'
doFirst {
getToolArgsList().ifPresent {
args it
}
def isDirectRunner =
args.contains('DirectRunner') || args.contains('--runner=DirectRunner')
// The dependency containing DirectRunner is intentionally excluded from the
// production binary, so that it won't be chosen by mistake: we definitely do
// not want to use it for the real jobs, yet DirectRunner is the default if
// the user forgets to override it.
// DirectRunner is required for tests and is already on testRuntimeClasspath.
// For simplicity, we add testRuntimeClasspath to this task's classpath instead
// of defining a new configuration just for the DirectRunner dependency.
classpath =
isDirectRunner
? sourceSets.main.runtimeClasspath.plus(sourceSets.test.runtimeClasspath)
: sourceSets.main.runtimeClasspath
}
}
project.tasks.create('generateSqlSchema', JavaExec) {
classpath = sourceSets.nonprod.runtimeClasspath
main = 'google.registry.tools.DevTool'
@@ -847,22 +877,6 @@ task generateGoldenImages(type: FilteringTest) {
}
generateGoldenImages.finalizedBy(findGoldenImages)
task flowDocsTool(type: JavaExec) {
systemProperty 'test.projectRoot', rootProject.projectRootDir
systemProperty 'test.resourcesDir', resourcesDir
classpath = sourceSets.main.runtimeClasspath
main = 'google.registry.documentation.FlowDocumentationTool'
def arguments = []
if (rootProject.flowDocsFile) {
arguments << "--output_file=${rootProject.flowDocsFile}"
} else {
arguments << "--output_file=${rootProject.projectRootDir}/docs/flows.md"
}
args arguments
}
task standardTest(type: FilteringTest) {
includeAllTests()
exclude fragileTestPatterns
@@ -887,10 +901,6 @@ task standardTest(type: FilteringTest) {
systemProperty 'test.projectRoot', rootProject.projectRootDir
systemProperty 'test.resourcesDir', resourcesDir
doFirst {
new File(screenshotsDir).deleteDir()
}
}
test {
@@ -964,6 +974,49 @@ task buildToolImage(dependsOn: nomulus, type: Exec) {
commandLine 'docker', 'build', '-t', 'nomulus-tool', '.'
}
task generateInitSqlPipelineGraph(type: Test) {
include "**/InitSqlPipelineGraphTest.*"
testNameIncludePatterns = ["**createPipeline_compareGraph"]
ignoreFailures = true
}
task updateInitSqlPipelineGraph(type: Copy) {
def graphRelativePath = 'google/registry/beam/initsql/'
from ("${projectDir}/build/resources/test/${graphRelativePath}") {
include 'pipeline_curr.dot'
rename 'curr', 'golden'
}
into "src/test/resources/${graphRelativePath}"
dependsOn generateInitSqlPipelineGraph
doLast {
if (com.google.common.base.Strings.isNullOrEmpty(project.dot_path)) {
getLogger().info('Property dot_path is null. Not creating image for pipeline graph.')
}
def dotPath = project.dot_path
if (!new File(dotPath).exists()) {
throw new RuntimeException(
"""\
${dotPath} not found. Make sure graphviz is installed
and the dot_path property is set correctly."""
.stripIndent())
}
def goldenGraph = "src/test/resources/${graphRelativePath}/pipeline_golden.dot"
def goldenImage = "src/test/resources/${graphRelativePath}/pipeline_golden.png"
def cmd = "${dotPath} -Tpng -o \"${goldenImage}\" \"${goldenGraph}\""
try {
rootProject.ext.execInBash(cmd, projectDir)
} catch (Throwable throwable) {
throw new RuntimeException(
"""\
Failed to generate golden image with command ${cmd}
Error: ${throwable.getMessage()}
""")
}
}
}
// Build the devtool jar.
createUberJar(
'devtool',

View File

@@ -7,10 +7,10 @@ com.google.auto.value:auto-value:1.6.3
com.google.auto:auto-common:0.10
com.google.code.findbugs:jFormatString:3.0.0
com.google.code.findbugs:jsr305:3.0.2
com.google.dagger:dagger-compiler:2.21
com.google.dagger:dagger-producers:2.21
com.google.dagger:dagger-spi:2.21
com.google.dagger:dagger:2.21
com.google.dagger:dagger-compiler:2.28
com.google.dagger:dagger-producers:2.28
com.google.dagger:dagger-spi:2.28
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotation:2.3.3
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_check_api:2.3.3
@@ -19,7 +19,7 @@ com.google.errorprone:error_prone_type_annotations:2.3.3
com.google.errorprone:javac-shaded:9-dev-r4023-3
com.google.googlejavaformat:google-java-format:1.5
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.protobuf:protobuf-java:3.4.0
@@ -28,8 +28,13 @@ com.squareup:javapoet:1.12.1
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.persistence:javax.persistence-api:2.2
net.ltgt.gradle.incap:incap:0.2
org.checkerframework:checker-compat-qual:2.5.3
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.checkerframework:dataflow:2.5.3
org.checkerframework:javacutil:2.5.3
org.jetbrains.kotlin:kotlin-stdlib-common:1.3.61
org.jetbrains.kotlin:kotlin-stdlib:1.3.61
org.jetbrains.kotlinx:kotlinx-metadata-jvm:0.1.0
org.jetbrains:annotations:13.0
org.pcollections:pcollections:2.1.2

View File

@@ -10,14 +10,14 @@ com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.github.jnr:jffi:1.2.17
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.9.11
com.github.jnr:jnr-enxio:0.19
com.github.jnr:jnr-ffi:2.1.9
com.github.jnr:jnr-posix:3.0.47
com.github.jnr:jnr-unixsocket:0.21
com.github.jnr:jnr-constants:0.9.15
com.github.jnr:jnr-enxio:0.26
com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.27.0
@@ -59,7 +59,7 @@ com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev56-1.23.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20190827-1.30.1
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
@@ -67,15 +67,15 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud.sql:jdbc-socket-factory-core:1.0.12
com.google.cloud.sql:jdbc-socket-factory-core:1.0.16
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
@@ -86,13 +86,13 @@ com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.3.1
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
@@ -164,7 +164,7 @@ io.opencensus:opencensus-contrib-http-util:0.24.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:javax.annotation-api:1.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
@@ -177,9 +177,9 @@ javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.12
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
net.bytebuddy:byte-buddy:1.10.10
net.java.dev.jna:jna-platform:5.5.0
net.java.dev.jna:jna:5.5.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
@@ -192,30 +192,30 @@ org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpcore:4.4.13
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.17
org.dom4j:dom4j:2.1.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.hamcrest:hamcrest-core:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.hibernate:hibernate-core:5.4.17.Final
org.hibernate:hibernate-hikaricp:5.4.17.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.jboss:jandex:2.1.3.Final
org.jetbrains:annotations:19.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.jvnet.staxex:stax-ex:1.8
@@ -223,20 +223,21 @@ org.mockito:mockito-core:1.9.5
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.ow2.asm:asm-analysis:8.0.1
org.ow2.asm:asm-commons:7.1
org.ow2.asm:asm-tree:8.0.1
org.ow2.asm:asm-util:8.0.1
org.ow2.asm:asm:8.0.1
org.postgresql:postgresql:42.2.14
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
org.scijava:native-lib-loader:2.0.2
org.slf4j:slf4j-api:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:testcontainers:1.12.1
org.slf4j:slf4j-api:1.7.30
org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3

View File

@@ -10,14 +10,14 @@ com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.github.jnr:jffi:1.2.17
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.9.11
com.github.jnr:jnr-enxio:0.19
com.github.jnr:jnr-ffi:2.1.9
com.github.jnr:jnr-posix:3.0.47
com.github.jnr:jnr-unixsocket:0.21
com.github.jnr:jnr-constants:0.9.15
com.github.jnr:jnr-enxio:0.26
com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.27.0
@@ -59,7 +59,7 @@ com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev56-1.23.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20190827-1.30.1
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
@@ -67,15 +67,15 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud.sql:jdbc-socket-factory-core:1.0.12
com.google.cloud.sql:jdbc-socket-factory-core:1.0.16
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
@@ -86,12 +86,12 @@ com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
@@ -162,7 +162,7 @@ io.opencensus:opencensus-contrib-http-util:0.24.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:javax.annotation-api:1.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
@@ -174,9 +174,9 @@ javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
net.bytebuddy:byte-buddy:1.10.10
net.java.dev.jna:jna-platform:5.5.0
net.java.dev.jna:jna:5.5.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
@@ -189,49 +189,50 @@ org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpcore:4.4.13
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.17
org.dom4j:dom4j:2.1.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.hibernate:hibernate-core:5.4.17.Final
org.hibernate:hibernate-hikaricp:5.4.17.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.jboss:jandex:2.1.3.Final
org.jetbrains:annotations:19.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.jvnet.staxex:stax-ex:1.8
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.ow2.asm:asm-analysis:8.0.1
org.ow2.asm:asm-commons:7.1
org.ow2.asm:asm-tree:8.0.1
org.ow2.asm:asm-util:8.0.1
org.ow2.asm:asm:8.0.1
org.postgresql:postgresql:42.2.14
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
org.scijava:native-lib-loader:2.0.2
org.slf4j:slf4j-api:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:testcontainers:1.12.1
org.slf4j:slf4j-api:1.7.30
org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3

View File

@@ -10,14 +10,14 @@ com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.github.jnr:jffi:1.2.17
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.9.11
com.github.jnr:jnr-enxio:0.19
com.github.jnr:jnr-ffi:2.1.9
com.github.jnr:jnr-posix:3.0.47
com.github.jnr:jnr-unixsocket:0.21
com.github.jnr:jnr-constants:0.9.15
com.github.jnr:jnr-enxio:0.26
com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.27.0
@@ -59,7 +59,7 @@ com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev56-1.23.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20190827-1.30.1
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
@@ -67,16 +67,16 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud.sql:jdbc-socket-factory-core:1.0.12
com.google.cloud.sql:postgres-socket-factory:1.0.12
com.google.cloud.sql:jdbc-socket-factory-core:1.0.16
com.google.cloud.sql:postgres-socket-factory:1.0.16
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
@@ -87,13 +87,13 @@ com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.3.1
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
@@ -165,7 +165,7 @@ io.opencensus:opencensus-contrib-http-util:0.24.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:javax.annotation-api:1.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
@@ -178,9 +178,9 @@ javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.12
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
net.bytebuddy:byte-buddy:1.10.10
net.java.dev.jna:jna-platform:5.5.0
net.java.dev.jna:jna:5.5.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
@@ -193,30 +193,30 @@ org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpcore:4.4.13
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.17
org.dom4j:dom4j:2.1.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.hamcrest:hamcrest-core:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.hibernate:hibernate-core:5.4.17.Final
org.hibernate:hibernate-hikaricp:5.4.17.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.jboss:jandex:2.1.3.Final
org.jetbrains:annotations:19.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.jvnet.staxex:stax-ex:1.8
@@ -224,22 +224,22 @@ org.mockito:mockito-core:1.9.5
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.postgresql:postgresql:42.2.6
org.ow2.asm:asm-analysis:8.0.1
org.ow2.asm:asm-commons:7.1
org.ow2.asm:asm-tree:8.0.1
org.ow2.asm:asm-util:8.0.1
org.ow2.asm:asm:8.0.1
org.postgresql:postgresql:42.2.14
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
org.scijava:native-lib-loader:2.0.2
org.slf4j:slf4j-api:1.7.28
org.slf4j:slf4j-api:1.7.30
org.slf4j:slf4j-jdk14:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:testcontainers:1.12.1
org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3

View File

@@ -10,14 +10,14 @@ com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.github.jnr:jffi:1.2.17
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.9.11
com.github.jnr:jnr-enxio:0.19
com.github.jnr:jnr-ffi:2.1.9
com.github.jnr:jnr-posix:3.0.47
com.github.jnr:jnr-unixsocket:0.21
com.github.jnr:jnr-constants:0.9.15
com.github.jnr:jnr-enxio:0.26
com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.27.0
@@ -59,7 +59,7 @@ com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev56-1.23.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20190827-1.30.1
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
@@ -67,16 +67,16 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud.sql:jdbc-socket-factory-core:1.0.12
com.google.cloud.sql:postgres-socket-factory:1.0.12
com.google.cloud.sql:jdbc-socket-factory-core:1.0.16
com.google.cloud.sql:postgres-socket-factory:1.0.16
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
@@ -87,13 +87,13 @@ com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.3.1
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
@@ -165,7 +165,7 @@ io.opencensus:opencensus-contrib-http-util:0.24.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:javax.annotation-api:1.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
@@ -177,9 +177,9 @@ javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
net.bytebuddy:byte-buddy:1.10.10
net.java.dev.jna:jna-platform:5.5.0
net.java.dev.jna:jna:5.5.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
@@ -192,51 +192,51 @@ org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpcore:4.4.13
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.17
org.dom4j:dom4j:2.1.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.hibernate:hibernate-core:5.4.17.Final
org.hibernate:hibernate-hikaricp:5.4.17.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.jboss:jandex:2.1.3.Final
org.jetbrains:annotations:19.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.jvnet.staxex:stax-ex:1.8
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.postgresql:postgresql:42.2.6
org.ow2.asm:asm-analysis:8.0.1
org.ow2.asm:asm-commons:7.1
org.ow2.asm:asm-tree:8.0.1
org.ow2.asm:asm-util:8.0.1
org.ow2.asm:asm:8.0.1
org.postgresql:postgresql:42.2.14
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
org.scijava:native-lib-loader:2.0.2
org.slf4j:slf4j-api:1.7.28
org.slf4j:slf4j-api:1.7.30
org.slf4j:slf4j-jdk14:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:testcontainers:1.12.1
org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3

View File

@@ -1,4 +1,3 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.errorprone:javac:9+181-r4173-1

View File

@@ -10,14 +10,14 @@ com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.github.jnr:jffi:1.2.17
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.9.11
com.github.jnr:jnr-enxio:0.19
com.github.jnr:jnr-ffi:2.1.9
com.github.jnr:jnr-posix:3.0.47
com.github.jnr:jnr-unixsocket:0.21
com.github.jnr:jnr-constants:0.9.15
com.github.jnr:jnr-enxio:0.26
com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.27.0
@@ -59,7 +59,7 @@ com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev56-1.23.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20190827-1.30.1
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
@@ -67,15 +67,15 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud.sql:jdbc-socket-factory-core:1.0.12
com.google.cloud.sql:jdbc-socket-factory-core:1.0.16
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
@@ -86,13 +86,13 @@ com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.3.1
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
@@ -164,7 +164,7 @@ io.opencensus:opencensus-contrib-http-util:0.24.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:javax.annotation-api:1.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
@@ -177,9 +177,9 @@ javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.12
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
net.bytebuddy:byte-buddy:1.10.10
net.java.dev.jna:jna-platform:5.5.0
net.java.dev.jna:jna:5.5.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
@@ -192,30 +192,30 @@ org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpcore:4.4.13
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.17
org.dom4j:dom4j:2.1.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.hamcrest:hamcrest-core:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.hibernate:hibernate-core:5.4.17.Final
org.hibernate:hibernate-hikaricp:5.4.17.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.jboss:jandex:2.1.3.Final
org.jetbrains:annotations:19.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.jvnet.staxex:stax-ex:1.8
@@ -223,20 +223,21 @@ org.mockito:mockito-core:1.9.5
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.ow2.asm:asm-analysis:8.0.1
org.ow2.asm:asm-commons:7.1
org.ow2.asm:asm-tree:8.0.1
org.ow2.asm:asm-util:8.0.1
org.ow2.asm:asm:8.0.1
org.postgresql:postgresql:42.2.14
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
org.scijava:native-lib-loader:2.0.2
org.slf4j:slf4j-api:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:testcontainers:1.12.1
org.slf4j:slf4j-api:1.7.30
org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3

View File

@@ -10,14 +10,14 @@ com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.github.jnr:jffi:1.2.17
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.9.11
com.github.jnr:jnr-enxio:0.19
com.github.jnr:jnr-ffi:2.1.9
com.github.jnr:jnr-posix:3.0.47
com.github.jnr:jnr-unixsocket:0.21
com.github.jnr:jnr-constants:0.9.15
com.github.jnr:jnr-enxio:0.26
com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.27.0
@@ -59,7 +59,7 @@ com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev56-1.23.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20190827-1.30.1
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
@@ -67,15 +67,15 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud.sql:jdbc-socket-factory-core:1.0.12
com.google.cloud.sql:jdbc-socket-factory-core:1.0.16
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
@@ -86,12 +86,12 @@ com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
@@ -162,7 +162,7 @@ io.opencensus:opencensus-contrib-http-util:0.24.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:javax.annotation-api:1.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
@@ -175,9 +175,9 @@ javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.12
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
net.bytebuddy:byte-buddy:1.10.10
net.java.dev.jna:jna-platform:5.5.0
net.java.dev.jna:jna:5.5.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
@@ -190,30 +190,30 @@ org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpcore:4.4.13
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.17
org.dom4j:dom4j:2.1.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.hamcrest:hamcrest-core:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.hibernate:hibernate-core:5.4.17.Final
org.hibernate:hibernate-hikaricp:5.4.17.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.jboss:jandex:2.1.3.Final
org.jetbrains:annotations:19.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.jvnet.staxex:stax-ex:1.8
@@ -221,20 +221,21 @@ org.mockito:mockito-core:1.9.5
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.ow2.asm:asm-analysis:8.0.1
org.ow2.asm:asm-commons:7.1
org.ow2.asm:asm-tree:8.0.1
org.ow2.asm:asm-util:8.0.1
org.ow2.asm:asm:8.0.1
org.postgresql:postgresql:42.2.14
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
org.scijava:native-lib-loader:2.0.2
org.slf4j:slf4j-api:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:testcontainers:1.12.1
org.slf4j:slf4j-api:1.7.30
org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3

View File

@@ -10,14 +10,14 @@ com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.github.jnr:jffi:1.2.17
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.9.11
com.github.jnr:jnr-enxio:0.19
com.github.jnr:jnr-ffi:2.1.9
com.github.jnr:jnr-posix:3.0.47
com.github.jnr:jnr-unixsocket:0.21
com.github.jnr:jnr-constants:0.9.15
com.github.jnr:jnr-enxio:0.26
com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.27.0
@@ -59,7 +59,7 @@ com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev56-1.23.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20190827-1.30.1
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
@@ -67,15 +67,15 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud.sql:jdbc-socket-factory-core:1.0.12
com.google.cloud.sql:jdbc-socket-factory-core:1.0.16
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
@@ -86,13 +86,13 @@ com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.3.1
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
@@ -164,7 +164,7 @@ io.opencensus:opencensus-contrib-http-util:0.24.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:javax.annotation-api:1.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
@@ -177,9 +177,9 @@ javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.12
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
net.bytebuddy:byte-buddy:1.10.10
net.java.dev.jna:jna-platform:5.5.0
net.java.dev.jna:jna:5.5.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
@@ -192,30 +192,30 @@ org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpcore:4.4.13
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.17
org.dom4j:dom4j:2.1.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.hamcrest:hamcrest-core:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.hibernate:hibernate-core:5.4.17.Final
org.hibernate:hibernate-hikaricp:5.4.17.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.jboss:jandex:2.1.3.Final
org.jetbrains:annotations:19.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.jvnet.staxex:stax-ex:1.8
@@ -223,21 +223,21 @@ org.mockito:mockito-core:1.9.5
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.postgresql:postgresql:42.2.6
org.ow2.asm:asm-analysis:8.0.1
org.ow2.asm:asm-commons:7.1
org.ow2.asm:asm-tree:8.0.1
org.ow2.asm:asm-util:8.0.1
org.ow2.asm:asm:8.0.1
org.postgresql:postgresql:42.2.14
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
org.scijava:native-lib-loader:2.0.2
org.slf4j:slf4j-api:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:testcontainers:1.12.1
org.slf4j:slf4j-api:1.7.30
org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3

View File

@@ -10,14 +10,14 @@ com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.github.jnr:jffi:1.2.17
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.9.11
com.github.jnr:jnr-enxio:0.19
com.github.jnr:jnr-ffi:2.1.9
com.github.jnr:jnr-posix:3.0.47
com.github.jnr:jnr-unixsocket:0.21
com.github.jnr:jnr-constants:0.9.15
com.github.jnr:jnr-enxio:0.26
com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.27.0
@@ -59,7 +59,7 @@ com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev56-1.23.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20190827-1.30.1
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
@@ -67,15 +67,15 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud.sql:jdbc-socket-factory-core:1.0.12
com.google.cloud.sql:jdbc-socket-factory-core:1.0.16
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
@@ -86,13 +86,13 @@ com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.3.1
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
@@ -164,7 +164,7 @@ io.opencensus:opencensus-contrib-http-util:0.24.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:javax.annotation-api:1.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
@@ -177,9 +177,9 @@ javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.12
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
net.bytebuddy:byte-buddy:1.10.10
net.java.dev.jna:jna-platform:5.5.0
net.java.dev.jna:jna:5.5.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
@@ -192,30 +192,30 @@ org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpcore:4.4.13
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.17
org.dom4j:dom4j:2.1.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.hamcrest:hamcrest-core:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.hibernate:hibernate-core:5.4.17.Final
org.hibernate:hibernate-hikaricp:5.4.17.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.jboss:jandex:2.1.3.Final
org.jetbrains:annotations:19.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.jvnet.staxex:stax-ex:1.8
@@ -223,21 +223,21 @@ org.mockito:mockito-core:1.9.5
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.postgresql:postgresql:42.2.6
org.ow2.asm:asm-analysis:8.0.1
org.ow2.asm:asm-commons:7.1
org.ow2.asm:asm-tree:8.0.1
org.ow2.asm:asm-util:8.0.1
org.ow2.asm:asm:8.0.1
org.postgresql:postgresql:42.2.14
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
org.scijava:native-lib-loader:2.0.2
org.slf4j:slf4j-api:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:testcontainers:1.12.1
org.slf4j:slf4j-api:1.7.30
org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3

View File

@@ -10,14 +10,14 @@ com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.github.jnr:jffi:1.2.17
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.9.11
com.github.jnr:jnr-enxio:0.19
com.github.jnr:jnr-ffi:2.1.9
com.github.jnr:jnr-posix:3.0.47
com.github.jnr:jnr-unixsocket:0.21
com.github.jnr:jnr-constants:0.9.15
com.github.jnr:jnr-enxio:0.26
com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.27.0
@@ -59,7 +59,7 @@ com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev56-1.23.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20190827-1.30.1
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
@@ -67,15 +67,15 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud.sql:jdbc-socket-factory-core:1.0.12
com.google.cloud.sql:jdbc-socket-factory-core:1.0.16
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
@@ -86,13 +86,13 @@ com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.3.1
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
@@ -164,7 +164,7 @@ io.opencensus:opencensus-contrib-http-util:0.24.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:javax.annotation-api:1.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
@@ -177,9 +177,9 @@ javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.12
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
net.bytebuddy:byte-buddy:1.10.10
net.java.dev.jna:jna-platform:5.5.0
net.java.dev.jna:jna:5.5.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
@@ -192,30 +192,30 @@ org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpcore:4.4.13
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.17
org.dom4j:dom4j:2.1.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.hamcrest:hamcrest-core:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.hibernate:hibernate-core:5.4.17.Final
org.hibernate:hibernate-hikaricp:5.4.17.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.jboss:jandex:2.1.3.Final
org.jetbrains:annotations:19.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.jvnet.staxex:stax-ex:1.8
@@ -223,21 +223,21 @@ org.mockito:mockito-core:1.9.5
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.postgresql:postgresql:42.2.6
org.ow2.asm:asm-analysis:8.0.1
org.ow2.asm:asm-commons:7.1
org.ow2.asm:asm-tree:8.0.1
org.ow2.asm:asm-util:8.0.1
org.ow2.asm:asm:8.0.1
org.postgresql:postgresql:42.2.14
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
org.scijava:native-lib-loader:2.0.2
org.slf4j:slf4j-api:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:testcontainers:1.12.1
org.slf4j:slf4j-api:1.7.30
org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3

View File

@@ -10,14 +10,14 @@ com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.github.jnr:jffi:1.2.17
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.9.11
com.github.jnr:jnr-enxio:0.19
com.github.jnr:jnr-ffi:2.1.9
com.github.jnr:jnr-posix:3.0.47
com.github.jnr:jnr-unixsocket:0.21
com.github.jnr:jnr-constants:0.9.15
com.github.jnr:jnr-enxio:0.26
com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.27.0
@@ -59,7 +59,7 @@ com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev56-1.23.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20190827-1.30.1
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
@@ -67,16 +67,16 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud.sql:jdbc-socket-factory-core:1.0.12
com.google.cloud.sql:postgres-socket-factory:1.0.12
com.google.cloud.sql:jdbc-socket-factory-core:1.0.16
com.google.cloud.sql:postgres-socket-factory:1.0.16
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
@@ -87,13 +87,13 @@ com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.3.1
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
@@ -165,7 +165,7 @@ io.opencensus:opencensus-contrib-http-util:0.24.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:javax.annotation-api:1.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
@@ -177,9 +177,9 @@ javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
net.bytebuddy:byte-buddy:1.10.10
net.java.dev.jna:jna-platform:5.5.0
net.java.dev.jna:jna:5.5.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
@@ -192,51 +192,51 @@ org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.11
org.apache.httpcomponents:httpcore:4.4.13
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.17
org.dom4j:dom4j:2.1.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.hibernate:hibernate-core:5.4.17.Final
org.hibernate:hibernate-hikaricp:5.4.17.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.jboss:jandex:2.1.3.Final
org.jetbrains:annotations:19.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.jvnet.staxex:stax-ex:1.8
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.postgresql:postgresql:42.2.6
org.ow2.asm:asm-analysis:8.0.1
org.ow2.asm:asm-commons:7.1
org.ow2.asm:asm-tree:8.0.1
org.ow2.asm:asm-util:8.0.1
org.ow2.asm:asm:8.0.1
org.postgresql:postgresql:42.2.14
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
org.scijava:native-lib-loader:2.0.2
org.slf4j:slf4j-api:1.7.28
org.slf4j:slf4j-api:1.7.30
org.slf4j:slf4j-jdk14:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:testcontainers:1.12.1
org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3

View File

@@ -7,10 +7,10 @@ com.google.auto.value:auto-value:1.6.3
com.google.auto:auto-common:0.10
com.google.code.findbugs:jFormatString:3.0.0
com.google.code.findbugs:jsr305:3.0.2
com.google.dagger:dagger-compiler:2.21
com.google.dagger:dagger-producers:2.21
com.google.dagger:dagger-spi:2.21
com.google.dagger:dagger:2.21
com.google.dagger:dagger-compiler:2.28
com.google.dagger:dagger-producers:2.28
com.google.dagger:dagger-spi:2.28
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotation:2.3.3
com.google.errorprone:error_prone_annotations:2.3.4
com.google.errorprone:error_prone_check_api:2.3.3
@@ -19,7 +19,7 @@ com.google.errorprone:error_prone_type_annotations:2.3.3
com.google.errorprone:javac-shaded:9-dev-r4023-3
com.google.googlejavaformat:google-java-format:1.5
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.2-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.protobuf:protobuf-java:3.4.0
@@ -28,8 +28,13 @@ com.squareup:javapoet:1.12.1
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.persistence:javax.persistence-api:2.2
net.ltgt.gradle.incap:incap:0.2
org.checkerframework:checker-compat-qual:2.5.3
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.checkerframework:dataflow:2.5.3
org.checkerframework:javacutil:2.5.3
org.jetbrains.kotlin:kotlin-stdlib-common:1.3.61
org.jetbrains.kotlin:kotlin-stdlib:1.3.61
org.jetbrains.kotlinx:kotlinx-metadata-jvm:0.1.0
org.jetbrains:annotations:13.0
org.pcollections:pcollections:2.1.2

View File

@@ -1,3 +0,0 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.

View File

@@ -10,14 +10,14 @@ com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.github.jnr:jffi:1.2.17
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.9.11
com.github.jnr:jnr-enxio:0.19
com.github.jnr:jnr-ffi:2.1.9
com.github.jnr:jnr-posix:3.0.47
com.github.jnr:jnr-unixsocket:0.21
com.github.jnr:jnr-constants:0.9.15
com.github.jnr:jnr-enxio:0.26
com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.27.0
@@ -59,7 +59,7 @@ com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev56-1.23.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20190827-1.30.1
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
@@ -68,15 +68,15 @@ com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-api-stubs:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud.sql:jdbc-socket-factory-core:1.0.12
com.google.cloud.sql:jdbc-socket-factory-core:1.0.16
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
@@ -87,14 +87,14 @@ com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.3.1
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava-testlib:28.2-jre
com.google.guava:guava:28.2-jre
com.google.guava:guava-testlib:29.0-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
@@ -186,10 +186,10 @@ javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.13
net.bytebuddy:byte-buddy-agent:1.9.7
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
net.bytebuddy:byte-buddy-agent:1.10.5
net.bytebuddy:byte-buddy:1.10.10
net.java.dev.jna:jna-platform:5.5.0
net.java.dev.jna:jna:5.5.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
@@ -203,7 +203,7 @@ org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
@@ -221,11 +221,11 @@ org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcpkix-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.17
org.dom4j:dom4j:2.1.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.glassfish.jaxb:jaxb-runtime:2.3.1
@@ -234,36 +234,39 @@ org.hamcrest:hamcrest-all:1.3
org.hamcrest:hamcrest-core:1.3
org.hamcrest:hamcrest-library:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.hibernate:hibernate-core:5.4.17.Final
org.hibernate:hibernate-hikaricp:5.4.17.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.jboss:jandex:2.1.3.Final
org.jetbrains:annotations:19.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.junit.jupiter:junit-jupiter-api:5.6.1
org.junit.jupiter:junit-jupiter-engine:5.6.1
org.junit.jupiter:junit-jupiter-migrationsupport:5.6.1
org.junit.platform:junit-platform-commons:1.6.1
org.junit.platform:junit-platform-engine:1.6.1
org.junit.platform:junit-platform-launcher:1.6.1
org.junit.platform:junit-platform-runner:1.6.1
org.junit.platform:junit-platform-suite-api:1.6.1
org.junit.vintage:junit-vintage-engine:5.6.1
org.junit:junit-bom:5.6.1
org.junit-pioneer:junit-pioneer:0.7.0
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.jupiter:junit-jupiter-migrationsupport:5.6.2
org.junit.jupiter:junit-jupiter-params:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit.platform:junit-platform-launcher:1.6.2
org.junit.platform:junit-platform-runner:1.6.2
org.junit.platform:junit-platform-suite-api:1.6.2
org.junit:junit-bom:5.6.2
org.jvnet.staxex:stax-ex:1.8
org.mockito:mockito-core:2.25.0
org.mockito:mockito-core:3.3.3
org.mockito:mockito-junit-jupiter:3.3.3
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:2.6
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.ow2.asm:asm-analysis:8.0.1
org.ow2.asm:asm-commons:7.1
org.ow2.asm:asm-tree:8.0.1
org.ow2.asm:asm-util:8.0.1
org.ow2.asm:asm:8.0.1
org.postgresql:postgresql:42.2.14
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
@@ -278,17 +281,17 @@ org.seleniumhq.selenium:selenium-opera-driver:3.141.59
org.seleniumhq.selenium:selenium-remote-driver:3.141.59
org.seleniumhq.selenium:selenium-safari-driver:3.141.59
org.seleniumhq.selenium:selenium-support:3.141.59
org.slf4j:slf4j-api:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:selenium:1.12.1
org.testcontainers:testcontainers:1.12.1
org.slf4j:slf4j-api:1.7.30
org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:junit-jupiter:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:selenium:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4
org.yaml:snakeyaml:1.17
pl.pragmatists:JUnitParams:1.1.1
xerces:xmlParserAPIs:2.6.2
xpp3:xpp3:1.1.4c

View File

@@ -10,14 +10,14 @@ com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.github.jnr:jffi:1.2.17
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.9.11
com.github.jnr:jnr-enxio:0.19
com.github.jnr:jnr-ffi:2.1.9
com.github.jnr:jnr-posix:3.0.47
com.github.jnr:jnr-unixsocket:0.21
com.github.jnr:jnr-constants:0.9.15
com.github.jnr:jnr-enxio:0.26
com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.27.0
@@ -59,7 +59,7 @@ com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev56-1.23.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20190827-1.30.1
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
@@ -68,15 +68,15 @@ com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-api-stubs:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud.sql:jdbc-socket-factory-core:1.0.12
com.google.cloud.sql:jdbc-socket-factory-core:1.0.16
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
@@ -87,13 +87,13 @@ com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava-testlib:28.2-jre
com.google.guava:guava:28.2-jre
com.google.guava:guava-testlib:29.0-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
@@ -184,10 +184,10 @@ javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.13
net.bytebuddy:byte-buddy-agent:1.9.7
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
net.bytebuddy:byte-buddy-agent:1.10.5
net.bytebuddy:byte-buddy:1.10.10
net.java.dev.jna:jna-platform:5.5.0
net.java.dev.jna:jna:5.5.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
@@ -201,7 +201,7 @@ org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
@@ -219,11 +219,11 @@ org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcpkix-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.17
org.dom4j:dom4j:2.1.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.glassfish.jaxb:jaxb-runtime:2.3.1
@@ -232,36 +232,39 @@ org.hamcrest:hamcrest-all:1.3
org.hamcrest:hamcrest-core:1.3
org.hamcrest:hamcrest-library:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.hibernate:hibernate-core:5.4.17.Final
org.hibernate:hibernate-hikaricp:5.4.17.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.jboss:jandex:2.1.3.Final
org.jetbrains:annotations:19.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.junit.jupiter:junit-jupiter-api:5.6.1
org.junit.jupiter:junit-jupiter-engine:5.6.1
org.junit.jupiter:junit-jupiter-migrationsupport:5.6.1
org.junit.platform:junit-platform-commons:1.6.1
org.junit.platform:junit-platform-engine:1.6.1
org.junit.platform:junit-platform-launcher:1.6.1
org.junit.platform:junit-platform-runner:1.6.1
org.junit.platform:junit-platform-suite-api:1.6.1
org.junit.vintage:junit-vintage-engine:5.6.1
org.junit:junit-bom:5.6.1
org.junit-pioneer:junit-pioneer:0.7.0
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.jupiter:junit-jupiter-migrationsupport:5.6.2
org.junit.jupiter:junit-jupiter-params:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit.platform:junit-platform-launcher:1.6.2
org.junit.platform:junit-platform-runner:1.6.2
org.junit.platform:junit-platform-suite-api:1.6.2
org.junit:junit-bom:5.6.2
org.jvnet.staxex:stax-ex:1.8
org.mockito:mockito-core:2.25.0
org.mockito:mockito-core:3.3.3
org.mockito:mockito-junit-jupiter:3.3.3
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:2.6
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.ow2.asm:asm-analysis:8.0.1
org.ow2.asm:asm-commons:7.1
org.ow2.asm:asm-tree:8.0.1
org.ow2.asm:asm-util:8.0.1
org.ow2.asm:asm:8.0.1
org.postgresql:postgresql:42.2.14
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
@@ -276,17 +279,17 @@ org.seleniumhq.selenium:selenium-opera-driver:3.141.59
org.seleniumhq.selenium:selenium-remote-driver:3.141.59
org.seleniumhq.selenium:selenium-safari-driver:3.141.59
org.seleniumhq.selenium:selenium-support:3.141.59
org.slf4j:slf4j-api:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:selenium:1.12.1
org.testcontainers:testcontainers:1.12.1
org.slf4j:slf4j-api:1.7.30
org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:junit-jupiter:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:selenium:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4
org.yaml:snakeyaml:1.17
pl.pragmatists:JUnitParams:1.1.1
xerces:xmlParserAPIs:2.6.2
xpp3:xpp3:1.1.4c

View File

@@ -10,14 +10,14 @@ com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.github.jnr:jffi:1.2.17
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.9.11
com.github.jnr:jnr-enxio:0.19
com.github.jnr:jnr-ffi:2.1.9
com.github.jnr:jnr-posix:3.0.47
com.github.jnr:jnr-unixsocket:0.21
com.github.jnr:jnr-constants:0.9.15
com.github.jnr:jnr-enxio:0.26
com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.27.0
@@ -59,7 +59,7 @@ com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev56-1.23.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20190827-1.30.1
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
@@ -68,16 +68,16 @@ com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-api-stubs:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud.sql:jdbc-socket-factory-core:1.0.12
com.google.cloud.sql:postgres-socket-factory:1.0.12
com.google.cloud.sql:jdbc-socket-factory-core:1.0.16
com.google.cloud.sql:postgres-socket-factory:1.0.16
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
@@ -88,14 +88,14 @@ com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.3.1
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava-testlib:28.2-jre
com.google.guava:guava:28.2-jre
com.google.guava:guava-testlib:29.0-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
@@ -188,10 +188,10 @@ javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.13
net.bytebuddy:byte-buddy-agent:1.9.7
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
net.bytebuddy:byte-buddy-agent:1.10.5
net.bytebuddy:byte-buddy:1.10.10
net.java.dev.jna:jna-platform:5.5.0
net.java.dev.jna:jna:5.5.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
@@ -205,7 +205,7 @@ org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
@@ -223,11 +223,11 @@ org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcpkix-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.17
org.dom4j:dom4j:2.1.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.flywaydb:flyway-core:5.2.4
@@ -237,37 +237,39 @@ org.hamcrest:hamcrest-all:1.3
org.hamcrest:hamcrest-core:1.3
org.hamcrest:hamcrest-library:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.hibernate:hibernate-core:5.4.17.Final
org.hibernate:hibernate-hikaricp:5.4.17.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.jboss:jandex:2.1.3.Final
org.jetbrains:annotations:19.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.junit.jupiter:junit-jupiter-api:5.6.1
org.junit.jupiter:junit-jupiter-engine:5.6.1
org.junit.jupiter:junit-jupiter-migrationsupport:5.6.1
org.junit.platform:junit-platform-commons:1.6.1
org.junit.platform:junit-platform-engine:1.6.1
org.junit.platform:junit-platform-launcher:1.6.1
org.junit.platform:junit-platform-runner:1.6.1
org.junit.platform:junit-platform-suite-api:1.6.1
org.junit.vintage:junit-vintage-engine:5.6.1
org.junit:junit-bom:5.6.1
org.junit-pioneer:junit-pioneer:0.7.0
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.jupiter:junit-jupiter-migrationsupport:5.6.2
org.junit.jupiter:junit-jupiter-params:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit.platform:junit-platform-launcher:1.6.2
org.junit.platform:junit-platform-runner:1.6.2
org.junit.platform:junit-platform-suite-api:1.6.2
org.junit:junit-bom:5.6.2
org.jvnet.staxex:stax-ex:1.8
org.mockito:mockito-core:2.25.0
org.mockito:mockito-core:3.3.3
org.mockito:mockito-junit-jupiter:3.3.3
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:2.6
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.postgresql:postgresql:42.2.6
org.ow2.asm:asm-analysis:8.0.1
org.ow2.asm:asm-commons:7.1
org.ow2.asm:asm-tree:8.0.1
org.ow2.asm:asm-util:8.0.1
org.ow2.asm:asm:8.0.1
org.postgresql:postgresql:42.2.14
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
@@ -282,17 +284,17 @@ org.seleniumhq.selenium:selenium-opera-driver:3.141.59
org.seleniumhq.selenium:selenium-remote-driver:3.141.59
org.seleniumhq.selenium:selenium-safari-driver:3.141.59
org.seleniumhq.selenium:selenium-support:3.141.59
org.slf4j:slf4j-api:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:selenium:1.12.1
org.testcontainers:testcontainers:1.12.1
org.slf4j:slf4j-api:1.7.30
org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:junit-jupiter:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:selenium:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4
org.yaml:snakeyaml:1.17
pl.pragmatists:JUnitParams:1.1.1
xerces:xmlParserAPIs:2.6.2
xpp3:xpp3:1.1.4c

View File

@@ -10,14 +10,14 @@ com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.10.2
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.github.jnr:jffi:1.2.17
com.fasterxml:classmate:1.5.1
com.github.jnr:jffi:1.2.23
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.9.11
com.github.jnr:jnr-enxio:0.19
com.github.jnr:jnr-ffi:2.1.9
com.github.jnr:jnr-posix:3.0.47
com.github.jnr:jnr-unixsocket:0.21
com.github.jnr:jnr-constants:0.9.15
com.github.jnr:jnr-enxio:0.26
com.github.jnr:jnr-ffi:2.1.13
com.github.jnr:jnr-posix:3.0.55
com.github.jnr:jnr-unixsocket:0.30
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.30.8
com.google.api-client:google-api-client-jackson2:1.27.0
@@ -59,7 +59,7 @@ com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev56-1.23.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20190827-1.30.1
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
@@ -68,16 +68,16 @@ com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-api-stubs:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auth:google-auth-library-credentials:0.20.0
com.google.auth:google-auth-library-oauth2-http:0.20.0
com.google.auto.value:auto-value-annotations:1.7
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud.sql:jdbc-socket-factory-core:1.0.12
com.google.cloud.sql:postgres-socket-factory:1.0.12
com.google.cloud.sql:jdbc-socket-factory-core:1.0.16
com.google.cloud.sql:postgres-socket-factory:1.0.16
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
@@ -88,14 +88,14 @@ com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.dagger:dagger:2.28
com.google.errorprone:error_prone_annotations:2.3.4
com.google.flogger:flogger-system-backend:0.3.1
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava-testlib:28.2-jre
com.google.guava:guava:28.2-jre
com.google.guava:guava-testlib:29.0-jre
com.google.guava:guava:29.0-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.34.1
@@ -188,10 +188,10 @@ javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.13
net.bytebuddy:byte-buddy-agent:1.9.7
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
net.bytebuddy:byte-buddy-agent:1.10.5
net.bytebuddy:byte-buddy:1.10.10
net.java.dev.jna:jna-platform:5.5.0
net.java.dev.jna:jna:5.5.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
@@ -205,7 +205,7 @@ org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
@@ -223,11 +223,11 @@ org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcpkix-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.10.0
org.checkerframework:checker-qual:2.11.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.17
org.dom4j:dom4j:2.1.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.flywaydb:flyway-core:5.2.4
@@ -237,37 +237,39 @@ org.hamcrest:hamcrest-all:1.3
org.hamcrest:hamcrest-core:1.3
org.hamcrest:hamcrest-library:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.hibernate:hibernate-core:5.4.17.Final
org.hibernate:hibernate-hikaricp:5.4.17.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.jboss:jandex:2.1.3.Final
org.jetbrains:annotations:19.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.junit.jupiter:junit-jupiter-api:5.6.1
org.junit.jupiter:junit-jupiter-engine:5.6.1
org.junit.jupiter:junit-jupiter-migrationsupport:5.6.1
org.junit.platform:junit-platform-commons:1.6.1
org.junit.platform:junit-platform-engine:1.6.1
org.junit.platform:junit-platform-launcher:1.6.1
org.junit.platform:junit-platform-runner:1.6.1
org.junit.platform:junit-platform-suite-api:1.6.1
org.junit.vintage:junit-vintage-engine:5.6.1
org.junit:junit-bom:5.6.1
org.junit-pioneer:junit-pioneer:0.7.0
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.jupiter:junit-jupiter-migrationsupport:5.6.2
org.junit.jupiter:junit-jupiter-params:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit.platform:junit-platform-launcher:1.6.2
org.junit.platform:junit-platform-runner:1.6.2
org.junit.platform:junit-platform-suite-api:1.6.2
org.junit:junit-bom:5.6.2
org.jvnet.staxex:stax-ex:1.8
org.mockito:mockito-core:2.25.0
org.mockito:mockito-core:3.3.3
org.mockito:mockito-junit-jupiter:3.3.3
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:2.6
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.postgresql:postgresql:42.2.6
org.ow2.asm:asm-analysis:8.0.1
org.ow2.asm:asm-commons:7.1
org.ow2.asm:asm-tree:8.0.1
org.ow2.asm:asm-util:8.0.1
org.ow2.asm:asm:8.0.1
org.postgresql:postgresql:42.2.14
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
@@ -282,18 +284,18 @@ org.seleniumhq.selenium:selenium-opera-driver:3.141.59
org.seleniumhq.selenium:selenium-remote-driver:3.141.59
org.seleniumhq.selenium:selenium-safari-driver:3.141.59
org.seleniumhq.selenium:selenium-support:3.141.59
org.slf4j:slf4j-api:1.7.28
org.slf4j:slf4j-api:1.7.30
org.slf4j:slf4j-jdk14:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:selenium:1.12.1
org.testcontainers:testcontainers:1.12.1
org.testcontainers:database-commons:1.14.3
org.testcontainers:jdbc:1.14.3
org.testcontainers:junit-jupiter:1.14.3
org.testcontainers:postgresql:1.14.3
org.testcontainers:selenium:1.14.3
org.testcontainers:testcontainers:1.14.3
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4
org.yaml:snakeyaml:1.17
pl.pragmatists:JUnitParams:1.1.1
xerces:xmlParserAPIs:2.6.2
xpp3:xpp3:1.1.4c

View File

@@ -39,7 +39,7 @@ module.exports = function(config) {
included: false
},
{
pattern: 'core/build/generated/source/custom/main/**/*.soy.js',
pattern: 'core/build/generated/sources/custom/java/main/**/*.soy.js',
included: false
},
{
@@ -65,7 +65,7 @@ module.exports = function(config) {
'node_modules/google-closure-library/closure/goog/base.js': ['closure'],
'node_modules/google-closure-library/closure/**/*.js': ['closure'],
'core/src/*/javascript/**/*.js': ['closure'],
'core/build/generated/source/custom/main/**/*.soy.js': ['closure'],
'core/build/generated/sources/custom/java/main/**/*.soy.js': ['closure'],
},
proxies: {
"/assets/": "/base/core/build/resources/main/google/registry/ui/assets/"

View File

@@ -0,0 +1,81 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import com.google.apphosting.api.ApiProxy;
import com.google.apphosting.api.ApiProxy.Environment;
import com.google.common.collect.ImmutableMap;
import java.io.Closeable;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
/**
* Sets up a fake {@link Environment} so that the following operations can be performed without the
* Datastore service:
*
* <ul>
* <li>Create Objectify {@code Keys}.
* <li>Instantiate Objectify objects.
* <li>Convert Datastore {@code Entities} to their corresponding Objectify objects.
* </ul>
*
* <p>User has the option to specify their desired {@code appId} string, which forms part of an
* Objectify {@code Key} and is included in the equality check. This feature makes it easy to
* compare a migrated object in SQL with the original in Objectify.
*
* <p>Note that conversion from Objectify objects to Datastore {@code Entities} still requires the
* Datastore service.
*/
public class AppEngineEnvironment implements Closeable {
private boolean isPlaceHolderNeeded;
public AppEngineEnvironment() {
this("PlaceholderAppId");
}
public AppEngineEnvironment(String appId) {
isPlaceHolderNeeded = ApiProxy.getCurrentEnvironment() == null;
// isPlaceHolderNeeded may be true when we are invoked in a test with AppEngineRule.
if (isPlaceHolderNeeded) {
ApiProxy.setEnvironmentForCurrentThread(createAppEngineEnvironment(appId));
}
}
@Override
public void close() {
if (isPlaceHolderNeeded) {
ApiProxy.setEnvironmentForCurrentThread(null);
}
}
/** Returns a placeholder {@link Environment} that can return hardcoded AppId and Attributes. */
private static Environment createAppEngineEnvironment(String appId) {
return (Environment)
Proxy.newProxyInstance(
Environment.class.getClassLoader(),
new Class[] {Environment.class},
(Object proxy, Method method, Object[] args) -> {
switch (method.getName()) {
case "getAppId":
return appId;
case "getAttributes":
return ImmutableMap.<String, Object>of();
default:
throw new UnsupportedOperationException(method.getName());
}
});
}
}

View File

@@ -0,0 +1,95 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.backup.BackupUtils.createDeserializingIterator;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Streams;
import google.registry.model.ImmutableObject;
import google.registry.model.ofy.CommitLogCheckpoint;
import google.registry.model.ofy.CommitLogManifest;
import google.registry.model.ofy.CommitLogMutation;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.channels.Channels;
import java.nio.channels.ReadableByteChannel;
import java.util.Iterator;
import java.util.stream.Stream;
/**
* Helpers for reading CommitLog records from a file.
*
* <p>This class is adapted from {@link RestoreCommitLogsAction}, and will be used in the initial
* population of the Cloud SQL database.
*/
public final class CommitLogImports {
private CommitLogImports() {}
/**
* Returns entities in an {@code inputStream} (from a single CommitLog file) as an {@link
* ImmutableList} of {@link VersionedEntity} records. Upon completion the {@code inputStream} is
* closed.
*
* <p>The returned list may be empty, since CommitLogs are written at fixed intervals regardless
* if actual changes exist.
*
* <p>A CommitLog file starts with a {@link CommitLogCheckpoint}, followed by (repeated)
* subsequences of [{@link CommitLogManifest}, [{@link CommitLogMutation}] ...]. Each subsequence
* represents the changes in one transaction. The {@code CommitLogManifest} contains deleted
* entity keys, whereas each {@code CommitLogMutation} contains one whole entity.
*/
public static ImmutableList<VersionedEntity> loadEntities(InputStream inputStream) {
try (AppEngineEnvironment appEngineEnvironment = new AppEngineEnvironment();
InputStream input = new BufferedInputStream(inputStream)) {
Iterator<ImmutableObject> commitLogs = createDeserializingIterator(input);
checkState(commitLogs.hasNext());
checkState(commitLogs.next() instanceof CommitLogCheckpoint);
return Streams.stream(commitLogs)
.map(
e ->
e instanceof CommitLogManifest
? VersionedEntity.fromManifest((CommitLogManifest) e)
: Stream.of(VersionedEntity.fromMutation((CommitLogMutation) e)))
.flatMap(s -> s)
.collect(ImmutableList.toImmutableList());
} catch (IOException e) {
throw new RuntimeException(e);
}
}
/** Covenience method that adapts {@link #loadEntities(InputStream)} to a {@link File}. */
public static ImmutableList<VersionedEntity> loadEntities(File commitLogFile) {
try {
return loadEntities(new FileInputStream(commitLogFile));
} catch (IOException e) {
throw new RuntimeException(e);
}
}
/**
* Covenience method that adapts {@link #loadEntities(InputStream)} to a {@link
* ReadableByteChannel}.
*/
public static ImmutableList<VersionedEntity> loadEntities(ReadableByteChannel channel) {
return loadEntities(Channels.newInputStream(channel));
}
}

View File

@@ -0,0 +1,198 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.EntityTranslator;
import com.google.appengine.api.datastore.Key;
import com.google.auto.value.AutoValue;
import com.google.auto.value.extension.memoized.Memoized;
import google.registry.model.ofy.CommitLogManifest;
import google.registry.model.ofy.CommitLogMutation;
import java.io.Serializable;
import java.util.Arrays;
import java.util.Optional;
import java.util.stream.Stream;
import javax.annotation.Nullable;
/**
* A Datastore {@link Entity Entity's} serialized state with timestamp. The intended use case is a
* multi-stage pipeline where an Entity's Java form is not needed in most stages.
*
* <p>For a new or updated Entity, its serialized bytes are stored along with its Datastore {@link
* Key}. For a deleted entity, only its Datastore {@link Key} is stored, and the {@link
* #entityProtoBytes} field is left unset.
*
* <p>Storing raw bytes is motivated by two factors. First, since I/O is frequent and the Java
* objects are rarely needed in our target use case, storing raw bytes is the most efficient
* approach. More importantly, due to our data model and our customization of {@link
* google.registry.model.ofy.ObjectifyService ObjectifyService}, it is challenging to implement a
* serializer for Objectify entities that preserves the value of all properties. Without such
* serializers, Objectify entities cannot be used in a pipeline.
*
* <p>Objectify entities do not implement {@link Serializable}, serialization of such objects is as
* follows:
*
* <ul>
* <li>Convert an Objectify entity to a Datastore {@link Entity}: {@code
* ofy().save().toEntity(..)}
* <li>Entity is serializable, but the more efficient approach is to convert an Entity to a
* ProtocolBuffer ({@link com.google.storage.onestore.v3.OnestoreEntity.EntityProto}) and then
* to raw bytes.
* </ul>
*
* <p>When the first conversion above is applied to an Objectify entity, a property value in the
* output may differ from the input in two situations:
*
* <ul>
* <li>If a property is of an assign-on-persist data type, e.g., {@link
* google.registry.model.UpdateAutoTimestamp}.
* <li>If it is related to CommitLog management, e.g., {@link google.registry.model.EppResource
* EppResource.revisions}.
* </ul>
*
* <p>Working around the side effects caused by our customization is difficult. Any solution would
* likely rely on Objectify's stack of context. However, many Objectify invocations in our code base
* are hardcoded to call the customized version of ObjectifyService, rendering Objectify's stack
* useless.
*
* <p>For now, this inability to use Objectify entities in pipelines is mostly a testing problem: we
* can not perform {@link org.apache.beam.sdk.testing.PAssert BEAM pipeline assertions} on Objectify
* entities. {@code InitSqlTestUtils.assertContainsExactlyElementsIn} is an example of a workaround.
*
* <p>Note that {@link Optional java.util.Optional} is not serializable, therefore cannot be used as
* property type in this class.
*/
@AutoValue
public abstract class VersionedEntity implements Serializable {
private static final long serialVersionUID = 1L;
public abstract long commitTimeMills();
/** The {@link Key} of the {@link Entity}. */
public abstract Key key();
/** Serialized form of the {@link Entity}. This property is {@code null} for a deleted Entity. */
@Nullable
abstract ImmutableBytes entityProtoBytes();
@Memoized
public Optional<Entity> getEntity() {
return Optional.ofNullable(entityProtoBytes())
.map(ImmutableBytes::getBytes)
.map(EntityTranslator::createFromPbBytes);
}
public boolean isDelete() {
return entityProtoBytes() == null;
}
/**
* Converts deleted entity keys in {@code manifest} into a {@link Stream} of {@link
* VersionedEntity VersionedEntities}. See {@link CommitLogImports#loadEntities} for more
* information.
*/
public static Stream<VersionedEntity> fromManifest(CommitLogManifest manifest) {
long commitTimeMillis = manifest.getCommitTime().getMillis();
return manifest.getDeletions().stream()
.map(com.googlecode.objectify.Key::getRaw)
.map(key -> builder().commitTimeMills(commitTimeMillis).key(key).build());
}
/* Converts a {@link CommitLogMutation} to a {@link VersionedEntity}. */
public static VersionedEntity fromMutation(CommitLogMutation mutation) {
return from(
com.googlecode.objectify.Key.create(mutation).getParent().getId(),
mutation.getEntityProtoBytes());
}
public static VersionedEntity from(long commitTimeMillis, byte[] entityProtoBytes) {
return builder()
.entityProtoBytes(entityProtoBytes)
.key(EntityTranslator.createFromPbBytes(entityProtoBytes).getKey())
.commitTimeMills(commitTimeMillis)
.build();
}
static Builder builder() {
return new AutoValue_VersionedEntity.Builder();
}
@AutoValue.Builder
public abstract static class Builder {
public abstract Builder commitTimeMills(long commitTimeMillis);
abstract Builder entityProtoBytes(ImmutableBytes bytes);
public abstract Builder key(Key key);
public abstract VersionedEntity build();
public Builder entityProtoBytes(byte[] bytes) {
return entityProtoBytes(new ImmutableBytes(bytes));
}
}
/**
* Wraps a byte array and prevents it from being modified by its original owner.
*
* <p>While this class seems an overkill, it exists for two reasons:
*
* <ul>
* <li>It is easier to override the {@link #equals} method here (for value-equivalence check)
* than to override the AutoValue-generated {@code equals} method.
* <li>To appease the style checker, which forbids arrays as AutoValue property.
* </ul>
*/
static final class ImmutableBytes implements Serializable {
private static final long serialVersionUID = 1L;
private final byte[] bytes;
ImmutableBytes(byte[] bytes) {
this.bytes = Arrays.copyOf(bytes, bytes.length);
}
/**
* Returns the saved byte array. Invocation is restricted to trusted callers, who must not
* modify the array.
*/
byte[] getBytes() {
return bytes;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof ImmutableBytes)) {
return false;
}
ImmutableBytes that = (ImmutableBytes) o;
// Do not use Objects.equals, which checks reference identity instead of data in array.
return Arrays.equals(bytes, that.bytes);
}
@Override
public int hashCode() {
// Do not use Objects.hashCode, which hashes the reference, not the data in array.
return Arrays.hashCode(bytes);
}
}
}

View File

@@ -150,7 +150,7 @@ public final class AsyncTaskEnqueuer {
/** Enqueues a task to asynchronously refresh DNS for a renamed host. */
public void enqueueAsyncDnsRefresh(HostResource host, DateTime now) {
VKey<HostResource> hostKey = host.createKey();
VKey<HostResource> hostKey = host.createVKey();
logger.atInfo().log("Enqueuing async DNS refresh for renamed host %s.", hostKey);
addTaskToQueueWithRetry(
asyncDnsRefreshPullQueue,
@@ -169,10 +169,12 @@ public final class AsyncTaskEnqueuer {
lock.getRelockDuration().isPresent(),
"Lock with ID %s not configured for relock",
lock.getRevisionId());
String backendHostname = appEngineServiceUtils.getServiceHostname("backend");
addTaskToQueueWithRetry(
asyncActionsPushQueue,
TaskOptions.Builder.withUrl(RelockDomainAction.PATH)
.method(Method.POST)
.header("Host", backendHostname)
.param(
RelockDomainAction.OLD_UNLOCK_REVISION_ID_PARAM,
String.valueOf(lock.getRevisionId()))

View File

@@ -285,11 +285,9 @@ public class DeleteContactsAndHostsAction implements Runnable {
if (resourceKey.getKind().equals(KIND_CONTACT)) {
return domain
.getReferencedContacts()
.contains(VKey.createOfy(ContactResource.class, (Key<ContactResource>) resourceKey));
.contains(VKey.from((Key<ContactResource>) resourceKey));
} else if (resourceKey.getKind().equals(KIND_HOST)) {
return domain
.getNameservers()
.contains(VKey.createOfy(HostResource.class, (Key<HostResource>) resourceKey));
return domain.getNameservers().contains(VKey.from((Key<HostResource>) resourceKey));
} else {
throw new IllegalStateException("EPP resource key of unknown type: " + resourceKey);
}
@@ -347,7 +345,7 @@ public class DeleteContactsAndHostsAction implements Runnable {
String resourceClientId = resource.getPersistedCurrentSponsorClientId();
if (resource instanceof HostResource && ((HostResource) resource).isSubordinate()) {
resourceClientId =
ofy().load().key(((HostResource) resource).getSuperordinateDomain()).now()
tm().load(((HostResource) resource).getSuperordinateDomain())
.cloneProjectedAtTime(now)
.getCurrentSponsorClientId();
}
@@ -426,7 +424,7 @@ public class DeleteContactsAndHostsAction implements Runnable {
if (resource instanceof HostResource) {
return ImmutableList.of(
HostPendingActionNotificationResponse.create(
((HostResource) resource).getFullyQualifiedHostName(), deleteAllowed, trid, now));
((HostResource) resource).getHostName(), deleteAllowed, trid, now));
} else if (resource instanceof ContactResource) {
return ImmutableList.of(
ContactPendingActionNotificationResponse.create(
@@ -465,11 +463,12 @@ public class DeleteContactsAndHostsAction implements Runnable {
} else if (existingResource instanceof HostResource) {
HostResource host = (HostResource) existingResource;
if (host.isSubordinate()) {
dnsQueue.addHostRefreshTask(host.getFullyQualifiedHostName());
ofy().save().entity(
ofy().load().key(host.getSuperordinateDomain()).now().asBuilder()
.removeSubordinateHost(host.getFullyQualifiedHostName())
.build());
dnsQueue.addHostRefreshTask(host.getHostName());
tm().saveNewOrUpdate(
tm().load(host.getSuperordinateDomain())
.asBuilder()
.removeSubordinateHost(host.getHostName())
.build());
}
} else {
throw new IllegalStateException(
@@ -533,7 +532,7 @@ public class DeleteContactsAndHostsAction implements Runnable {
resource.getClass().getSimpleName());
return new AutoValue_DeleteContactsAndHostsAction_DeletionRequest.Builder()
.setKey(resourceKey)
.setLastUpdateTime(resource.getUpdateAutoTimestamp().getTimestamp())
.setLastUpdateTime(resource.getUpdateTimestamp().getTimestamp())
.setRequestingClientId(
checkNotNull(
params.get(PARAM_REQUESTING_CLIENT_ID), "Requesting client id not specified"))

View File

@@ -177,7 +177,7 @@ public class DeleteProberDataAction implements Runnable {
return;
}
String domainName = domain.getFullyQualifiedDomainName();
String domainName = domain.getDomainName();
if (domainName.equals("nic." + domain.getTld())) {
getContext().incrementCounter("skipped, NIC domain");
return;
@@ -246,28 +246,32 @@ public class DeleteProberDataAction implements Runnable {
}
private void softDeleteDomain(final DomainBase domain) {
tm().transactNew(() -> {
DomainBase deletedDomain = domain
.asBuilder()
.setDeletionTime(tm().getTransactionTime())
.setStatusValues(null)
.build();
HistoryEntry historyEntry = new HistoryEntry.Builder()
.setParent(domain)
.setType(DOMAIN_DELETE)
.setModificationTime(tm().getTransactionTime())
.setBySuperuser(true)
.setReason("Deletion of prober data")
.setClientId(registryAdminClientId)
.build();
// Note that we don't bother handling grace periods, billing events, pending transfers,
// poll messages, or auto-renews because these will all be hard-deleted the next time the
// mapreduce runs anyway.
ofy().save().entities(deletedDomain, historyEntry);
updateForeignKeyIndexDeletionTime(deletedDomain);
dnsQueue.addDomainRefreshTask(deletedDomain.getFullyQualifiedDomainName());
}
);
tm().transactNew(
() -> {
DomainBase deletedDomain =
domain
.asBuilder()
.setDeletionTime(tm().getTransactionTime())
.setStatusValues(null)
.build();
HistoryEntry historyEntry =
new HistoryEntry.Builder()
.setParent(domain)
.setType(DOMAIN_DELETE)
.setModificationTime(tm().getTransactionTime())
.setBySuperuser(true)
.setReason("Deletion of prober data")
.setClientId(registryAdminClientId)
.build();
// Note that we don't bother handling grace periods, billing events, pending
// transfers,
// poll messages, or auto-renews because these will all be hard-deleted the next
// time the
// mapreduce runs anyway.
ofy().save().entities(deletedDomain, historyEntry);
updateForeignKeyIndexDeletionTime(deletedDomain);
dnsQueue.addDomainRefreshTask(deletedDomain.getDomainName());
});
}
}
}

View File

@@ -207,20 +207,18 @@ public class RefreshDnsOnHostRenameAction implements Runnable {
Key<HostResource> referencingHostKey = null;
for (DnsRefreshRequest request : refreshRequests) {
if (isActive(domain, request.lastUpdateTime())
&& domain
.getNameservers()
.contains(VKey.createOfy(HostResource.class, request.hostKey()))) {
&& domain.getNameservers().contains(VKey.from(request.hostKey()))) {
referencingHostKey = request.hostKey();
break;
}
}
if (referencingHostKey != null) {
retrier.callWithRetry(
() -> dnsQueue.addDomainRefreshTask(domain.getFullyQualifiedDomainName()),
() -> dnsQueue.addDomainRefreshTask(domain.getDomainName()),
TransientFailureException.class);
logger.atInfo().log(
"Enqueued DNS refresh for domain %s referenced by host %s.",
domain.getFullyQualifiedDomainName(), referencingHostKey);
domain.getDomainName(), referencingHostKey);
getContext().incrementCounter("domains refreshed");
} else {
getContext().incrementCounter("domains not refreshed");
@@ -321,13 +319,13 @@ public class RefreshDnsOnHostRenameAction implements Runnable {
HostResource host =
checkNotNull(ofy().load().key(hostKey).now(), "Host to refresh doesn't exist");
boolean isHostDeleted =
isDeleted(host, latestOf(now, host.getUpdateAutoTimestamp().getTimestamp()));
isDeleted(host, latestOf(now, host.getUpdateTimestamp().getTimestamp()));
if (isHostDeleted) {
logger.atInfo().log("Host %s is already deleted, not refreshing DNS.", hostKey);
}
return new AutoValue_RefreshDnsOnHostRenameAction_DnsRefreshRequest.Builder()
.setHostKey(hostKey)
.setLastUpdateTime(host.getUpdateAutoTimestamp().getTimestamp())
.setLastUpdateTime(host.getUpdateTimestamp().getTimestamp())
.setRequestedTime(
DateTime.parse(
checkNotNull(params.get(PARAM_REQUESTED_TIME), "Requested time not specified")))

View File

@@ -96,7 +96,7 @@ public class RelockDomainAction implements Runnable {
String message =
String.format(
"Domain %s is already manually relocked, skipping automated relock.",
domain.getFullyQualifiedDomainName());
domain.getDomainName());
logger.atInfo().log(message);
// SC_NO_CONTENT (204) skips retry -- see the comment below
response.setStatus(SC_NO_CONTENT);
@@ -144,7 +144,7 @@ public class RelockDomainAction implements Runnable {
private void verifyDomainAndLockState(RegistryLock oldLock, DomainBase domain) {
// Domain shouldn't be deleted or have a pending transfer/delete
String domainName = domain.getFullyQualifiedDomainName();
String domainName = domain.getDomainName();
checkArgument(
!DateTimeUtils.isAtOrAfter(jpaTm().getTransactionTime(), domain.getDeletionTime()),
"Domain %s has been deleted",

View File

@@ -0,0 +1,117 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Strings.isNullOrEmpty;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Streams;
import org.joda.time.DateTime;
/**
* Helpers for determining the fully qualified paths to Nomulus backup files. A backup consists of a
* Datastore export and Nomulus CommitLogs that overlap with the export.
*/
public final class BackupPaths {
private BackupPaths() {}
private static final String WILDCARD_CHAR = "*";
private static final String EXPORT_PATTERN_TEMPLATE = "%s/all_namespaces/kind_%s/output-%s";
public static final String COMMIT_LOG_NAME_PREFIX = "commit_diff_until_";
private static final String COMMIT_LOG_PATTERN_TEMPLATE = "%s/" + COMMIT_LOG_NAME_PREFIX + "*";
/**
* Pattern of the per-project file with Cloud SQL connection information. To get a concrete path,
* user needs to provide the name of the environment, alpha, crash, sandbox, or production. This
* file is meant for applications without access to secrets stored in Datastore.
*
* <p>In production, this is an base-64 encoded encrypted file with one line, which contains
* space-separated values of Cloud SQL instance name, login, and password.
*
* <p>A plain text may be used for tests to a local database. Replace Cloud SQL instance name with
* JDBC URL.
*/
private static final String SQL_CONN_INFO_FILE_PATTERN =
"gs://domain-registry-dev-deploy/cloudsql-credentials/%s/admin_credential.enc";
private static final ImmutableSet<String> ALLOWED_ENV =
ImmutableSet.of("alpha", "crash", "sandbox", "production");
/**
* Returns a regex pattern that matches all Datastore export files of a given {@code kind}.
*
* @param exportDir path to the top directory of a Datastore export
* @param kind the 'kind' of the Datastore entity
*/
public static String getExportFileNamePattern(String exportDir, String kind) {
checkArgument(!isNullOrEmpty(exportDir), "Null or empty exportDir.");
checkArgument(!isNullOrEmpty(kind), "Null or empty kind.");
return String.format(EXPORT_PATTERN_TEMPLATE, exportDir, kind, WILDCARD_CHAR);
}
/**
* Returns an {@link ImmutableList} of regex patterns that match all Datastore export files of the
* given {@code kinds}.
*
* @param exportDir path to the top directory of a Datastore export
* @param kinds all entity 'kinds' to be matched
*/
public static ImmutableList<String> getExportFilePatterns(
String exportDir, Iterable<String> kinds) {
checkNotNull(kinds, "kinds");
return Streams.stream(kinds)
.map(kind -> getExportFileNamePattern(exportDir, kind))
.collect(ImmutableList.toImmutableList());
}
/**
* Returns the fully qualified path of a Datastore export file with the given {@code kind} and
* {@code shard}.
*
* @param exportDir path to the top directory of a Datastore export
* @param kind the 'kind' of the Datastore entity
* @param shard an integer suffix of the file name
*/
public static String getExportFileNameByShard(String exportDir, String kind, int shard) {
checkArgument(!isNullOrEmpty(exportDir), "Null or empty exportDir.");
checkArgument(!isNullOrEmpty(kind), "Null or empty kind.");
checkArgument(shard >= 0, "Negative shard %s not allowed.", shard);
return String.format(EXPORT_PATTERN_TEMPLATE, exportDir, kind, Integer.toString(shard));
}
/** Returns an {@link ImmutableList} of regex patterns that match all CommitLog files. */
public static ImmutableList<String> getCommitLogFilePatterns(String commitLogDir) {
return ImmutableList.of(String.format(COMMIT_LOG_PATTERN_TEMPLATE, commitLogDir));
}
/** Gets the Commit timestamp from a CommitLog file name. */
public static DateTime getCommitLogTimestamp(String fileName) {
checkArgument(!isNullOrEmpty(fileName), "Null or empty fileName.");
int start = fileName.lastIndexOf(COMMIT_LOG_NAME_PREFIX);
checkArgument(start >= 0, "Illegal file name %s.", fileName);
return DateTime.parse(fileName.substring(start + COMMIT_LOG_NAME_PREFIX.length()));
}
public static ImmutableList<String> getCloudSQLCredentialFilePatterns(String environmentName) {
checkArgument(
ALLOWED_ENV.contains(environmentName), "Invalid environment name %s", environmentName);
return ImmutableList.of(String.format(SQL_CONN_INFO_FILE_PATTERN, environmentName));
}
}

View File

@@ -0,0 +1,180 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.base.Strings.isNullOrEmpty;
import com.google.common.base.Splitter;
import dagger.Component;
import dagger.Lazy;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.keyring.kms.KmsModule;
import google.registry.persistence.PersistenceModule;
import google.registry.persistence.PersistenceModule.JdbcJpaTm;
import google.registry.persistence.PersistenceModule.SocketFactoryJpaTm;
import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.util.UtilsModule;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.nio.channels.Channels;
import java.nio.charset.StandardCharsets;
import java.util.List;
import javax.annotation.Nullable;
import javax.inject.Singleton;
import org.apache.beam.sdk.io.FileSystems;
import org.apache.beam.sdk.io.fs.ResourceId;
/**
* Provides bindings for {@link JpaTransactionManager} to Cloud SQL.
*
* <p>This module is intended for use in BEAM pipelines, and uses a BEAM utility to access GCS like
* a regular file system.
*/
@Module
public class BeamJpaModule {
private static final String GCS_SCHEME = "gs://";
@Nullable private final String sqlAccessInfoFile;
@Nullable private final String cloudKmsProjectId;
/**
* Constructs a new instance of {@link BeamJpaModule}.
*
* <p>Note: it is an unfortunately necessary antipattern to check for the validity of
* sqlAccessInfoFile in {@link #provideCloudSqlAccessInfo} rather than in the constructor.
* Unfortunately, this is a restriction imposed upon us by Dagger. Specifically, because we use
* this in at least one 1 {@link google.registry.tools.RegistryTool} command(s), it must be
* instantiated in {@code google.registry.tools.RegistryToolComponent} for all possible commands;
* Dagger doesn't permit it to ever be null. For the vast majority of commands, it will never be
* used (so a null credential file path is fine in those cases).
*
* @param sqlAccessInfoFile the path to a Cloud SQL credential file. This must refer to either a
* real encrypted file on GCS as returned by {@link
* BackupPaths#getCloudSQLCredentialFilePatterns} or an unencrypted file on local filesystem
* with credentials to a test database.
*/
public BeamJpaModule(@Nullable String sqlAccessInfoFile, @Nullable String cloudKmsProjectId) {
this.sqlAccessInfoFile = sqlAccessInfoFile;
this.cloudKmsProjectId = cloudKmsProjectId;
}
/** Returns true if the credential file is on GCS (and therefore expected to be encrypted). */
private boolean isCloudSqlCredential() {
return sqlAccessInfoFile.startsWith(GCS_SCHEME);
}
@Provides
@Singleton
SqlAccessInfo provideCloudSqlAccessInfo(Lazy<CloudSqlCredentialDecryptor> lazyDecryptor) {
checkArgument(!isNullOrEmpty(sqlAccessInfoFile), "Null or empty credentialFilePath");
String line = readOnlyLineFromCredentialFile();
if (isCloudSqlCredential()) {
line = lazyDecryptor.get().decrypt(line);
}
// See ./BackupPaths.java for explanation of the line format.
List<String> parts = Splitter.on(' ').splitToList(line.trim());
checkState(parts.size() == 3, "Expecting three phrases in %s", line);
if (isCloudSqlCredential()) {
return SqlAccessInfo.createCloudSqlAccessInfo(parts.get(0), parts.get(1), parts.get(2));
} else {
return SqlAccessInfo.createLocalSqlAccessInfo(parts.get(0), parts.get(1), parts.get(2));
}
}
String readOnlyLineFromCredentialFile() {
try {
ResourceId resourceId = FileSystems.matchSingleFileSpec(sqlAccessInfoFile).resourceId();
try (BufferedReader reader =
new BufferedReader(
new InputStreamReader(
Channels.newInputStream(FileSystems.open(resourceId)), StandardCharsets.UTF_8))) {
return reader.readLine();
}
} catch (IOException e) {
throw new RuntimeException(e);
}
}
@Provides
@Config("beamCloudSqlJdbcUrl")
String provideJdbcUrl(SqlAccessInfo sqlAccessInfo) {
return sqlAccessInfo.jdbcUrl();
}
@Provides
@Config("beamCloudSqlInstanceConnectionName")
String provideSqlInstanceName(SqlAccessInfo sqlAccessInfo) {
return sqlAccessInfo
.cloudSqlInstanceName()
.orElseThrow(() -> new IllegalStateException("Cloud SQL not provisioned."));
}
@Provides
@Config("beamCloudSqlUsername")
String provideSqlUsername(SqlAccessInfo sqlAccessInfo) {
return sqlAccessInfo.user();
}
@Provides
@Config("beamCloudSqlPassword")
String provideSqlPassword(SqlAccessInfo sqlAccessInfo) {
return sqlAccessInfo.password();
}
@Provides
@Config("beamCloudKmsProjectId")
String kmsProjectId() {
return cloudKmsProjectId;
}
@Provides
@Config("beamCloudKmsKeyRing")
static String keyRingName() {
return "nomulus-tool-keyring";
}
@Provides
@Config("beamHibernateHikariMaximumPoolSize")
static int getBeamHibernateHikariMaximumPoolSize() {
// TODO(weiminyu): make this configurable. Should be equal to number of cores.
return 4;
}
@Singleton
@Component(
modules = {
ConfigModule.class,
CredentialModule.class,
BeamJpaModule.class,
KmsModule.class,
PersistenceModule.class,
UtilsModule.class
})
public interface JpaTransactionManagerComponent {
@SocketFactoryJpaTm
JpaTransactionManager cloudSqlJpaTransactionManager();
@JdbcJpaTm
JpaTransactionManager localDbJpaTransactionManager();
}
}

View File

@@ -0,0 +1,50 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import static com.google.common.base.Preconditions.checkArgument;
import com.google.api.services.cloudkms.v1.model.DecryptRequest;
import com.google.common.base.Strings;
import google.registry.config.RegistryConfig.Config;
import google.registry.keyring.kms.KmsConnection;
import java.nio.charset.StandardCharsets;
import java.util.Base64;
import javax.inject.Inject;
/**
* Decrypts data using Cloud KMS, with the same crypto key with which Cloud SQL credential files on
* GCS was encrypted. See {@link BackupPaths#getCloudSQLCredentialFilePatterns} for more
* information.
*/
public class CloudSqlCredentialDecryptor {
private static final String CRYPTO_KEY_NAME = "nomulus-tool-key";
private final KmsConnection kmsConnection;
@Inject
CloudSqlCredentialDecryptor(@Config("beamKmsConnection") KmsConnection kmsConnection) {
this.kmsConnection = kmsConnection;
}
public String decrypt(String data) {
checkArgument(!Strings.isNullOrEmpty(data), "Null or empty data.");
byte[] ciphertext = Base64.getDecoder().decode(data);
// Re-encode for Cloud KMS JSON REST API, invoked through kmsConnection.
String urlSafeCipherText = new DecryptRequest().encodeCiphertext(ciphertext).getCiphertext();
return new String(
kmsConnection.decrypt(CRYPTO_KEY_NAME, urlSafeCipherText), StandardCharsets.UTF_8);
}
}

View File

@@ -0,0 +1,75 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import com.google.appengine.api.datastore.Entity;
import java.util.Objects;
/** Helper for manipulating {@code DomainBase} when migrating from Datastore to SQL database */
final class DomainBaseUtil {
private DomainBaseUtil() {}
/**
* Removes {@link google.registry.model.billing.BillingEvent.Recurring}, {@link
* google.registry.model.poll.PollMessage PollMessages} and {@link
* google.registry.model.host.HostResource name servers} from a Datastore {@link Entity} that
* represents an Ofy {@link google.registry.model.domain.DomainBase}. This breaks the cycle of
* foreign key constraints between these entity kinds, allowing {@code DomainBases} to be inserted
* into the SQL database. See {@link InitSqlPipeline} for a use case, where the full {@code
* DomainBases} are written again during the last stage of the pipeline.
*
* <p>The returned object may be in bad state. Specifically, {@link
* google.registry.model.eppcommon.StatusValue#INACTIVE} is not added after name servers are
* removed. This only impacts tests.
*
* <p>This operation is performed on an Datastore {@link Entity} instead of Ofy Java object
* because Objectify requires access to a Datastore service when converting an Ofy object to a
* Datastore {@code Entity}. If we insist on working with Objectify objects, we face a few
* unsatisfactory options:
*
* <ul>
* <li>Connect to our production Datastore, which incurs unnecessary security and code health
* risk.
* <li>Connect to a separate real Datastore instance, which is a waster and overkill.
* <li>Use an in-memory test Datastore, which is a project health risk in that the test
* Datastore would be added to Nomulus' production binary unless we create a separate
* project for this pipeline.
* </ul>
*
* <p>Given our use case, operating on Datastore entities is the best option.
*
* @throws IllegalArgumentException if input does not represent a DomainBase
*/
static Entity removeBillingAndPollAndHosts(Entity domainBase) {
checkNotNull(domainBase, "domainBase");
checkArgument(
Objects.equals(domainBase.getKind(), "DomainBase"),
"Expecting DomainBase, got %s",
domainBase.getKind());
Entity clone = domainBase.clone();
clone.removeProperty("autorenewBillingEvent");
clone.removeProperty("autorenewPollMessage");
clone.removeProperty("deletePollMessage");
clone.removeProperty("nsHosts");
domainBase.getProperties().keySet().stream()
.filter(s -> s.startsWith("transferData."))
.forEach(s -> clone.removeProperty(s));
return clone;
}
}

View File

@@ -0,0 +1,244 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import static com.google.common.base.Preconditions.checkArgument;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.backup.AppEngineEnvironment;
import google.registry.backup.VersionedEntity;
import google.registry.beam.initsql.BeamJpaModule.JpaTransactionManagerComponent;
import google.registry.beam.initsql.Transforms.RemoveDomainBaseForeignKeys;
import google.registry.model.billing.BillingEvent;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.host.HostResource;
import google.registry.model.poll.PollMessage;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarContact;
import google.registry.model.registry.Registry;
import google.registry.model.reporting.HistoryEntry;
import google.registry.persistence.transaction.JpaTransactionManager;
import java.io.Serializable;
import java.util.Collection;
import java.util.Optional;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.PipelineResult;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.transforms.SerializableFunction;
import org.apache.beam.sdk.transforms.Wait;
import org.apache.beam.sdk.values.PCollection;
import org.apache.beam.sdk.values.PCollectionTuple;
import org.apache.beam.sdk.values.TupleTag;
import org.joda.time.DateTime;
/**
* A BEAM pipeline that populates a SQL database with data from a Datastore backup.
*
* <p>This pipeline migrates EPP resources and related entities that cross-reference each other. To
* avoid violating foreign key constraints, writes to SQL are ordered by entity kinds. In addition,
* the {@link DomainBase} kind is written twice (see details below). The write order is presented
* below. Although some kinds can be written concurrently, e.g. {@code ContactResource} and {@code
* RegistrarContact}, we do not expect any performance benefit since the limiting resource is the
* number of JDBC connections. Google internal users may refer to <a
* href="http://go/registry-r3-init-sql">the design doc</a> for more information.
*
* <ol>
* <li>{@link Registry}: Assumes that {@code PremiumList} and {@code ReservedList} have been set
* up in the SQL database.
* <li>{@link Registrar}: Logically depends on {@code Registry}, Foreign key not modeled yet.
* <li>{@link ContactResource}: references {@code Registrar}
* <li>{@link RegistrarContact}: references {@code Registrar}.
* <li>Cleansed {@link DomainBase}: with references to {@code BillingEvent}, {@code Recurring},
* {@code Cancellation} and {@code HostResource} removed, still references {@code Registrar}
* and {@code ContactResource}. The removal breaks circular Foreign Key references.
* <li>{@link HostResource}: references {@code DomainBase}.
* <li>{@link HistoryEntry}: maps to one of three SQL entity types and may reference {@code
* Registrar}, {@code ContactResource}, {@code HostResource}, and {@code DomainBase}.
* <li>{@link AllocationToken}: references {@code HistoryEntry}.
* <li>{@link BillingEvent.Recurring}: references {@code Registrar}, {@code DomainBase} and {@code
* HistoryEntry}.
* <li>{@link BillingEvent.OneTime}: references {@code Registrar}, {@code DomainBase}, {@code
* BillingEvent.Recurring}, {@code HistoryEntry} and {@code AllocationToken}.
* <li>{@link BillingEvent.Modification}: SQL model TBD. Will reference {@code Registrar}, {@code
* DomainBase} and {@code BillingEvent.OneTime}.
* <li>{@link BillingEvent.Cancellation}: references {@code Registrar}, {@code DomainBase}, {@code
* BillingEvent.Recurring}, {@code BillingEvent.OneTime}, and {@code HistoryEntry}.
* <li>{@link PollMessage}: references {@code Registrar}, {@code DomainBase}, {@code
* ContactResource}, {@code HostResource}, and {@code HistoryEntry}.
* <li>{@link DomainBase}, original copy from Datastore.
* </ol>
*/
public class InitSqlPipeline implements Serializable {
/**
* Datastore kinds to be written to the SQL database before the cleansed version of {@link
* DomainBase}.
*/
// TODO(weiminyu): include Registry.class when it is modeled in JPA.
private static final ImmutableList<Class<?>> PHASE_ONE_ORDERED =
ImmutableList.of(Registrar.class, ContactResource.class);
/**
* Datastore kinds to be written to the SQL database after the cleansed version of {@link
* DomainBase}.
*
* <p>The following entities are missing from the list:
*
* <ul>
* <li>Those not modeled in JPA yet, e.g., {@code BillingEvent.Modification}.
* <li>Those waiting for sanitation, e.g., {@code HistoryEntry}, which would have duplicate keys
* after converting to SQL model.
* <li>Those that have foreign key constraints on the above.
* </ul>
*/
// TODO(weiminyu): add more entities when available.
private static final ImmutableList<Class<?>> PHASE_TWO_ORDERED =
ImmutableList.of(HostResource.class);
private final InitSqlPipelineOptions options;
private final Pipeline pipeline;
private final SerializableFunction<JpaTransactionManagerComponent, JpaTransactionManager>
jpaGetter;
InitSqlPipeline(InitSqlPipelineOptions options) {
this.options = options;
pipeline = Pipeline.create(options);
jpaGetter = JpaTransactionManagerComponent::cloudSqlJpaTransactionManager;
}
@VisibleForTesting
InitSqlPipeline(InitSqlPipelineOptions options, Pipeline pipeline) {
this.options = options;
this.pipeline = pipeline;
jpaGetter = JpaTransactionManagerComponent::localDbJpaTransactionManager;
}
public PipelineResult run() {
setupPipeline();
return pipeline.run();
}
@VisibleForTesting
void setupPipeline() {
PCollectionTuple datastoreSnapshot =
pipeline.apply(
"Load Datastore snapshot",
Transforms.loadDatastoreSnapshot(
options.getDatastoreExportDir(),
options.getCommitLogDir(),
DateTime.parse(options.getCommitLogStartTimestamp()),
DateTime.parse(options.getCommitLogEndTimestamp()),
ImmutableSet.<String>builder()
.add("DomainBase")
.addAll(toKindStrings(PHASE_ONE_ORDERED))
.addAll(toKindStrings(PHASE_TWO_ORDERED))
.build()));
// Set up the pipeline to write entity kinds from PHASE_ONE_ORDERED to SQL. Return a object
// that signals the completion of the phase.
PCollection<Void> blocker =
scheduleOnePhaseWrites(datastoreSnapshot, PHASE_ONE_ORDERED, Optional.empty(), null);
blocker =
writeToSql(
"DomainBase without circular foreign keys",
removeDomainBaseForeignKeys(datastoreSnapshot)
.apply("Wait on phase one", Wait.on(blocker)));
// Set up the pipeline to write entity kinds from PHASE_TWO_ORDERED to SQL. This phase won't
// start until all cleansed DomainBases have been written (started by line above).
scheduleOnePhaseWrites(
datastoreSnapshot, PHASE_TWO_ORDERED, Optional.of(blocker), "DomainBaseNoFkeys");
}
private PCollection<VersionedEntity> removeDomainBaseForeignKeys(
PCollectionTuple datastoreSnapshot) {
PCollection<VersionedEntity> domainBases =
datastoreSnapshot.get(Transforms.createTagForKind("DomainBase"));
return domainBases.apply(
"Remove circular foreign keys from DomainBase",
ParDo.of(new RemoveDomainBaseForeignKeys()));
}
/**
* Sets up the pipeline to write entities in {@code entityClasses} to SQL. Entities are written
* one kind at a time based on each kind's position in {@code entityClasses}. Concurrency exists
* within each kind.
*
* @param datastoreSnapshot the Datastore snapshot of all data to be migrated to SQL
* @param entityClasses the entity types in write order
* @param blockingPCollection the pipeline stage that blocks this phase
* @param blockingTag description of the stage (if exists) that blocks this phase. Needed for
* generating unique transform ids
* @return the output {@code PCollection} from the writing of the last entity kind. Other parts of
* the pipeline can {@link Wait} on this object
*/
private PCollection<Void> scheduleOnePhaseWrites(
PCollectionTuple datastoreSnapshot,
Collection<Class<?>> entityClasses,
Optional<PCollection<Void>> blockingPCollection,
String blockingTag) {
checkArgument(!entityClasses.isEmpty(), "Each phase must have at least one kind.");
ImmutableList<TupleTag<VersionedEntity>> tags =
toKindStrings(entityClasses).stream()
.map(Transforms::createTagForKind)
.collect(ImmutableList.toImmutableList());
PCollection<Void> prev = blockingPCollection.orElse(null);
String prevTag = blockingTag;
for (TupleTag<VersionedEntity> tag : tags) {
PCollection<VersionedEntity> curr = datastoreSnapshot.get(tag);
if (prev != null) {
curr = curr.apply("Wait on " + prevTag, Wait.on(prev));
}
prev = writeToSql(tag.getId(), curr);
prevTag = tag.getId();
}
return prev;
}
private PCollection<Void> writeToSql(String transformId, PCollection<VersionedEntity> data) {
String credentialFileUrl =
options.getSqlCredentialUrlOverride() != null
? options.getSqlCredentialUrlOverride()
: BackupPaths.getCloudSQLCredentialFilePatterns(options.getEnvironment()).get(0);
return data.apply(
"Write to sql: " + transformId,
Transforms.writeToSql(
transformId,
options.getMaxConcurrentSqlWriters(),
options.getSqlWriteBatchSize(),
new JpaSupplierFactory(credentialFileUrl, options.getCloudKmsProjectId(), jpaGetter)));
}
private static ImmutableList<String> toKindStrings(Collection<Class<?>> entityClasses) {
try (AppEngineEnvironment env = new AppEngineEnvironment()) {
return entityClasses.stream().map(Key::getKind).collect(ImmutableList.toImmutableList());
}
}
public static void main(String[] args) {
InitSqlPipelineOptions options =
PipelineOptionsFactory.fromArgs(args).withValidation().as(InitSqlPipelineOptions.class);
new InitSqlPipeline(options).run();
}
}

View File

@@ -0,0 +1,91 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import javax.annotation.Nullable;
import org.apache.beam.sdk.extensions.gcp.options.GcpOptions;
import org.apache.beam.sdk.options.Default;
import org.apache.beam.sdk.options.Description;
import org.apache.beam.sdk.options.Validation;
/** Pipeline options for {@link InitSqlPipeline} */
public interface InitSqlPipelineOptions extends GcpOptions {
@Description(
"Overrides the URL to the SQL credential file. " + "Required if environment is not provided.")
@Nullable
String getSqlCredentialUrlOverride();
void setSqlCredentialUrlOverride(String credentialUrlOverride);
@Description("The root directory of the export to load.")
String getDatastoreExportDir();
void setDatastoreExportDir(String datastoreExportDir);
@Description("The directory that contains all CommitLog files.")
String getCommitLogDir();
void setCommitLogDir(String commitLogDir);
@Description("The earliest CommitLogs to load, in ISO8601 format.")
@Validation.Required
String getCommitLogStartTimestamp();
void setCommitLogStartTimestamp(String commitLogStartTimestamp);
@Description("The latest CommitLogs to load, in ISO8601 format.")
@Validation.Required
String getCommitLogEndTimestamp();
void setCommitLogEndTimestamp(String commitLogEndTimestamp);
@Description(
"The deployed environment, alpha, crash, sandbox, or production. "
+ "Not required only if sqlCredentialUrlOverride is provided.")
@Nullable
String getEnvironment();
void setEnvironment(String environment);
@Description(
"The GCP project that contains the keyring used for decrypting the " + "SQL credential file.")
@Nullable
String getCloudKmsProjectId();
void setCloudKmsProjectId(String cloudKmsProjectId);
@Description(
"The maximum JDBC connection pool size on a VM. "
+ "This value should be equal to or greater than the number of cores on the VM.")
@Default.Integer(4)
int getJdbcMaxPoolSize();
void setJdbcMaxPoolSize(int jdbcMaxPoolSize);
@Description(
"A hint to the pipeline runner of the maximum number of concurrent SQL writers to create. "
+ "Note that multiple writers may run on the same VM and share the connection pool.")
@Default.Integer(4)
int getMaxConcurrentSqlWriters();
void setMaxConcurrentSqlWriters(int maxConcurrentSqlWriters);
@Description("The number of entities to be written to the SQL database in one transaction.")
@Default.Integer(20)
int getSqlWriteBatchSize();
void setSqlWriteBatchSize(int sqlWriteBatchSize);
}

View File

@@ -0,0 +1,48 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import google.registry.beam.initsql.BeamJpaModule.JpaTransactionManagerComponent;
import google.registry.beam.initsql.Transforms.SerializableSupplier;
import google.registry.persistence.transaction.JpaTransactionManager;
import javax.annotation.Nullable;
import org.apache.beam.sdk.transforms.SerializableFunction;
public class JpaSupplierFactory implements SerializableSupplier<JpaTransactionManager> {
private static final long serialVersionUID = 1L;
private final String credentialFileUrl;
@Nullable private final String cloudKmsProjectId;
private final SerializableFunction<JpaTransactionManagerComponent, JpaTransactionManager>
jpaGetter;
public JpaSupplierFactory(
String credentialFileUrl,
@Nullable String cloudKmsProjectId,
SerializableFunction<JpaTransactionManagerComponent, JpaTransactionManager> jpaGetter) {
this.credentialFileUrl = credentialFileUrl;
this.cloudKmsProjectId = cloudKmsProjectId;
this.jpaGetter = jpaGetter;
}
@Override
public JpaTransactionManager get() {
return jpaGetter.apply(
DaggerBeamJpaModule_JpaTransactionManagerComponent.builder()
.beamJpaModule(new BeamJpaModule(credentialFileUrl, cloudKmsProjectId))
.build());
}
}

View File

@@ -0,0 +1,17 @@
## Summary
This package contains a BEAM pipeline that populates a Cloud SQL database from a
Datastore backup. The pipeline uses an unsynchronized Datastore export and
overlapping CommitLogs generated by the Nomulus server to recreate a consistent
Datastore snapshot, and writes the data to a Cloud SQL instance.
## Pipeline Visualization
The golden flow graph of the InitSqlPipeline is saved both as a text-base
[DOT file](../../../../../../test/resources/google/registry/beam/initsql/pipeline_golden.dot)
and a
[.png file](../../../../../../test/resources/google/registry/beam/initsql/pipeline_golden.png).
A test compares the flow graph of the current pipeline with the golden graph,
and will fail if changes are detected. When this happens, run the Gradle task
':core:updateInitSqlPipelineGraph' to update the golden files and review the
changes.

View File

@@ -0,0 +1,45 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import com.google.auto.value.AutoValue;
import java.util.Optional;
/**
* Information needed to connect to a database, including JDBC URL, user name, password, and in the
* case of Cloud SQL, the database instance's name.
*/
@AutoValue
abstract class SqlAccessInfo {
abstract String jdbcUrl();
abstract String user();
abstract String password();
abstract Optional<String> cloudSqlInstanceName();
public static SqlAccessInfo createCloudSqlAccessInfo(
String sqlInstanceName, String username, String password) {
return new AutoValue_SqlAccessInfo(
"jdbc:postgresql://google/postgres", username, password, Optional.of(sqlInstanceName));
}
public static SqlAccessInfo createLocalSqlAccessInfo(
String jdbcUrl, String username, String password) {
return new AutoValue_SqlAccessInfo(jdbcUrl, username, password, Optional.empty());
}
}

View File

@@ -0,0 +1,491 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.base.Throwables.throwIfUnchecked;
import static google.registry.beam.initsql.BackupPaths.getCommitLogTimestamp;
import static google.registry.beam.initsql.BackupPaths.getExportFilePatterns;
import static google.registry.persistence.JpaRetries.isFailedTxnRetriable;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.setJpaTm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import static java.util.Comparator.comparing;
import static org.apache.beam.sdk.values.TypeDescriptors.integers;
import static org.apache.beam.sdk.values.TypeDescriptors.kvs;
import static org.apache.beam.sdk.values.TypeDescriptors.strings;
import avro.shaded.com.google.common.collect.Iterators;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.EntityTranslator;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.Streams;
import google.registry.backup.AppEngineEnvironment;
import google.registry.backup.CommitLogImports;
import google.registry.backup.VersionedEntity;
import google.registry.model.domain.DomainBase;
import google.registry.model.ofy.ObjectifyService;
import google.registry.model.ofy.Ofy;
import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.tools.LevelDbLogReader;
import google.registry.util.SystemSleeper;
import java.io.Serializable;
import java.util.Collection;
import java.util.Iterator;
import java.util.Optional;
import java.util.Set;
import java.util.concurrent.ThreadLocalRandom;
import java.util.function.Supplier;
import org.apache.beam.sdk.coders.StringUtf8Coder;
import org.apache.beam.sdk.io.Compression;
import org.apache.beam.sdk.io.FileIO;
import org.apache.beam.sdk.io.FileIO.ReadableFile;
import org.apache.beam.sdk.io.fs.EmptyMatchTreatment;
import org.apache.beam.sdk.io.fs.MatchResult.Metadata;
import org.apache.beam.sdk.transforms.Create;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.Flatten;
import org.apache.beam.sdk.transforms.GroupByKey;
import org.apache.beam.sdk.transforms.GroupIntoBatches;
import org.apache.beam.sdk.transforms.MapElements;
import org.apache.beam.sdk.transforms.PTransform;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.transforms.ProcessFunction;
import org.apache.beam.sdk.values.KV;
import org.apache.beam.sdk.values.PBegin;
import org.apache.beam.sdk.values.PCollection;
import org.apache.beam.sdk.values.PCollectionList;
import org.apache.beam.sdk.values.PCollectionTuple;
import org.apache.beam.sdk.values.TupleTag;
import org.apache.beam.sdk.values.TupleTagList;
import org.apache.beam.sdk.values.TypeDescriptor;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/**
* {@link PTransform Pipeline transforms} used in pipelines that load from both Datastore export
* files and Nomulus CommitLog files.
*/
public final class Transforms {
private Transforms() {}
/**
* The commitTimestamp assigned to all entities loaded from a Datastore export file. The exact
* value does not matter, but it must be lower than the timestamps of real CommitLog records.
*/
@VisibleForTesting static final long EXPORT_ENTITY_TIME_STAMP = START_OF_TIME.getMillis();
/**
* Returns a {@link TupleTag} that can be used to retrieve entities of the given {@code kind} from
* the Datastore snapshot returned by {@link #loadDatastoreSnapshot}.
*/
public static TupleTag<VersionedEntity> createTagForKind(String kind) {
// When used with PCollectionTuple the result must retain generic type information.
// Both the Generic param and the empty bracket below are important.
return new TupleTag<VersionedEntity>(Transforms.class.getSimpleName() + ":" + kind) {};
}
/**
* Composite {@link PTransform transform} that loads the Datastore snapshot at {@code
* commitLogToTime} for caller specified {@code kinds}.
*
* <p>Caller must provide the location of a Datastore export that started AFTER {@code
* commitLogFromTime} and completed BEFORE {@code commitLogToTime}, as well as the root directory
* of all CommitLog files.
*
* <p>Selection of {@code commitLogFromTime} and {@code commitLogToTime} should follow the
* guidelines below to ensure that all incremental changes concurrent with the export are covered:
*
* <ul>
* <li>Two or more CommitLogs should exist between {@code commitLogFromTime} and the starting
* time of the Datastore export. This ensures that the earlier CommitLog file was complete
* before the export started.
* <li>Two or more CommitLogs should exit between the export completion time and {@code
* commitLogToTime}.
* </ul>
*
* <p>The output from the returned transform is a {@link PCollectionTuple} consisting of {@link
* VersionedEntity VersionedEntities} grouped into {@link PCollection PCollections} by {@code
* kind}.
*/
public static PTransform<PBegin, PCollectionTuple> loadDatastoreSnapshot(
String exportDir,
String commitLogDir,
DateTime commitLogFromTime,
DateTime commitLogToTime,
Set<String> kinds) {
checkArgument(kinds != null && !kinds.isEmpty(), "At least one kind is expected.");
// Create tags to collect entities by kind in final step.
final ImmutableMap<String, TupleTag<VersionedEntity>> outputTags =
kinds.stream()
.collect(ImmutableMap.toImmutableMap(kind -> kind, Transforms::createTagForKind));
// Arbitrarily select one tag as mainOutTag and put the remaining ones in a TupleTagList.
// This separation is required by ParDo's config API.
Iterator<TupleTag<VersionedEntity>> tagsIt = outputTags.values().iterator();
final TupleTag<VersionedEntity> mainOutputTag = tagsIt.next();
final TupleTagList additionalTags = TupleTagList.of(ImmutableList.copyOf(tagsIt));
return new PTransform<PBegin, PCollectionTuple>() {
@Override
public PCollectionTuple expand(PBegin input) {
PCollection<VersionedEntity> exportedEntities =
input
.apply("Get export file patterns", getDatastoreExportFilePatterns(exportDir, kinds))
.apply("Find export files", getFilesByPatterns())
.apply("Load export data", loadExportDataFromFiles());
PCollection<VersionedEntity> commitLogEntities =
input
.apply("Get commitlog file patterns", getCommitLogFilePatterns(commitLogDir))
.apply("Find commitlog files", getFilesByPatterns())
.apply(
"Filter commitLog by time",
filterCommitLogsByTime(commitLogFromTime, commitLogToTime))
.apply("Load commitlog data", loadCommitLogsFromFiles(kinds));
return PCollectionList.of(exportedEntities)
.and(commitLogEntities)
.apply("Merge exports and CommitLogs", Flatten.pCollections())
.apply(
"Key entities by Datastore Keys",
// Converting to KV<String, VE> instead of KV<Key, VE> b/c default coder for Key
// (SerializableCoder) is not deterministic and cannot be used with GroupBy.
MapElements.into(kvs(strings(), TypeDescriptor.of(VersionedEntity.class)))
.via((VersionedEntity e) -> KV.of(e.key().toString(), e)))
.apply("Gather entities by key", GroupByKey.create())
.apply(
"Output latest version per entity",
ParDo.of(
new DoFn<KV<String, Iterable<VersionedEntity>>, VersionedEntity>() {
@ProcessElement
public void processElement(
@Element KV<String, Iterable<VersionedEntity>> kv,
MultiOutputReceiver out) {
Optional<VersionedEntity> latest =
Streams.stream(kv.getValue())
.sorted(comparing(VersionedEntity::commitTimeMills).reversed())
.findFirst();
// Throw to abort (after default retries). Investigate, fix, and rerun.
checkState(
latest.isPresent(), "Unexpected key with no data", kv.getKey());
if (latest.get().isDelete()) {
return;
}
String kind = latest.get().getEntity().get().getKind();
out.get(outputTags.get(kind)).output(latest.get());
}
})
.withOutputTags(mainOutputTag, additionalTags));
}
};
}
/**
* Returns a {@link PTransform transform} that can generate a collection of patterns that match
* all Datastore CommitLog files.
*/
public static PTransform<PBegin, PCollection<String>> getCommitLogFilePatterns(
String commitLogDir) {
return toStringPCollection(BackupPaths.getCommitLogFilePatterns(commitLogDir));
}
/**
* Returns a {@link PTransform transform} that can generate a collection of patterns that match
* all Datastore export files of the given {@code kinds}.
*/
public static PTransform<PBegin, PCollection<String>> getDatastoreExportFilePatterns(
String exportDir, Collection<String> kinds) {
return toStringPCollection(getExportFilePatterns(exportDir, kinds));
}
public static PTransform<PBegin, PCollection<String>> getCloudSqlConnectionInfoFilePatterns(
String gcpProjectName) {
return toStringPCollection(BackupPaths.getCloudSQLCredentialFilePatterns(gcpProjectName));
}
/**
* Returns a {@link PTransform} from file name patterns to file {@link Metadata Metadata records}.
*/
public static PTransform<PCollection<String>, PCollection<Metadata>> getFilesByPatterns() {
return new PTransform<PCollection<String>, PCollection<Metadata>>() {
@Override
public PCollection<Metadata> expand(PCollection<String> input) {
return input.apply(FileIO.matchAll().withEmptyMatchTreatment(EmptyMatchTreatment.ALLOW));
}
};
}
/**
* Returns CommitLog files with timestamps between {@code fromTime} (inclusive) and {@code
* endTime} (exclusive).
*/
public static PTransform<PCollection<? extends Metadata>, PCollection<Metadata>>
filterCommitLogsByTime(DateTime fromTime, DateTime toTime) {
return ParDo.of(new FilterCommitLogFileByTime(fromTime, toTime));
}
/** Returns a {@link PTransform} from file {@link Metadata} to {@link VersionedEntity}. */
public static PTransform<PCollection<Metadata>, PCollection<VersionedEntity>>
loadExportDataFromFiles() {
return processFiles(
new BackupFileReader(
file ->
Iterators.transform(
LevelDbLogReader.from(file.open()),
(byte[] bytes) -> VersionedEntity.from(EXPORT_ENTITY_TIME_STAMP, bytes))));
}
/** Returns a {@link PTransform} from file {@link Metadata} to {@link VersionedEntity}. */
public static PTransform<PCollection<Metadata>, PCollection<VersionedEntity>>
loadCommitLogsFromFiles(Set<String> kinds) {
return processFiles(
new BackupFileReader(
file ->
CommitLogImports.loadEntities(file.open()).stream()
.filter(e -> kinds.contains(e.key().getKind()))
.iterator()));
}
/**
* Returns a {@link PTransform} that writes a {@link PCollection} of entities to a SQL database.
* and outputs an empty {@code PCollection<Void>}. This allows other operations to {@link
* org.apache.beam.sdk.transforms.Wait wait} for the completion of this transform.
*
* <p>Errors are handled according to the pipeline runner's default policy. As part of a one-time
* job, we will not add features unless proven necessary.
*
* @param transformId a unique ID for an instance of the returned transform
* @param maxWriters the max number of concurrent writes to SQL, which also determines the max
* number of connection pools created
* @param batchSize the number of entities to write in each operation
* @param jpaSupplier supplier of a {@link JpaTransactionManager}
*/
public static PTransform<PCollection<VersionedEntity>, PCollection<Void>> writeToSql(
String transformId,
int maxWriters,
int batchSize,
SerializableSupplier<JpaTransactionManager> jpaSupplier) {
return new PTransform<PCollection<VersionedEntity>, PCollection<Void>>() {
@Override
public PCollection<Void> expand(PCollection<VersionedEntity> input) {
return input
.apply(
"Shard data for " + transformId,
MapElements.into(kvs(integers(), TypeDescriptor.of(VersionedEntity.class)))
.via(ve -> KV.of(ThreadLocalRandom.current().nextInt(maxWriters), ve)))
.apply("Batch output by shard " + transformId, GroupIntoBatches.ofSize(batchSize))
.apply("Write in batch for " + transformId, ParDo.of(new SqlBatchWriter(jpaSupplier)));
}
};
}
/** Interface for serializable {@link Supplier suppliers}. */
public interface SerializableSupplier<T> extends Supplier<T>, Serializable {}
/**
* Returns a {@link PTransform} that produces a {@link PCollection} containing all elements in the
* given {@link Iterable}.
*/
static PTransform<PBegin, PCollection<String>> toStringPCollection(Iterable<String> strings) {
return Create.of(strings).withCoder(StringUtf8Coder.of());
}
/**
* Returns a {@link PTransform} from file {@link Metadata} to {@link VersionedEntity} using
* caller-provided {@code transformer}.
*/
static PTransform<PCollection<Metadata>, PCollection<VersionedEntity>> processFiles(
DoFn<ReadableFile, VersionedEntity> transformer) {
return new PTransform<PCollection<Metadata>, PCollection<VersionedEntity>>() {
@Override
public PCollection<VersionedEntity> expand(PCollection<Metadata> input) {
return input
.apply(FileIO.readMatches().withCompression(Compression.UNCOMPRESSED))
.apply(transformer.getClass().getSimpleName(), ParDo.of(transformer));
}
};
}
private static class FilterCommitLogFileByTime extends DoFn<Metadata, Metadata> {
private final DateTime fromTime;
private final DateTime toTime;
public FilterCommitLogFileByTime(DateTime fromTime, DateTime toTime) {
checkNotNull(fromTime, "fromTime");
checkNotNull(toTime, "toTime");
checkArgument(
fromTime.isBefore(toTime),
"Invalid time range: fromTime (%s) is before endTime (%s)",
fromTime,
toTime);
this.fromTime = fromTime;
this.toTime = toTime;
}
@ProcessElement
public void processElement(@Element Metadata fileMeta, OutputReceiver<Metadata> out) {
DateTime timestamp = getCommitLogTimestamp(fileMeta.resourceId().toString());
if (isBeforeOrAt(fromTime, timestamp) && timestamp.isBefore(toTime)) {
out.output(fileMeta);
}
}
}
/**
* Reads from a Datastore backup file and converts its content into {@link VersionedEntity
* VersionedEntities}.
*
* <p>The input file may be either a LevelDb file from a Datastore export or a CommitLog file
* generated by the Nomulus server. In either case, the file contains variable-length records and
* must be read sequentially from the beginning. If the read fails, the file needs to be retried
* from the beginning.
*/
private static class BackupFileReader extends DoFn<ReadableFile, VersionedEntity> {
private final ProcessFunction<ReadableFile, Iterator<VersionedEntity>> reader;
private BackupFileReader(ProcessFunction<ReadableFile, Iterator<VersionedEntity>> reader) {
this.reader = reader;
}
@ProcessElement
public void processElement(@Element ReadableFile file, OutputReceiver<VersionedEntity> out) {
try {
reader.apply(file).forEachRemaining(out::output);
} catch (Exception e) {
// Let the pipeline use default retry strategy on the whole file. For GCP Dataflow this
// means retrying up to 4 times (may include other files grouped with this one), and failing
// the pipeline if no success.
throw new RuntimeException(e);
}
}
}
/**
* Writes a batch of entities to a SQL database.
*
* <p>Note that an arbitrary number of instances of this class may be created and freed in
* arbitrary order in a single JVM. Due to the tech debt that forced us to use a static variable
* to hold the {@code JpaTransactionManager} instance, we must ensure that JpaTransactionManager
* is not changed or torn down while being used by some instance.
*/
private static class SqlBatchWriter extends DoFn<KV<Integer, Iterable<VersionedEntity>>, Void> {
private static int instanceCount = 0;
private static JpaTransactionManager originalJpa;
private final SerializableSupplier<JpaTransactionManager> jpaSupplier;
private transient Ofy ofy;
private transient SystemSleeper sleeper;
SqlBatchWriter(SerializableSupplier<JpaTransactionManager> jpaSupplier) {
this.jpaSupplier = jpaSupplier;
}
@Setup
public void setup() {
sleeper = new SystemSleeper();
try (AppEngineEnvironment env = new AppEngineEnvironment()) {
ObjectifyService.initOfy();
ofy = ObjectifyService.ofy();
}
synchronized (SqlBatchWriter.class) {
if (instanceCount == 0) {
originalJpa = jpaTm();
setJpaTm(jpaSupplier);
}
instanceCount++;
}
}
@Teardown
public void teardown() {
synchronized (SqlBatchWriter.class) {
instanceCount--;
if (instanceCount == 0) {
jpaTm().teardown();
setJpaTm(() -> originalJpa);
}
}
}
@ProcessElement
public void processElement(@Element KV<Integer, Iterable<VersionedEntity>> kv) {
try (AppEngineEnvironment env = new AppEngineEnvironment()) {
ImmutableList<Object> ofyEntities =
Streams.stream(kv.getValue())
.map(VersionedEntity::getEntity)
.map(Optional::get)
.map(ofy::toPojo)
.collect(ImmutableList.toImmutableList());
retry(() -> jpaTm().transact(() -> jpaTm().saveNewOrUpdateAll(ofyEntities)));
}
}
// TODO(b/160632289): Enhance Retrier and use it here.
private void retry(Runnable runnable) {
int maxAttempts = 5;
int initialDelayMillis = 100;
double jitterRatio = 0.2;
for (int attempt = 0; attempt < maxAttempts; attempt++) {
try {
runnable.run();
return;
} catch (Throwable throwable) {
if (!isFailedTxnRetriable(throwable)) {
throwIfUnchecked(throwable);
throw new RuntimeException(throwable);
}
int sleepMillis = (1 << attempt) * initialDelayMillis;
int jitter =
ThreadLocalRandom.current().nextInt((int) (sleepMillis * jitterRatio))
- (int) (sleepMillis * jitterRatio / 2);
sleeper.sleepUninterruptibly(Duration.millis(sleepMillis + jitter));
}
}
}
}
/**
* Removes BillingEvents, {@link google.registry.model.poll.PollMessage PollMessages} and {@link
* google.registry.model.host.HostResource} from a {@link DomainBase}. These are circular foreign
* key constraints that prevent migration of {@code DomainBase} to SQL databases.
*
* <p>See {@link InitSqlPipeline} for more information.
*/
static class RemoveDomainBaseForeignKeys extends DoFn<VersionedEntity, VersionedEntity> {
@ProcessElement
public void processElement(
@Element VersionedEntity domainBase, OutputReceiver<VersionedEntity> out) {
checkArgument(
domainBase.getEntity().isPresent(), "Unexpected delete entity %s", domainBase.key());
Entity outputEntity =
DomainBaseUtil.removeBillingAndPollAndHosts(domainBase.getEntity().get());
out.output(
VersionedEntity.from(
domainBase.commitTimeMills(),
EntityTranslator.convertToPb(outputEntity).toByteArray()));
}
}
}

View File

@@ -235,6 +235,7 @@ public abstract class BillingEvent implements Serializable {
DATE_TIME_FORMATTER.format(eventTime()),
registrarId(),
billingId(),
poNumber(),
tld(),
action(),
domain(),

View File

@@ -56,7 +56,8 @@ FROM (
`%PROJECT_ID%.%DATASTORE_EXPORT_DATA_SET%.%REGISTRY_TABLE%`
WHERE
-- TODO(b/18092292): Add a filter for tldState (not PDT/PREDELEGATION)
tldType = 'REAL') ) AS BillingEvent
tldType = 'REAL'
AND disableInvoicing is not TRUE) ) AS BillingEvent
-- Gather billing ID from registrar table
-- This is a 'JOIN' as opposed to 'LEFT JOIN' to filter out
-- non-billable registrars

View File

@@ -141,7 +141,7 @@ public class SafeBrowsingTransforms {
@ProcessElement
public void processElement(ProcessContext context) {
Subdomain subdomain = context.element();
subdomainBuffer.put(subdomain.fullyQualifiedDomainName(), subdomain);
subdomainBuffer.put(subdomain.domainName(), subdomain);
if (subdomainBuffer.size() >= BATCH_SIZE) {
ImmutableSet<KV<Subdomain, ThreatMatch>> results = evaluateAndFlush();
results.forEach(context::output);
@@ -239,7 +239,7 @@ public class SafeBrowsingTransforms {
String url = match.getJSONObject("threat").getString("url");
Subdomain subdomain = subdomainBuffer.get(url);
resultBuilder.add(
KV.of(subdomain, ThreatMatch.create(match, subdomain.fullyQualifiedDomainName())));
KV.of(subdomain, ThreatMatch.create(match, subdomain.domainName())));
}
}
}

View File

@@ -77,7 +77,7 @@ public class Spec11Pipeline implements Serializable {
public static final String REGISTRAR_EMAIL_FIELD = "registrarEmailAddress";
/** The JSON object field into which we put the registrar's name for Spec11 reports. */
public static final String REGISTRAR_CLIENT_ID_FIELD = "registrarClientId";
/** The JSON object field we put the threat match array for Spec11 reports. */
/** The JSON object field into which we put the threat match array for Spec11 reports. */
public static final String THREAT_MATCHES_FIELD = "threatMatches";
private final String projectId;
@@ -94,8 +94,7 @@ public class Spec11Pipeline implements Serializable {
@Config("spec11TemplateUrl") String spec11TemplateUrl,
@Config("reportingBucketUrl") String reportingBucketUrl,
@LocalCredential GoogleCredentialsBundle googleCredentialsBundle,
Retrier retrier
) {
Retrier retrier) {
this.projectId = projectId;
this.beamStagingUrl = beamStagingUrl;
this.spec11TemplateUrl = spec11TemplateUrl;
@@ -177,9 +176,11 @@ public class Spec11Pipeline implements Serializable {
PCollection<Subdomain> domains,
EvaluateSafeBrowsingFn evaluateSafeBrowsingFn,
ValueProvider<String> dateProvider) {
PCollection<KV<Subdomain, ThreatMatch>> subdomains =
/* Store ThreatMatch objects in JSON. */
PCollection<KV<Subdomain, ThreatMatch>> subdomainsJson =
domains.apply("Run through SafeBrowsingAPI", ParDo.of(evaluateSafeBrowsingFn));
subdomains
subdomainsJson
.apply(
"Map registrar client ID to email/ThreatMatch pair",
MapElements.into(
@@ -188,7 +189,7 @@ public class Spec11Pipeline implements Serializable {
.via(
(KV<Subdomain, ThreatMatch> kv) ->
KV.of(
kv.getKey().registrarClientId(),
kv.getKey().registrarId(),
EmailAndThreatMatch.create(
kv.getKey().registrarEmailAddress(), kv.getValue()))))
.apply("Group by registrar client ID", GroupByKey.create())

View File

@@ -36,12 +36,14 @@ import org.apache.beam.sdk.io.gcp.bigquery.SchemaAndRecord;
public abstract class Subdomain implements Serializable {
private static final ImmutableList<String> FIELD_NAMES =
ImmutableList.of("fullyQualifiedDomainName", "registrarClientId", "registrarEmailAddress");
ImmutableList.of("domainName", "domainRepoId", "registrarId", "registrarEmailAddress");
/** Returns the fully qualified domain name. */
abstract String fullyQualifiedDomainName();
/** Returns the client ID of the associated registrar for this domain. */
abstract String registrarClientId();
abstract String domainName();
/** Returns the domain repo ID (the primary key of the domain table). */
abstract String domainRepoId();
/** Returns the registrar ID of the associated registrar for this domain. */
abstract String registrarId();
/** Returns the email address of the registrar associated with this domain. */
abstract String registrarEmailAddress();
@@ -56,8 +58,9 @@ public abstract class Subdomain implements Serializable {
checkFieldsNotNull(FIELD_NAMES, schemaAndRecord);
GenericRecord record = schemaAndRecord.getRecord();
return create(
extractField(record, "fullyQualifiedDomainName"),
extractField(record, "registrarClientId"),
extractField(record, "domainName"),
extractField(record, "domainRepoId"),
extractField(record, "registrarId"),
extractField(record, "registrarEmailAddress"));
}
@@ -69,9 +72,11 @@ public abstract class Subdomain implements Serializable {
*/
@VisibleForTesting
static Subdomain create(
String fullyQualifiedDomainName, String registrarClientId, String registrarEmailAddress) {
String domainName,
String domainRepoId,
String registrarId,
String registrarEmailAddress) {
return new AutoValue_Subdomain(
fullyQualifiedDomainName, registrarClientId, registrarEmailAddress);
domainName, domainRepoId, registrarId, registrarEmailAddress);
}
}

View File

@@ -24,22 +24,10 @@ import org.json.JSONObject;
public abstract class ThreatMatch implements Serializable {
private static final String THREAT_TYPE_FIELD = "threatType";
private static final String PLATFORM_TYPE_FIELD = "platformType";
private static final String METADATA_FIELD = "threatEntryMetadata";
private static final String DOMAIN_NAME_FIELD = "fullyQualifiedDomainName";
/** Returns what kind of threat it is (malware, phishing etc.) */
public abstract String threatType();
/** Returns what platforms it affects (Windows, Linux etc.) */
abstract String platformType();
/**
* Returns a String representing a JSON Object containing arbitrary metadata associated with this
* threat, or "NONE" if there is no metadata to retrieve.
*
* <p>This ideally would be a {@link JSONObject} type, but can't be due to serialization
* requirements.
*/
abstract String metadata();
/** Returns the fully qualified domain name [SLD].[TLD] of the matched threat. */
public abstract String fullyQualifiedDomainName();
@@ -52,29 +40,19 @@ public abstract class ThreatMatch implements Serializable {
static ThreatMatch create(JSONObject threatMatchJSON, String fullyQualifiedDomainName)
throws JSONException {
return new AutoValue_ThreatMatch(
threatMatchJSON.getString(THREAT_TYPE_FIELD),
threatMatchJSON.getString(PLATFORM_TYPE_FIELD),
threatMatchJSON.has(METADATA_FIELD)
? threatMatchJSON.getJSONObject(METADATA_FIELD).toString()
: "NONE",
fullyQualifiedDomainName);
threatMatchJSON.getString(THREAT_TYPE_FIELD), fullyQualifiedDomainName);
}
/** Returns a {@link JSONObject} representing a subset of this object's data. */
JSONObject toJSON() throws JSONException {
return new JSONObject()
.put(THREAT_TYPE_FIELD, threatType())
.put(PLATFORM_TYPE_FIELD, platformType())
.put(METADATA_FIELD, metadata())
.put(DOMAIN_NAME_FIELD, fullyQualifiedDomainName());
}
/** Parses a {@link JSONObject} and returns an equivalent {@link ThreatMatch}. */
public static ThreatMatch fromJSON(JSONObject threatMatch) throws JSONException {
return new AutoValue_ThreatMatch(
threatMatch.getString(THREAT_TYPE_FIELD),
threatMatch.getString(PLATFORM_TYPE_FIELD),
threatMatch.getString(METADATA_FIELD),
threatMatch.getString(DOMAIN_NAME_FIELD));
threatMatch.getString(THREAT_TYPE_FIELD), threatMatch.getString(DOMAIN_NAME_FIELD));
}
}

View File

@@ -19,11 +19,13 @@
-- email address.
SELECT
domain.fullyQualifiedDomainName AS fullyQualifiedDomainName,
registrar.clientId AS registrarClientId,
domain.fullyQualifiedDomainName AS domainName,
domain.__key__.name AS domainRepoId,
registrar.clientId AS registrarId,
COALESCE(registrar.emailAddress, '') AS registrarEmailAddress
FROM ( (
SELECT
__key__,
fullyQualifiedDomainName,
currentSponsorClientId,
creationTime

View File

@@ -1527,6 +1527,21 @@ public final class RegistryConfig {
return CONFIG_SETTINGS.get().hibernate.hikariIdleTimeout;
}
/**
* Returns whether to replicate cloud SQL transactions to datastore.
*
* <p>If true, all cloud SQL transactions will be persisted as TransactionEntity objects in the
* Transaction table and replayed against datastore in a cron job.
*/
public static boolean getCloudSqlReplicateTransactions() {
return CONFIG_SETTINGS.get().cloudSql.replicateTransactions;
}
@VisibleForTesting
public static void overrideCloudSqlReplicateTransactions(boolean replicateTransactions) {
CONFIG_SETTINGS.get().cloudSql.replicateTransactions = replicateTransactions;
}
/** Returns the roid suffix to be used for the roids of all contacts and hosts. */
public static String getContactAndHostRoidSuffix() {
return CONFIG_SETTINGS.get().registryPolicy.contactAndHostRoidSuffix;

View File

@@ -122,6 +122,7 @@ public class RegistryConfigSettings {
public String jdbcUrl;
public String username;
public String instanceConnectionName;
public boolean replicateTransactions;
}
/** Configuration for Apache Beam (Cloud Dataflow). */

View File

@@ -230,6 +230,9 @@ cloudSql:
username: username
# This name is used by Cloud SQL when connecting to the database.
instanceConnectionName: project-id:region:instance-id
# Set this to true to replicate cloud SQL transactions to datastore in the
# background.
replicateTransactions: false
cloudDns:
# Set both properties to null in Production.

View File

@@ -119,35 +119,31 @@ public class DnsQueue {
.param(PARAM_TLD, tld));
}
/**
* Adds a task to the queue to refresh the DNS information for the specified subordinate host.
*/
public TaskHandle addHostRefreshTask(String fullyQualifiedHostName) {
Optional<InternetDomainName> tld =
Registries.findTldForName(InternetDomainName.from(fullyQualifiedHostName));
checkArgument(tld.isPresent(),
String.format("%s is not a subordinate host to a known tld", fullyQualifiedHostName));
return addToQueue(TargetType.HOST, fullyQualifiedHostName, tld.get().toString(), Duration.ZERO);
/** Adds a task to the queue to refresh the DNS information for the specified subordinate host. */
public TaskHandle addHostRefreshTask(String hostName) {
Optional<InternetDomainName> tld = Registries.findTldForName(InternetDomainName.from(hostName));
checkArgument(
tld.isPresent(), String.format("%s is not a subordinate host to a known tld", hostName));
return addToQueue(TargetType.HOST, hostName, tld.get().toString(), Duration.ZERO);
}
/** Enqueues a task to refresh DNS for the specified domain now. */
public TaskHandle addDomainRefreshTask(String fullyQualifiedDomainName) {
return addDomainRefreshTask(fullyQualifiedDomainName, Duration.ZERO);
public TaskHandle addDomainRefreshTask(String domainName) {
return addDomainRefreshTask(domainName, Duration.ZERO);
}
/** Enqueues a task to refresh DNS for the specified domain at some point in the future. */
public TaskHandle addDomainRefreshTask(String fullyQualifiedDomainName, Duration countdown) {
public TaskHandle addDomainRefreshTask(String domainName, Duration countdown) {
return addToQueue(
TargetType.DOMAIN,
fullyQualifiedDomainName,
assertTldExists(getTldFromDomainName(fullyQualifiedDomainName)),
domainName,
assertTldExists(getTldFromDomainName(domainName)),
countdown);
}
/** Adds a task to the queue to refresh the DNS information for the specified zone. */
public TaskHandle addZoneRefreshTask(String fullyQualifiedZoneName) {
return addToQueue(
TargetType.ZONE, fullyQualifiedZoneName, fullyQualifiedZoneName, Duration.ZERO);
public TaskHandle addZoneRefreshTask(String zoneName) {
return addToQueue(TargetType.ZONE, zoneName, zoneName, Duration.ZERO);
}
/**

View File

@@ -89,7 +89,7 @@ public final class RefreshDnsAction implements Runnable {
private static void verifyHostIsSubordinate(HostResource host) {
if (!host.isSubordinate()) {
throw new BadRequestException(
String.format("%s isn't a subordinate hostname", host.getFullyQualifiedHostName()));
String.format("%s isn't a subordinate hostname", host.getHostName()));
}
}
}

View File

@@ -154,7 +154,7 @@ public class CloudDnsWriter extends BaseDnsWriter {
}
// Construct NS records (if any).
Set<String> nameserverData = domainBase.get().loadNameserverFullyQualifiedHostNames();
Set<String> nameserverData = domainBase.get().loadNameserverHostNames();
Set<String> subordinateHosts = domainBase.get().getSubordinateHosts();
if (!nameserverData.isEmpty()) {
HashSet<String> nsRrData = new HashSet<>();

View File

@@ -189,7 +189,7 @@ public class DnsUpdateWriter extends BaseDnsWriter {
for (DelegationSignerData signerData : domain.getDsData()) {
DSRecord dsRecord =
new DSRecord(
toAbsoluteName(domain.getFullyQualifiedDomainName()),
toAbsoluteName(domain.getDomainName()),
DClass.IN,
dnsDefaultDsTtl.getStandardSeconds(),
signerData.getKeyTag(),
@@ -215,8 +215,7 @@ public class DnsUpdateWriter extends BaseDnsWriter {
private void addInBailiwickNameServerSet(DomainBase domain, Update update) {
for (String hostName :
intersection(
domain.loadNameserverFullyQualifiedHostNames(), domain.getSubordinateHosts())) {
intersection(domain.loadNameserverHostNames(), domain.getSubordinateHosts())) {
Optional<HostResource> host = loadByForeignKey(HostResource.class, hostName, clock.nowUtc());
checkState(host.isPresent(), "Host %s cannot be loaded", hostName);
update.add(makeAddressSet(host.get()));
@@ -226,10 +225,10 @@ public class DnsUpdateWriter extends BaseDnsWriter {
private RRset makeNameServerSet(DomainBase domain) {
RRset nameServerSet = new RRset();
for (String hostName : domain.loadNameserverFullyQualifiedHostNames()) {
for (String hostName : domain.loadNameserverHostNames()) {
NSRecord record =
new NSRecord(
toAbsoluteName(domain.getFullyQualifiedDomainName()),
toAbsoluteName(domain.getDomainName()),
DClass.IN,
dnsDefaultNsTtl.getStandardSeconds(),
toAbsoluteName(hostName));
@@ -244,7 +243,7 @@ public class DnsUpdateWriter extends BaseDnsWriter {
if (address instanceof Inet4Address) {
ARecord record =
new ARecord(
toAbsoluteName(host.getFullyQualifiedHostName()),
toAbsoluteName(host.getHostName()),
DClass.IN,
dnsDefaultATtl.getStandardSeconds(),
address);
@@ -260,7 +259,7 @@ public class DnsUpdateWriter extends BaseDnsWriter {
if (address instanceof Inet6Address) {
AAAARecord record =
new AAAARecord(
toAbsoluteName(host.getFullyQualifiedHostName()),
toAbsoluteName(host.getHostName()),
DClass.IN,
dnsDefaultATtl.getStandardSeconds(),
address);

View File

@@ -1,275 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.documentation;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.MoreCollectors.onlyElement;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMultimap;
import com.google.common.collect.ListMultimap;
import com.google.common.collect.Multimaps;
import com.sun.javadoc.AnnotationDesc;
import com.sun.javadoc.ClassDoc;
import com.sun.javadoc.FieldDoc;
import com.sun.javadoc.SeeTag;
import com.sun.javadoc.Tag;
import google.registry.model.eppoutput.Result.Code;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.List;
import java.util.NoSuchElementException;
import java.util.TreeMap;
import java.util.stream.Stream;
import javax.annotation.Nullable;
/**
* Class to represent documentation information for a single EPP flow.
*
* <p>The static method getFlowDocs() on this class returns a list of FlowDocumentation
* instances corresponding to the leaf flows in the flows package, constructing the instances
* from class information returned from the javadoc system. Each instance has methods for
* retrieving relevant information about the flow, such as a description, error conditions, etc.
*/
public class FlowDocumentation {
/** Constants for names of various relevant packages and classes. */
static final String FLOW_PACKAGE_NAME = "google.registry.flows";
static final String BASE_FLOW_CLASS_NAME = FLOW_PACKAGE_NAME + ".Flow";
static final String EXCEPTION_CLASS_NAME = FLOW_PACKAGE_NAME + ".EppException";
static final String CODE_ANNOTATION_NAME = EXCEPTION_CLASS_NAME + ".EppResultCode";
/** Name of the class for this flow. */
private final String name;
/** Fully qualified name of the class for this flow. */
private final String qualifiedName;
/** Name of the package in which this flow resides. */
private final String packageName;
/** Class docs for the flow. */
private final String classDocs;
/** Javadoc-tagged error conditions for this flow in list form. */
private final List<ErrorCase> errors;
/** Javadoc-tagged error conditions for this flow, organized by underlying error code. */
private final ListMultimap<Long, ErrorCase> errorsByCode;
/**
* Creates a FlowDocumentation for this flow class using data from javadoc tags. Not public
* because clients should get FlowDocumentation objects via the DocumentationGenerator class.
*/
protected FlowDocumentation(ClassDoc flowDoc) {
name = flowDoc.name();
qualifiedName = flowDoc.qualifiedName();
packageName = flowDoc.containingPackage().name();
classDocs = flowDoc.commentText();
errors = new ArrayList<>();
// Store error codes in sorted order, and leave reasons in insert order.
errorsByCode =
Multimaps.newListMultimap(new TreeMap<Long, Collection<ErrorCase>>(), ArrayList::new);
parseTags(flowDoc);
}
public String getName() {
return name;
}
public String getQualifiedName() {
return qualifiedName;
}
public String getPackageName() {
return packageName;
}
public String getClassDocs() {
return classDocs;
}
public ImmutableList<ErrorCase> getErrors() {
return ImmutableList.copyOf(errors);
}
public ImmutableMultimap<Long, ErrorCase> getErrorsByCode() {
return ImmutableMultimap.copyOf(errorsByCode);
}
/** Iterates through javadoc tags on the underlying class and calls specific parsing methods. */
private void parseTags(ClassDoc flowDoc) {
for (Tag tag : flowDoc.tags()) {
// Everything else is not a relevant tag.
if ("@error".equals(tag.name())) {
parseErrorTag(tag);
}
}
}
/** Exception to throw when an @error tag cannot be parsed correctly. */
private static class BadErrorTagFormatException extends IllegalStateException {
/** Makes a message to use as a prefix for the reason passed up to the superclass. */
private static String makeMessage(String reason, Tag tag) {
return String.format("Bad @error tag format at %s - %s", tag.position(), reason);
}
private BadErrorTagFormatException(String reason, Tag tag) {
super(makeMessage(reason, tag));
}
private BadErrorTagFormatException(String reason, Tag tag, Exception cause) {
super(makeMessage(reason, tag), cause);
}
}
/** Parses a javadoc tag corresponding to an error case and updates the error mapping. */
private void parseErrorTag(Tag tag) {
// Parse the @error tag text to find the @link inline tag.
SeeTag linkedTag;
try {
linkedTag =
Stream.of(tag.inlineTags())
.filter(SeeTag.class::isInstance)
.map(SeeTag.class::cast)
.collect(onlyElement());
} catch (NoSuchElementException | IllegalArgumentException e) {
throw new BadErrorTagFormatException(
String.format("expected one @link tag in tag text but found %s: %s",
(e instanceof NoSuchElementException ? "none" : "multiple"),
tag.text()),
tag, e);
}
// Check to see if the @link tag references a valid class.
ClassDoc exceptionRef = linkedTag.referencedClass();
if (exceptionRef == null) {
throw new BadErrorTagFormatException(
"could not resolve class from @link tag text: " + linkedTag.text(),
tag);
}
// Try to convert the referenced class into an ErrorCase; fail if it's not an EppException.
ErrorCase error;
try {
error = new ErrorCase(exceptionRef);
} catch (IllegalStateException | IllegalArgumentException e) {
throw new BadErrorTagFormatException(
"class referenced in @link is not a valid EppException: " + exceptionRef.qualifiedName(),
tag, e);
}
// Success; store this as a parsed error case.
errors.add(error);
errorsByCode.put(error.getCode(), error);
}
/**
* Represents an error case for a flow, with a reason for the error and the EPP error code.
*
* <p>This class is an immutable wrapper for the name of an EppException subclass that gets
* thrown to indicate an error condition. It overrides equals() and hashCode() so that
* instances of this class can be used in collections in the normal fashion.
*/
public static class ErrorCase {
/** The non-qualified name of the exception class. */
private final String name;
/** The fully-qualified name of the exception class. */
private final String className;
/** The reason this error was thrown, normally documented on the low-level exception class. */
private final String reason;
/** The EPP error code value corresponding to this error condition. */
private final long errorCode;
/** Constructs an ErrorCase from the corresponding class for a low-level flow exception. */
protected ErrorCase(ClassDoc exceptionDoc) {
name = exceptionDoc.name();
className = exceptionDoc.qualifiedName();
// The javadoc comment on the class explains the reason for the error condition.
reason = exceptionDoc.commentText();
ClassDoc highLevelExceptionDoc = getHighLevelExceptionFrom(exceptionDoc);
errorCode = extractErrorCode(highLevelExceptionDoc);
checkArgument(!exceptionDoc.isAbstract(),
"Cannot use an abstract subclass of EppException as an error case");
}
public String getName() {
return name;
}
protected String getClassName() {
return className;
}
public String getReason() {
return reason;
}
public long getCode() {
return errorCode;
}
/** Returns the direct subclass of EppException that this class is a subclass of (or is). */
private ClassDoc getHighLevelExceptionFrom(ClassDoc exceptionDoc) {
// While we're not yet at the root, move up the class hierarchy looking for EppException.
while (exceptionDoc.superclass() != null) {
if (exceptionDoc.superclass().qualifiedTypeName().equals(EXCEPTION_CLASS_NAME)) {
return exceptionDoc;
}
exceptionDoc = exceptionDoc.superclass();
}
// Failure; we reached the root without finding a subclass of EppException.
throw new IllegalArgumentException(
String.format("Class referenced is not a subclass of %s", EXCEPTION_CLASS_NAME));
}
/** Returns the corresponding EPP error code for an annotated subclass of EppException. */
private long extractErrorCode(ClassDoc exceptionDoc) {
try {
// We're looking for a specific annotation by name that should appear only once.
AnnotationDesc errorCodeAnnotation =
Arrays.stream(exceptionDoc.annotations())
.filter(
anno -> anno.annotationType().qualifiedTypeName().equals(CODE_ANNOTATION_NAME))
.findFirst()
.get();
// The annotation should have one element whose value converts to an EppResult.Code.
AnnotationDesc.ElementValuePair pair = errorCodeAnnotation.elementValues()[0];
String enumConstant = ((FieldDoc) pair.value().value()).name();
return Code.valueOf(enumConstant).code;
} catch (IllegalStateException e) {
throw new IllegalStateException(
"No error code annotation found on exception " + exceptionDoc.name(), e);
} catch (ArrayIndexOutOfBoundsException | ClassCastException | IllegalArgumentException e) {
throw new IllegalStateException("Bad annotation on exception " + exceptionDoc.name(), e);
}
}
@Override
public boolean equals(@Nullable Object object) {
// The className field canonically identifies the EppException wrapped by this class, and
// all other instance state is derived from that exception, so we only check className.
return object instanceof ErrorCase && this.className.equals(((ErrorCase) object).className);
}
@Override
public int hashCode() {
// See note for equals() - only className is needed for comparisons.
return className.hashCode();
}
}
}

View File

@@ -1,159 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.documentation;
import static google.registry.util.BuildPathUtils.getProjectRoot;
import static java.nio.charset.StandardCharsets.UTF_8;
import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import com.google.common.io.CharStreams;
import com.sun.javadoc.RootDoc;
import com.sun.tools.javac.file.JavacFileManager;
import com.sun.tools.javac.util.Context;
import com.sun.tools.javac.util.ListBuffer;
import com.sun.tools.javadoc.JavadocTool;
import com.sun.tools.javadoc.Messager;
import com.sun.tools.javadoc.ModifierFilter;
import java.io.File;
import java.io.IOException;
import java.io.PrintWriter;
import java.lang.reflect.Modifier;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import java.util.Locale;
import javax.annotation.Nullable;
import javax.tools.StandardLocation;
/**
* Wrapper class to simplify calls to the javadoc system and hide internal details. An instance
* represents a set of parameters for calling out to javadoc; these parameters can be set via
* the appropriate methods, and determine what files and packages javadoc will process. The
* actual running of javadoc occurs when calling getRootDoc() to retrieve a javadoc RootDoc.
*/
public final class JavadocWrapper {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
/** Shows any member visible at at least the default (package) level. */
private static final long VISIBILITY_MASK =
Modifier.PUBLIC | Modifier.PROTECTED | ModifierFilter.PACKAGE;
/** Root directory for source files. If null, will use the current directory. */
private static final String SOURCE_PATH = getProjectRoot().resolve("core/src/main/java")
.toString();
/** Specific source files to generate documentation for. */
private static final ImmutableSet<String> SOURCE_FILE_NAMES = ImmutableSet.of();
/** Specific packages to generate documentation for. */
private static final ImmutableSet<String> SOURCE_PACKAGE_NAMES =
ImmutableSet.of(FlowDocumentation.FLOW_PACKAGE_NAME);
/** Whether or not the Javadoc tool should eschew excessive log output. */
private static final boolean QUIET = true;
/**
* Obtains a Javadoc {@link RootDoc} object containing raw Javadoc documentation.
* Wraps a call to the static method createRootDoc() and passes in instance-specific settings.
*/
public static RootDoc getRootDoc() throws IOException {
logger.atInfo().log("Starting Javadoc tool");
File sourceFilePath = new File(SOURCE_PATH);
logger.atInfo().log("Using source directory: %s", sourceFilePath.getAbsolutePath());
try {
return createRootDoc(
SOURCE_PATH,
SOURCE_PACKAGE_NAMES,
SOURCE_FILE_NAMES,
VISIBILITY_MASK,
QUIET);
} finally {
logger.atInfo().log("Javadoc tool finished");
}
}
/**
* Obtains a Javadoc root document object for the specified source path and package/Java names.
* If the source path is null, then the working directory is assumed as the source path.
*
* <p>If a list of package names is provided, then Javadoc will run on these packages and all
* their subpackages, based out of the specified source path.
*
* <p>If a list of file names is provided, then Javadoc will also run on these Java source files.
* The specified source path is not considered in this case.
*
* @see <a href="http://relation.to/12969.lace">Testing Java doclets</a>
* @see <a href="http://www.docjar.com/docs/api/com/sun/tools/javadoc/JavadocTool.html">JavadocTool</a>
*/
private static RootDoc createRootDoc(
@Nullable String sourcePath,
Collection<String> packageNames,
Collection<String> fileNames,
long visibilityMask,
boolean quiet) throws IOException {
// Create a context to hold settings for Javadoc.
Context context = new Context();
// Redirect Javadoc stdout/stderr to null writers, since otherwise the Java compiler
// issues lots of errors for classes that are imported and visible to blaze but not
// visible locally to the compiler.
// TODO(b/19124943): Find a way to ignore those errors so we can show real ones?
Messager.preRegister(
context,
JavadocWrapper.class.getName(),
new PrintWriter(CharStreams.nullWriter()), // For errors.
new PrintWriter(CharStreams.nullWriter()), // For warnings.
new PrintWriter(CharStreams.nullWriter())); // For notices.
// Set source path option for Javadoc.
try (JavacFileManager fileManager = new JavacFileManager(context, true, UTF_8)) {
List<File> sourcePathFiles = new ArrayList<>();
if (sourcePath != null) {
for (String sourcePathEntry : Splitter.on(':').split(sourcePath)) {
sourcePathFiles.add(new File(sourcePathEntry));
}
}
fileManager.setLocation(StandardLocation.SOURCE_PATH, sourcePathFiles);
// Create an instance of Javadoc.
JavadocTool javadocTool = JavadocTool.make0(context);
// Convert the package and file lists to a format Javadoc can understand.
ListBuffer<String> subPackages = new ListBuffer<>();
subPackages.addAll(packageNames);
ListBuffer<String> javaNames = new ListBuffer<>();
javaNames.addAll(fileNames);
// Invoke Javadoc and ask it for a RootDoc containing the specified packages.
return javadocTool.getRootDocImpl(
Locale.US.toString(), // Javadoc comment locale
UTF_8.name(), // Source character encoding
new ModifierFilter(visibilityMask), // Element visibility filter
javaNames.toList(), // Included Java file names
com.sun.tools.javac.util.List.nil(), // Doclet options
com.sun.tools.javac.util.List.nil(), // Source files
false, // Don't use BreakIterator
subPackages.toList(), // Included sub-package names
com.sun.tools.javac.util.List.nil(), // Excluded package names
false, // Read source files, not classes
false, // Don't run legacy doclet
quiet); // If asked, run Javadoc quietly
}
}
private JavadocWrapper() {}
}

View File

@@ -1,38 +0,0 @@
#!/bin/bash
# Copyright 2017 The Nomulus Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Generate javadoc for the project
if (( $# != 3 )); then
echo "Usage: $0 JAVADOC ZIP OUT" 1>&2
exit 1
fi
JAVADOC_BINARY="$1"
ZIP_BINARY="$2"
TARGETFILE="$3"
TMPDIR="$(mktemp -d "${TMPDIR:-/tmp}/generate_javadoc.XXXXXXXX")"
PWDDIR="$(pwd)"
"${JAVADOC_BINARY}" -d "${TMPDIR}" \
$(find java -name \*.java) \
-tag error:t:'EPP Errors' \
-subpackages google.registry \
-exclude google.registry.dns:google.registry.proxy:google.registry.monitoring.blackbox
cd "${TMPDIR}"
"${PWDDIR}/${ZIP_BINARY}" -rXoq "${PWDDIR}/${TARGETFILE}" .
cd -
rm -rf "${TMPDIR}"

View File

@@ -367,6 +367,12 @@
<url-pattern>/_dr/task/linkRdeHosts</url-pattern>
</servlet-mapping>
<!-- Action to automatically re-lock a domain after unlocking it -->
<servlet-mapping>
<servlet-name>backend-servlet</servlet-name>
<url-pattern>/_dr/task/relockDomain</url-pattern>
</servlet-mapping>
<!-- Security config -->
<security-constraint>
<web-resource-collection>

View File

@@ -284,7 +284,7 @@
<description>
Checks if the monthly ICANN reports have been successfully uploaded. If they have not, attempts to upload them again.
Most of the time, this job should not do anything since the uploads are triggered when the reports are staged.
However, in the event that an upload failed for any reason (e.g. ICANN server is down, IP whitelist issues),
However, in the event that an upload failed for any reason (e.g. ICANN server is down, IP allow list issues),
this cron job will continue to retry uploads daily until they succeed.
</description>
<schedule>every day 15:00</schedule>

View File

@@ -7,7 +7,7 @@
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<manual-scaling>
<instances>10</instances>
<instances>20</instances>
</manual-scaling>
<system-properties>

View File

@@ -18,7 +18,14 @@
and streams it to cloud storage. When this job has finished successfully, it'll
launch a separate task that uploads the deposit file to Iron Mountain via SFTP.
</description>
<schedule>every day 00:07</schedule>
<!--
This only needs to run once per day, but we launch additional jobs in case the
cursor is lagging behind, so it'll catch up to the current date eventually.
See <a href="../../../production/default/WEB-INF/cron.xml">production config</a> for an
explanation of job starting times.
-->
<schedule>every 12 hours from 00:07 to 12:07</schedule>
<target>backend</target>
</cron>

Some files were not shown because too many files have changed in this diff Show More