1
0
mirror of https://github.com/google/nomulus synced 2026-01-22 05:33:12 +00:00

Compare commits

...

112 Commits

Author SHA1 Message Date
gbrodman
cf89d9354c Add a registration_behavior column to AllocationToken (#1695)
This is, as of now, unused but we can use it for b/237683906 and
b/237800445 in the future to allow for special behavior dictated by
allocation tokens rather than having to reserve specific domains.

Note that we enforce a tied domain for ANCHOR_TENANT tokens (because
they should be matched to a domain) but not for BYPASS_TLD_STATE tokens.
2022-07-20 12:50:25 -04:00
Lai Jiang
49b1b2d058 Remove support for @Mapify (#1691)
We no longer need to support Objectify's @Mapify logic. This
substantially simplified how we store maps in the database.
2022-07-19 11:13:23 -04:00
gbrodman
47ce568964 Delete unnecessary replay-related objects from DBs (#1692)
This deletes LastSqlTransaction and ReplayGap in Datastore, as well as
SqlReplayCheckpoint and TransactionEntity in SQL. These are all related
to replay and are no longer used.
2022-07-15 14:50:28 -04:00
Michael Muller
5934fecd4f Remove columns for unused ofy key reconstitution (#1706)
* Remove columns for unused ofy key reconstitution

Remove the columns in Domain, DomainHistory, GracePeriod and
GracePeriodHistory that were only used for Ofy key reconstitution.

All uses of these were removed in #1660.

* Add forgotten flyway file.
2022-07-14 16:49:08 -04:00
gbrodman
f72487fe2e Tell IANA not to encode the RDAP base URL response (#1705) 2022-07-13 14:31:16 -04:00
Ben McIlwain
5cb2a0a430 Re-add database migration state commands (#1702)
* Re-add database migration state commands

These were removed in PR #1661, but we do still need them for the time being
until we complete the ID migration as well.
2022-07-12 16:49:16 -04:00
gbrodman
dbb96d36d3 Refactor SMDRL + Claims CSV parsing (#1704)
This uses the Apache commons CSV parsing instead of rolling our own.

Annoyingly, the results that we're given aren't exactly proper CSVs
since they have a non-standard line of data at the top, and the header
is actually the second line.
2022-07-12 15:42:12 -04:00
gbrodman
02145d34d9 Use the new IANA url for registrar RDAP base URLs (#1703)
Fortunately this no longer requires a log-in, we can just send a GET
request and receive a CSV result in return.

This also adds the apache-commons CSV parser to the dependencies

See https://b.corp.google.com/issues/237784559 for more details
2022-07-12 14:02:38 -04:00
Ben McIlwain
36becfb54f Use new renew cost calculation in handleFeeRequest() (#1694)
* Resolve conflict

* Fix setup for existing test cases in info and check flow

* Revise info flow test cases

* Fix lint

* Merge branch 'master' into handlefeerequest-renew

* Address code review comments myself

* Merge branch 'master' into handlefeerequest-renew

* Get test passing

* Add check flow tests

* Format, consolidate test helpers

* Don't unnecessarily specify XML name
2022-07-07 17:28:45 -04:00
gbrodman
2a5b427a80 Add non-SQL removal code for Transaction and SqlReplayCheckpoint (#1700) 2022-07-07 14:36:01 -04:00
gbrodman
9a2fb6f8b4 Add SQL column for AllocationToken registration behavior (#1697)
First part of b/237683906

We'll add the Java behavior in a later PR; it'll be an enum with the
values DEFAULT, BYPASS_TLD_STATE, and ANCHOR_TENANT
2022-07-01 18:26:18 -04:00
gbrodman
abc240fc2d Revert "Remove DatabaseMigrationStateSchedule (#1689)" (#1698)
This reverts commit 18d51738ce.
2022-07-01 17:25:13 -04:00
Michael Muller
a424030a65 Deal with some getOfyKey() references (#1696)
Now that ofy keys aren't necessarily being restored, it seemed prudent to
audit existing uses to verify that we aren't relying on any keys that may now
be null.

This fixes one case that appeared to be potentially problematic (in
ResourceFlowUtils), removes a few methods that call getOfyKey() but are no
longer used, and adds comments to one use of the key that appears to be safe
after visual inspection.
2022-07-01 16:59:03 -04:00
gbrodman
72abc824d5 Delete DatastoreTM and most other references to Datastore (#1681)
This includes:
- deletion of helper DB methods in tests
- deletion of various old Datastore-only classes and removal of any
  endpoints
- removal of the dual-database test concept
- removal of 'ofy' from the AppEngineExtension
2022-07-01 13:33:38 -04:00
Lai Jiang
b2ec088749 Remove Ofy support from Registry (#1688)
Also made some code quality changes based on IntelliJ suggestions on
modified files.
2022-07-01 09:04:46 -04:00
Ben McIlwain
6d40fe41e6 Use DomainPricingLogic in ExpandRecurringBillingEventsAction (#1687)
* Inject DomainPricingLogic to action file

* Remove attempt to inject

* Merge conflict

* Fix non static to static issue

* Merge branch 'master' into at_internal/Expand
2022-06-29 17:39:15 -04:00
Weimin Yu
7dafbf6ae1 Remove retries for DatabaseSnapshot tests (#1690)
* Remove retries for DatabaseSnapshot tests
2022-06-29 14:45:41 -04:00
Michael Muller
9b8f10c595 Remove ofy scaffolding from DomainContent and core classes (#1660)
This removes the code that converts between ofy fields and SQL fields in DomainContent and a number of related core classes (basically anything that also needed modification to support the removal from DomainContent).
2022-06-29 14:39:25 -04:00
Lai Jiang
18d51738ce Remove DatabaseMigrationStateSchedule (#1689) 2022-06-29 12:14:45 -04:00
Lai Jiang
76d63b24a8 Remove Ofy support from Cursor (#1672)
Cursor was originally envisioned to support arbitary ImmutableObject
scopes. However, in practice only the Registry scope is used. The SQL
representation of Cursor assumes that and the schema uses a composite ID
with a string column for the primary key of the scope object. Without a
schema migration to persist the VKey of the scope, we cannot support any
ImmutableObject other than those with a primitive string primary key.

Given the complexity involved and the limited use case, the scope is now
explictly limited to Registry only.

Also removed mapreduces that depends on Ofy keys of Cursors, and made
some code quality improvement based on IntelliJ suggestions on modified
files.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1672)
<!-- Reviewable:end -->
2022-06-28 14:59:21 -04:00
Michael Muller
eb1b283ba3 In shell mode, only do database setup once (#1686)
We were initializing ofy and JPA every time the command was run, causing shell
commands to break after 64 transactions.
2022-06-28 09:27:39 -04:00
Lai Jiang
63e4f4f10a Remove Ofy from RegistrarContact (#1680)
Also renamed the class to RegistrarPoc and deleted some unused methods.
2022-06-27 20:17:28 -04:00
sarahcaseybot
2c3279ba95 Use VKeys instead of Ofy keys in mutating command (#1682)
* Use VKeys instead of Ofy keys in mutating command

* Add createVKey to ImmutableObject

* Use SQL only VKeys
2022-06-27 17:49:24 -04:00
Ben McIlwain
89925f9ff2 Fix license-checking on GWT 2.10.0, which is Apache 2.0 (#1685) 2022-06-27 12:24:32 -04:00
Lai Jiang
585765b83a Remove the beam parameter in RDE staging action (#1684)
The parameter was used to force a RDE beam run, which is no longer
necessary, now that the mapreduce pipeline is deleted.
2022-06-27 10:45:52 -04:00
Ben McIlwain
cddcfc49ed Make domain:renew commands use the renewal price behavior (#1683)
* Make domain:renew commands use the renewal price behavior

This is based on PR #1592 by @rachelguan.
2022-06-24 17:36:28 -04:00
sarahcaseybot
fb7558121b Add flyway files to remove billing_identifier from Registrar (#1652) 2022-06-24 12:16:33 -04:00
Lai Jiang
1719d066cf Disable all tests that uses Ofy (#1679)
These tests use Ofy exclusively and should not run anymore, as any class
they test also use Ofy and should be deleted.

More importantly, running tests in Ofy mode makes it hard to remove Ofy
from entities, especially Registrar and RegistrarContact, as some of
them are created as canned data when tests are initiated, and the
creation would fail if they are not registered as Ofy entities.

It is therefore a prerequisite to disable these tests before we can
remove Ofy from those entities. We could have deleted them, but I think
that should be done when the corresponding classes tested by them are
deleted.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1679)
<!-- Reviewable:end -->
2022-06-23 14:19:20 -04:00
gbrodman
fa1b34b020 Remove CommitLog and MapReduce-related code (#1670) 2022-06-23 12:54:47 -04:00
Lai Jiang
4298084406 Pass the withoutCannedData boolean to JpaTransactionManagerExtension builder (#1678)
This is an oversight from #1673

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1678)
<!-- Reviewable:end -->
2022-06-23 10:37:14 -04:00
Weimin Yu
9b5f1756f1 Remove obsolete gradle tasks (#1677)
* Remove obsolete gradle tasks
2022-06-22 22:45:34 -04:00
Lai Jiang
62236f7581 Only use GPG2 in tests (#1676)
GPG1 is deprecated and stuck in v1.4 from 2018. GPG2 is recommended. We
only use the GPG binary in tests and when the host system has both
versions it causes problems because we hardcode the GPG import command
in GpgSystemCommandExension to use the binary named "gpg", which could
be linked to either GPG1 or GPG2, causing the other test to fail when
the version of GPG that runs in tests is incompatible with the version of GPG
that imports the keys.

With this PR we only support GPG2 from now on.
2022-06-22 11:03:41 -04:00
Lai Jiang
e1ea176daa Add some helper methods to JPA test extension (#1673)
Added methods that exist in AppEngineExtension that preload some canned
data. This data is loaded by default and a lot of tests rely on them. As
we migrate away from App Engine, it is helpful to have them in the JPA
test extension which will replace the app engine extension that is
used to set up the database in dual database tests.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1673)
<!-- Reviewable:end -->
2022-06-21 16:53:56 -04:00
gbrodman
bb9b669014 Convert GenerateZoneFilesAction to SQL (#1668)
I'm not 100% sure that this is strictly necessary, but for now we can
replicate the ability to generate zonefiles for any point in time in the
recent past.
2022-06-21 15:51:52 -04:00
gbrodman
dddeb48356 Ignore buggy gpg2 tests (#1675)
See https://b.corp.google.com/issues/236723363 for more info. We're not
sure why these are failing in Kokoro.
2022-06-21 14:58:24 -04:00
Lai Jiang
c878679770 Migrate ReadDnsQueueAction to use CloudTasksUtils (#1669)
* Migrate ReadDnsQueueAction to use CloudTasksUtils

Also marked TaskQueueUtils as deprecated and fixed a few linter errors.

Note that DNS pull queue still requires the use of the GAE Task Queue API.

* Fix a test failure

* Remove TaskQueueUtils from VKeyTest

* Remove the @error exception that was inadvertently pulled in
2022-06-15 13:48:28 -04:00
gbrodman
2f8be045c7 Delete code relating to SQL init and scheduling (#1661)
One of the more significant changes introduced in this PR is that we use
SQL as the backing database in all tests unless otherwise specified,
e.g. by using the TmOverrideExtension. We change various ofy-related
tests to use this.

This includes various changes:
- Deletion of SqlEntity/DatastoreEntity and related classes. Includes
  any necessary changes because of that (e.g. getting a nice SQL key on
  error in RegistryJpaIO).
- Deletion of classes that used libraries from the init-sql code
  (RefreshDnsOnHostRenameAction)
- Removal of the JpaTransactionManager's backup implementation
- Modification of RegistryJpaWriteTest to not use init-sql code
- Removal of the Transaction class and related classes, however it does
  not remove the TransactionEntity class as that would require DB
  changes
- Removal of anything related to the actual usage of the database
  migration schedule or read-only phases
- Various test changes and fixes to account for the differences in SQL
  (like how foreign keys need to exist)

This deliberately doesn't do anything to alter the objects actually
stored in the DB yet, just how we use them
2022-06-13 15:10:35 -04:00
gbrodman
dcc11379c8 Remove TM references from GaeUserIdConverter (#1666)
This is the only user of the ofy code that will stick around at least
until we move to the new registrar console. By removing references to
the transaction manager, we will be able to delete all the tm code
without interfering with this.
2022-06-13 11:47:39 -04:00
Weimin Yu
6ff4aaeb1c Remove version pin for java-diff-utils dependency (#1667)
* Remove version pin for java-diff-utils dependency

Latest version of the lib introduces a small behavior change/bug fix.
It no longer ignores empty lines. This actually makes sense.

Update the test data to reflect this change.
2022-06-13 10:35:51 -04:00
Lai Jiang
199d02a4cc Fix EPP update NS delete bug (#1665) 2022-06-10 12:42:31 -04:00
Lai Jiang
48cc12e016 Add logging for nameserver deletion (#1663)
The main purpose of this PR is to help debug b/234189023, where a
registrar reported that in sandbox they observed seemingly successful EPP
update responses to delete NS records, which are not actually deleted after
the commands executed.

To actually load the persisted domain resource after an update would
require us to execute another transaction immediately after the update
transaction and that can only be achieved outside the flow (i. e. in
FlowRunner or EppController) and we need to test for the type of flows
before logging, which seems unnecessarily complex.

For now we are just adding logs inside the update transaction itself to
validate that:

1. The NS records to delete are as expected.
2. The Current NS records are as expected.
3. The new NS records to persist are as expected.

The EPP success reply is the default reply when no errors are thrown in
a transaction. If we see a success reply (which means that the
transaction finished successfully) and expected logs from the transaction, the
only explanation could be that somewhere in the ORM layer the java
representation of what the entity is is different from what is being
presented to the database. I think that signals a much bigger and
fundamental problem, which is quite unlikely given how isolated the
issue under consideration is.

In any case we would like to add the logging functionality in sandbox and ask
the registrar to report again when they see similar issues.

Also made some typo and linting fixes.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1663)
<!-- Reviewable:end -->
2022-06-09 16:56:58 -04:00
gbrodman
e30b3f9e0b Fix some small transactional issues in SQL mode (#1662)
* Fix some small transactional issues in SQL mode

These weren't caught until I switched the default database type in tests
to be SQL (separate PR). Fortunately these don't seem to be catastrophic
2022-06-09 15:01:34 -04:00
gbrodman
623356b1e8 Remove functional SQL<->DS replay code (#1659)
This includes:
- removing the actions that do the replay
- removing the tests for the replay
- removing the ReplayExtension and adjusting the various tests that used
  it appropriately
- removing functionality relating to "things that happen during replay",
  e.g. beforeSqlSaveOnReplay

This does not include:
- removing the InitSqlPipeline or similar tasks
- removing e.g. SqlEntity (it's used in other places)
- removing Transforms/RegistryJpaIO and other SQL-pipeline-creation code
2022-06-09 07:44:01 -04:00
Weimin Yu
4f69e1e0a6 Remove bracket in Cloud Build script (#1658)
* Remove bracket around varname in CloudBuild script

Due to spinnaker restriction: it cannot handle variable references where the var name has brackets around it.

Added spinnaker error message to the comments
2022-06-08 13:58:56 -04:00
gbrodman
9912e35ea2 Remove Ofy code from various flow-related classes (#1653)
This included removing ofy-specific code from various tests. Also, some
of the other tests (e.g. RdapDomainActionTest) had to be configured to
use only SQL -- otherwise, as it currently stands, they were trying to
use ofy.

We also delete the CreateSyntheticHistoryEntriesAction and pipeline
because they're no longer relevant, and impossible to test (the goal of
the actions were to create objects in ofy which doesn't happen any
more).
2022-06-07 11:43:33 -04:00
gbrodman
413fd1dc94 Update bigdataoss:util dependency (#1654)
We're running into issues pulling 2.1.3 from maven, possibly due to
vulnerabilities in dependencies, so this updates it to the most recent
version of 2.2.6.
2022-06-06 13:28:49 -04:00
Ben McIlwain
2377774bf9 Add a new recurrenceLastExpansion column to the BillingRecurrence table (#1650)
* Add a new recurrenceLastExpansion column to the BillingRecurrence table

This will be used to determine which recurrences are in scope for
expansion. Anything that has already been expanded within the past year is
automatically out of scope.

Note that the default value is set to just over a year ago, so that, initially,
everything is in scope for expansion, and then will gradually be winnowed down
over time so that most recurrences at any given point are out of scope. Newly
created recurrings (after the subsequent code change goes in) will have their
last expansion time set to the same as the event time of when the recurring is
written, such that they'll first be considered in-scope precisely one year
later.
2022-06-01 14:23:56 -04:00
Weimin Yu
857cb833a5 Summarize schema related tests (#1647)
* Summarize schema related tests

Document existing schema-related tests including presubmit tests and
the schema-verify predeployment test newly added to Spinnaker.
2022-05-31 11:03:58 -04:00
Ben McIlwain
82a50862c4 Inject a DomainPricingLogic into ExpandRecurringBillingEventsAction (#1648)
* Inject a DomainPricingLogic into ExpandRecurringBillingEventsAction

This will be used in other PRs to set the renewal price correctly based on the
renewal price behavior of the BillingRecurrence event.

Note that, in order for this to work, a not-null constraint has been lifted on
the EPP flow state when the DomainPricingCustomLogic is being constructed, as
the pricing here will occur in a backend action outside the context of any EPP
flow.
2022-05-27 11:46:36 -04:00
sarahcaseybot
56d1ea71fe Remove BillingId from schema (#1641)
* Remove BillingId from schema

* Add back java changes

* Add transient tag

* Remove java

* Remove flyway files
2022-05-26 15:28:58 -04:00
Ben McIlwain
fd3cec2a0f Slightly improve performance of ExpandRecurringBillingEventsAction (#1642)
* Slightly improve performance of ExpandRecurringBillingEventsAction

We don't need to log every single no-op batch of 50 Recurrences that are
processed (considering we have 1.5M total in our system), and we also don't need
to process Recurrences that already ended prior to the Cursor time (this gets us
down to 420k from 1.5M).
2022-05-25 17:27:21 -04:00
Michael Muller
b9c40dd68d Disable Ofy tests. (#1644)
* Disable Ofy tests.

This change just turns off the Ofy tests at the root, by removing processing
for dual tests and disassociating the TestOfyOnly annotation from test
annotations.

This is far less comprehensive than #1631, but it's probably worth entering as
a stopgap solution just because it should speed up our test runs and unblock a
lot of other cleanup work.

* Fix DualDatabaseTestInvocationContextProviderTest
2022-05-25 09:47:05 -04:00
Lai Jiang
754958ef3a Remove an unnecessary pair of parentheses. (#1645) 2022-05-24 17:37:10 -04:00
Michael Muller
1bf7c925bc Fix style warning (#1643)
THe build is giving a style warning over the extra parens.
2022-05-24 17:22:08 -04:00
Weimin Yu
eeca51667e Optimize RDAP entity event query (#1635)
* Optimize RDAP entity event query

For each EPP entity, directly load the latest HistoryEntry per event type
instead of loading all events through the HistoryEntryDao.

Although most entities have a small number of history entries, there are
a few entities with many entries, enough to cause OutOfMemory error.
2022-05-19 23:35:55 -04:00
sarahcaseybot
123d6359dc Change shouldPublish in GenerateInvoicesAction to default to false (#1640) 2022-05-19 17:51:21 -04:00
Rachel Guan
64fba55f06 Add renewal cost logic to DomainPricingLogic (#1610)
* Add renew cost calculation to DomainPricingLogic

* Fix typos and change assertions
2022-05-19 16:05:21 -04:00
gbrodman
3a7ac669f5 Set up jpaTm before loading data in the test server (#1633) 2022-05-19 12:13:43 -04:00
Michael Muller
fc029b5ad2 Added info on problematic max-instances param (#1639)
We have backend max-instances set to 100, which apparently exceeds the default
quota for GAE.  Add info on updating the quota or changing this parameter to
the configuration doc.
2022-05-19 11:51:27 -04:00
Ben McIlwain
ec5c2cdb68 Add batching to ExpandRecurringBillingEventsAction (#1636)
* Add batching to ExpandRecurringBillingEventsAction

It's OOMing on trying to load every single BillingRecurrence that needs to be
expanded simultaneously (which is to be expected). So this processes them in
transactional batches of 50.
2022-05-19 09:13:37 -04:00
Lai Jiang
c4cf128844 Make Caffeine cache loading work in non-GAE thread (#1634) 2022-05-18 22:01:44 -04:00
sarahcaseybot
c262ef82c9 Remove all uses of the billingIdentifier field (#1608)
* Remove all uses of the billingIdentifier field

* Add @ignore flag

* Add tag
2022-05-18 17:15:45 -04:00
Lai Jiang
03ca6cecc7 Add the ability to nullify the entire billing account map (#1630) 2022-05-17 10:33:42 -04:00
Weimin Yu
bee5e0a5a9 Verify schema using Cloud Build (#1627)
* Add tool to compare  golden and actual schema
2022-05-16 16:10:09 -04:00
Michael Muller
9f0138aeb2 Cleanup gpg-agent instances and home directories (#1629)
* Cleanup gpg-agent instances and home directories

The GpgSystemCommandException leaks home directories, but more importantly it
leaks gpg-agent instances.  This can cause problems with inotify limits, since
the agent seems to make use of inotify.  Do a proper cleanup in afterEach().

* Don't fail if we can't kill the agent
2022-05-16 14:06:26 -04:00
Rachel Guan
eca2b61d8b Add renewal support for AT and internal registrations in DomainCreateFlow (#1591)
* Add renewal info to create flow

* Improve PR
2022-05-13 10:04:15 -04:00
Michael Muller
d12897062b Fix new checkstyle violations (#1626)
* Fix new checkstyle violations

* Restore naming in javadocs
2022-05-11 14:23:21 -04:00
Ben McIlwain
cc46447f25 Remove cron entries for Datastore-specific jobs (#1615)
We'll delete the associated code soon enough too, but it's safer to delete the
cron jobs first and run in that state for a week, so we can still run them
manually if need be.
2022-05-09 17:52:34 -04:00
Lai Jiang
e3016bb7d8 Remove to-be-deprecated OOB OAuth flow in nomulus login (#1625) 2022-05-09 17:17:05 -04:00
Rachel Guan
cc62530345 Add renewal logic in allocation token related commands (#1596)
* Add renewal price behavior to allocation token related command

* Add details to renewal price behavior
2022-05-05 15:48:45 -04:00
Weimin Yu
dd9c576146 Update Postgres docker tag in tests (#1624)
* Update Postgres docker tag in tests
2022-05-05 14:59:19 -04:00
Weimin Yu
d361f7cf18 Tag nomulus-tool in schema deployment script (#1621)
* Tag nomulus-tool in schema deployment script
2022-05-05 12:46:32 -04:00
gbrodman
a9b0988c8f Fix compilation errors caused by Caffeine changes (#1623) 2022-05-05 11:21:10 -04:00
Ben McIlwain
08a9e2b64e Finish conversion from Guava Cache to Caffeine (#1616)
* Finish conversion from Guava Cache to Caffeine
2022-05-04 17:43:47 -04:00
Ben McIlwain
205b16fe8a Reduce the number of manually scaled instances for default/pubapi (#1620)
* Reduce the number of manually scaled instances for default/pubapi

This is in the spirit of "not always running significantly over-provisioned",
which helps to save costs and also expose potential scaling issues when they are
still small rather than all at once when they're a big problem.

This can always be reverted if necessary, and can be instantaneously adjusted by
running the `nomulus set_num_instances` command.
2022-05-04 17:43:27 -04:00
Ben McIlwain
95dfd5b76a Don't enforce billing account map check on TEST TLDs (#1622)
* Don't enforce billing account map check on TEST TLDs

This was affecting monitoring (i.e. prober TLDs). Note that test TLDs are
already excluded from the billing account map check in the Registrar builder()
method (see PR #1601), but we forgot to make that same test TLD exclusion in the
EPP flows check (see PR #1605).
2022-05-04 16:59:25 -04:00
Weimin Yu
ac3c8b7520 Add test for Java 8 Compatibility of third party jars (#1618)
* Add test for Java 8 Compatibility

Add a test to check for Java 8 compatibility of jars deployed to
AppEngine.

It is not enough to run existing tests with Java 8 VM, since many API
jars are not exercised by tests. For example, those for GCP services
like the SecretManager.

We take the conservative approach and verify that every class in every
jar are compiled for Java 8.
2022-05-04 15:44:17 -04:00
Lai Jiang
f4436b54cf Do not delete build cache when building release candidates (#1619)
We would like to re-use the build cache when building RCs for different
environments. There's not much practical use in doing a "clean" for
every build when Gradle should be able to figure out which artifacts
need to be rebuilt. It also does not make sense to build each
environment in a separate step, which also introduces redunency because
not all artifacts are cached across steps. The build cache is enabled by
default.

Lastly, the cache needs to be inside the /workspace folder, which is the
default persisted storage location.

TESTED=tried to build the RCs on alpha and saved about 10 min.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1619)
<!-- Reviewable:end -->
2022-05-04 12:08:10 -04:00
Michael Muller
05fcf73452 Add missing transaction for whois lookups (#1614)
* Add missing transaction for whois lookups

Nameserver whois lookups are failing under SQL for hosts with superordinate
domains because the query in this case is not done in a transaction.  We
missed this during testing because a) we didn't have a test for lookups of
hosts with superordinate domains and b) we missed converting
NameserverWhoisResponseTest to a DualDatabaseTest.

This PR fixes the problem and adds the requisite testing.

* Use a single transaction to get host registrars

* Replace streaming with Maps.toMap()
2022-05-04 07:29:45 -04:00
Weimin Yu
e24dba7d2b Downgrade dependencies that no longer support Java8 (#1617)
* Downgrade dependencies that no longer support Java8

Downgrade two dependencies whose latest versions no longer support
java8.

A follow up PR will add java8 compatibility to presubmit tests.
2022-05-04 02:03:34 -04:00
Lai Jiang
4ec8b71f42 Increase Nomulus build timeout (#1613)
We have recently started to routinely breach the 1h timeout. Increasing
this value to 2h. We should also look into reusing the artifacts when
building RCs for different environments.
2022-05-02 16:11:11 -04:00
Weimin Yu
8e8911870c Use Gradle dependency dynamic versioning (#1612)
* Use Gradle dependency dynamic versioning

Use dynamic versioning for Gradle dependencies when possible.
Please refer to go/dr-dependency-upgrade for more information about the
automation plan.

This PR calls out all dependencies that must be pinned to specific
versions for various reasons. The remaining ones are converted to
open-ended version ranges ("[version_str,)").
2022-05-02 14:10:52 -04:00
Ben McIlwain
fa135dcd85 Re-enable prober data deletion cron jobs in prod & sandbox (#1611)
This reverts commit 52c759d1db.
2022-05-02 13:46:02 -04:00
sarahcaseybot
5d82893478 Check PAK is present on billing domain flows (#1605)
* Check PAK on domain create

* Add unit test

* update docs

* Remove unneccesary setup

* Fix blank line

* Add check and test to all relevant flows

* Change error message
2022-04-29 14:30:24 -04:00
Rachel Guan
d0af81ecdf Change from assertThat(assertThrow()) to thrown = assertThrows() then assertThat(thrown) (#1606) 2022-04-28 16:09:36 -04:00
Michael Muller
f273783894 Ignore version UIDs during txn deserialization (#1607)
* Ignore version UIDs during txn deserialization

When deserializing transactions for replay to datastore, ignore class version
UIDs that don't match those of the local classes and just use the local class
descriptors instead.  This is a simple solution for the problem of persisted
VKeys containing references to classes where the class has been updated and
the serial version UID has changed.

Also add a "replay_txns" command that replays the transactions from a given
start point so we can verify all transactions are deserializable.

TESTED:
    Ran replay_txns against all transactions on sandbox beginning with
    transaction id 1828385, which includes Recurring billing events containing
    both the old and the new serial version UIDs.
2022-04-27 15:40:27 -04:00
Ben McIlwain
0dfabe1c64 Convert more Guava caches to Caffeine (#1603)
* Convert more Guava caches to Caffeine
2022-04-26 11:26:51 -04:00
Weimin Yu
60a011c593 Remove stray file that slipped in the repo (#1604)
* Remove stray file that slipped in the repo
2022-04-25 17:08:58 -04:00
Michael Muller
7716eebfff Change check for root directory during rollback (#1602)
* Change check for root directory during rollback

`rollback_tool` tries to infer the root of the nomulus tree by checking for a
directory named "nomulus".  This is potentially problematic (and, indeed, was
for me) since there is no guarantee what that directory will be named.

There are a number of features that characterize the root directory.  Check
for the presence of the `rollback_tool` wrapper script, as this is both at
root level and tightly coupled to the python code, so hopefully we won't
move it without testing that the script still works.
2022-04-25 12:39:16 -04:00
Ben McIlwain
1e76eeed37 Validate that a registrar has billing accounts for all its allowed TLDs (#1601)
This will require edits to a substantial number of registrars on sandbox (nearly
all of them) because almost all of them have access to at least one TLD, but
almost none of them have any billing accounts set. Until this is set, any updates
to the existing registrars that aren't adding the billing accounts will cause
failures.

Unfortunately, there wasn't any less invasive foolproof way to implement this
change, and we already had one attempt to implement it on create registrar
command that wasn't working (because allowed TLDs tend not to be added on
initial registrar creation, but rather, afterwards as an update).
2022-04-22 16:33:18 -04:00
sarahcaseybot
147d133aef Don't fail invoicing on missing PAK (#1595)
* Don't fail invoicing on missing PAK

* Skip line if missing PAK

* Add log check in test
2022-04-22 13:00:50 -04:00
Ben McIlwain
c2e1f2e640 Downgrade Caffeine to 2.9.3 (#1600)
* Downgrade Caffeine to 2.9.3

Apparently Caffeine >=3.* requires Java 11, and we're still stuck on Java 8
because of App Engine Standard.  Fortunately this doesn't affect the exposed
interface we're using, so we can simply go back to the newest Caffeine version
once Registry 3.0 Phase 3 (GKE migration) is completed.
2022-04-20 14:05:37 -04:00
Lai Jiang
5d2639834a Remove the BEAM RDE pipeline side job (#1599)
Now that SQL is the default, we do not need this side job to run
alongside the main one. Its purpose was to validate the BEAM pipeline
while Datastore was primary.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1599)
<!-- Reviewable:end -->
2022-04-20 12:12:23 -04:00
Rachel Guan
7912576e3d Add rachelguan@ to CONTRIBUTORS (#1598) 2022-04-19 19:18:44 -04:00
Rachel Guan
8424c85258 Fix build warning for unused variable (#1594) 2022-04-19 14:11:23 -04:00
Lai Jiang
e72dd73ed8 Fix build (#1597) 2022-04-19 08:27:06 -04:00
Rachel Guan
508d221b94 Replace .get()).isEqualTo() with hasValue() (#1589)
* Replace .get()).isEqualTo() with hasValue()

* Use containsExactly for list comparison

* Fix spacing
2022-04-18 18:35:46 -04:00
gbrodman
073d0a416a Create a Dataflow pipeline to resave EPP resources (#1553)
* Create a Dataflow pipeline to resave EPP resources

This has two modes.

If `fast` is false, then we will just load all EPP resources, project them to the current time, and save them.

If `fast` is true, we will attempt to intelligently load and save only resources that we expect to have changes applied when we project them to the current time. This means resources with pending transfers that have expired, domains with expired grace periods, and non-deleted domains that have expired (we expect that they autorenewed).
2022-04-15 15:46:35 -04:00
Rachel Guan
f2ead5a0e3 Make code change for AllocationToken schema change (#1581)
* Make code change for AllocationToken schema change
2022-04-15 15:28:39 -04:00
Lai Jiang
212dbbe520 Fix a in issue with RDE (#1593)
For some inexplicable reason, the RDE beam pipeline in both sandbox and
production has been broken for the past week or so. Our investigations
revealed that during the CoGropuByKey stage, some repo ID -> revision ID
pairs were duplicated. This may be a problem with the Dataflow runtime
which somehow introduced the duplicate during reshuffling.

This PR attempts to fix the symptom only by deduping the revision IDs. We
will do some more investigation and possibly follow up with the Dataflow
team if we determine it is an upstream issue.

TESTED=deployed the pipeline and successfully run sandbox RDE with it.
2022-04-15 10:32:44 -04:00
Ben McIlwain
8594a61fd4 Begin migration from Guava Cache to Caffeine (#1590)
* Begin migration from Guava Cache to Caffeine

Caffeine is apparently strictly superior to the older Guava Cache (and is even
recommended in lieu of Guava Cache on Guava Cache's own documentation).

This adds the relevant dependencies and switch over just a single call site to
use the new Caffeine cache. It also implements a new pattern, asynchronously
refreshing the cache value starting from half of our configuration time. For
frequently accessed entities this will allow us to NEVER block on a load, as it
will be asynchronously refreshed in the background long before it ever expires
synchronously during a read operation.
2022-04-14 13:38:53 -04:00
Rachel Guan
36837eb3e6 Change to hasSize() for assertions (#1588)
* Change to hasSize() for assertions
2022-04-13 18:20:34 -04:00
Rachel Guan
3a9a8c6557 Add new columns to BillingEvent (#1573)
* Add new columns to BillingEvent.java

* Improve PR and modifyJodaMoneyType to handle null currency in override

* Add test cases for edge cases of nullSafeGet in JodaMoneyType

* Improve assertions
2022-04-11 20:09:26 -04:00
Weimin Yu
65c2570b8f Remove dos.xml from the configs (#1587)
* Remove dos.xml from the configs

We don't have dos config right now, and applying dos from "gcloud app
deploy" is deprecated and has started causing problems.

If we add dos configs, it should be using "gcloud app firewall-rules".
2022-04-11 15:22:42 -04:00
Weimin Yu
86acaa1b31 Build Java8-compatible release (#1586)
* Build Java8-compatible release

Use the new options.release Gradle property to make sure builds are
compatible with Java 8, which is the runtime on Appengine.

This new property replaces sourceCompatibility, targetCompatibility, and
bootclasspath (wasn't previously set, which is the reason why we
couldn't detect Java9 api usage when building).
2022-04-11 11:00:00 -04:00
Weimin Yu
436cc03be9 Remove Optional.isEmpty() in code (#1585)
* Remove Optional.isEmpty() in code
2022-04-08 21:30:22 +00:00
Ben McIlwain
e110ddd412 Canonicalize domain/host names in nomulus tool commands (#1583)
* Canonicalize domain/host names in nomulus tool commands

This helps prevent some common user errors.
2022-04-06 18:35:38 -04:00
Michael Muller
214b23e99c Ignore read-only when saving commit logs (#1584)
* Ignore read-only when saving commit logs

Ignore read-only when saving commit logs and commit log mutations so that we
can safely replicate in read-only mode.  This should be safe, as we only ever
to the situation of saving commit logs and mutations when something has
already actually been modified in a transaction, meaning that we should have hit
the "read only" sentinel already.

This also introduces the ability to set the Clock in the
TransactionManagerFactory so that we can test this functionality.

* Changes per review

* Fix issues affecting tests

- Restore clobbered async phase in testNoInMigrationState_doesNothing
- Restore system clock to TransactionManagerFactory to avoid affecting other
  tests.
2022-04-06 13:06:08 -04:00
Rachel Guan
743dea9ca2 Add renewal price behavior to AllocationToken (#1580) 2022-04-04 18:51:49 -04:00
sarahcaseybot
41f9f1ef7d Change use of BillingIdentifier to BillingAccountMap in invoicing pipeline (#1577)
* Change billingIdentifier to BillingAccountMap in invoicing pipeline

* Add a default for billing account map

* Throw error on missing PAK

* Add unit test
2022-04-04 16:16:43 -04:00
Michael Muller
44ede2b022 Check for error suggesting another nomulus running (#1582)
Check for a PSQLException referencing a failed connection to "google:5433",
which likely indicates that there is another nomulus tool instance running.

It's worth giving this hint because in cases like this it's not at all obvious
that the other instance of nomulus is problematic.
2022-04-04 11:14:43 -04:00
Ben McIlwain
e4312322dc Add a no-async actions DB migration phase (#1579)
* Add a no-async actions DB migration phase

This needs to be set several hours prior to entering the READONLY stage. This is
not a read-only stage; all synchronous actions under Datastore (such as domain
creates) will continue to succeed. The only thing that will fail is host
deletes, host renames, and contact deletes, as these three actions require a
mapreduce to run before they are complete, and we don't want mapreduces hanging
around and executing during what is supposed to be a short duration READONLY
period.
2022-04-01 16:55:51 -04:00
991 changed files with 46404 additions and 81316 deletions

View File

@@ -36,3 +36,4 @@ Shicong Huang <shicong@google.com>
Gustav Brodman <gbrodman@google.com>
Sarah Botwinick <sarahbot@google.com>
Legina Chen <legina@google.com>
Rachel Guan <rachelguan@google.com>

View File

@@ -292,7 +292,8 @@ subprojects {
afterEvaluate {
if (rootProject.enableDependencyLocking.toBoolean()
&& project.name != 'integration') {
&& project.name != 'integration'
&& project.name != 'java8compatibility') {
// The ':integration' project runs server/schema integration tests using
// dynamically specified jars with no transitive dependency. Therefore
// dependency-locking does not make sense. Furthermore, during
@@ -300,6 +301,9 @@ subprojects {
// immutable. Locking activation would trigger an invalid operation
// exception.
//
// The ':java8compatibility' project is test-only. Its source does not go
// into production.
//
// For all other projects, due to problem with the gradle-license-report
// plugin, the dependencyLicenseReport configuration must opt out of
// dependency-locking. See dependency_lic.gradle for the reason why.
@@ -331,6 +335,13 @@ subprojects {
apply from: "${rootDir.path}/java_common.gradle"
if (project.name != 'docs') {
compileJava {
// TODO: Remove this once we migrate off AppEngine.
options.release = 8
}
}
if (project.name == 'third_party') return
project.tasks.test.dependsOn runPresubmits

View File

@@ -3,7 +3,7 @@
# This file is expected to be part of source control.
com.github.ben-manes.caffeine:caffeine:2.7.0
com.github.kevinstern:software-and-algorithms:1.0
com.google.auto.value:auto-value:1.7.4
com.google.auto.value:auto-value:1.9
com.google.auto:auto-common:0.10
com.google.code.findbugs:jFormatString:3.0.0
com.google.code.findbugs:jsr305:3.0.2

View File

@@ -3,61 +3,60 @@
# This file is expected to be part of source control.
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
com.fasterxml.jackson.core:jackson-core:2.12.1
com.fasterxml.jackson:jackson-bom:2.12.1
com.google.api-client:google-api-client:1.31.2
com.google.api.grpc:proto-google-common-protos:2.1.0
com.google.api.grpc:proto-google-iam-v1:1.0.10
com.google.api:api-common:1.10.1
com.google.api:gax-httpjson:0.79.0
com.google.api:gax:1.62.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.31.0
com.google.auth:google-auth-library-credentials:0.24.1
com.google.auth:google-auth-library-oauth2-http:0.24.1
com.google.auto.value:auto-value-annotations:1.7.4
com.google.cloud:google-cloud-core-http:1.94.1
com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.google.api-client:google-api-client:1.35.1
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.value:auto-value-annotations:1.9
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-storage:2.9.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
com.google.http-client:google-http-client-jackson2:1.39.0
com.google.http-client:google-http-client:1.39.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.template:soy:2021-02-01
com.ibm.icu:icu4j:57.1
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
io.grpc:grpc-context:1.36.0
io.opencensus:opencensus-api:0.28.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-context:1.47.0
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-http-util:0.31.1
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.commons:commons-lang3:3.11
org.apache.commons:commons-text:1.9
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.14
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.8.0
org.apache.httpcomponents:httpcore:4.4.15
org.checkerframework:checker-qual:3.22.2
org.json:json:20160212
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.threeten:threetenbp:1.5.0
org.threeten:threetenbp:1.6.0

View File

@@ -3,61 +3,60 @@
# This file is expected to be part of source control.
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
com.fasterxml.jackson.core:jackson-core:2.12.1
com.fasterxml.jackson:jackson-bom:2.12.1
com.google.api-client:google-api-client:1.31.2
com.google.api.grpc:proto-google-common-protos:2.1.0
com.google.api.grpc:proto-google-iam-v1:1.0.10
com.google.api:api-common:1.10.1
com.google.api:gax-httpjson:0.79.0
com.google.api:gax:1.62.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.31.0
com.google.auth:google-auth-library-credentials:0.24.1
com.google.auth:google-auth-library-oauth2-http:0.24.1
com.google.auto.value:auto-value-annotations:1.7.4
com.google.cloud:google-cloud-core-http:1.94.1
com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.google.api-client:google-api-client:1.35.1
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.value:auto-value-annotations:1.9
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-storage:2.9.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
com.google.http-client:google-http-client-jackson2:1.39.0
com.google.http-client:google-http-client:1.39.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.template:soy:2021-02-01
com.ibm.icu:icu4j:57.1
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
io.grpc:grpc-context:1.36.0
io.opencensus:opencensus-api:0.28.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-context:1.47.0
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-http-util:0.31.1
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.commons:commons-lang3:3.11
org.apache.commons:commons-text:1.9
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.14
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.8.0
org.apache.httpcomponents:httpcore:4.4.15
org.checkerframework:checker-qual:3.22.2
org.json:json:20160212
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.threeten:threetenbp:1.5.0
org.threeten:threetenbp:1.6.0

View File

@@ -3,61 +3,60 @@
# This file is expected to be part of source control.
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
com.fasterxml.jackson.core:jackson-core:2.12.1
com.fasterxml.jackson:jackson-bom:2.12.1
com.google.api-client:google-api-client:1.31.2
com.google.api.grpc:proto-google-common-protos:2.1.0
com.google.api.grpc:proto-google-iam-v1:1.0.10
com.google.api:api-common:1.10.1
com.google.api:gax-httpjson:0.79.0
com.google.api:gax:1.62.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.31.0
com.google.auth:google-auth-library-credentials:0.24.1
com.google.auth:google-auth-library-oauth2-http:0.24.1
com.google.auto.value:auto-value-annotations:1.7.4
com.google.cloud:google-cloud-core-http:1.94.1
com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.google.api-client:google-api-client:1.35.1
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.value:auto-value-annotations:1.9
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-storage:2.9.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
com.google.http-client:google-http-client-jackson2:1.39.0
com.google.http-client:google-http-client:1.39.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.template:soy:2021-02-01
com.ibm.icu:icu4j:57.1
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
io.grpc:grpc-context:1.36.0
io.opencensus:opencensus-api:0.28.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-context:1.47.0
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-http-util:0.31.1
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.commons:commons-lang3:3.11
org.apache.commons:commons-text:1.9
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.14
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.8.0
org.apache.httpcomponents:httpcore:4.4.15
org.checkerframework:checker-qual:3.22.2
org.json:json:20160212
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.threeten:threetenbp:1.5.0
org.threeten:threetenbp:1.6.0

View File

@@ -3,61 +3,60 @@
# This file is expected to be part of source control.
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
com.fasterxml.jackson.core:jackson-core:2.12.1
com.fasterxml.jackson:jackson-bom:2.12.1
com.google.api-client:google-api-client:1.31.2
com.google.api.grpc:proto-google-common-protos:2.1.0
com.google.api.grpc:proto-google-iam-v1:1.0.10
com.google.api:api-common:1.10.1
com.google.api:gax-httpjson:0.79.0
com.google.api:gax:1.62.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.31.0
com.google.auth:google-auth-library-credentials:0.24.1
com.google.auth:google-auth-library-oauth2-http:0.24.1
com.google.auto.value:auto-value-annotations:1.7.4
com.google.cloud:google-cloud-core-http:1.94.1
com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.google.api-client:google-api-client:1.35.1
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.value:auto-value-annotations:1.9
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-storage:2.9.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
com.google.http-client:google-http-client-jackson2:1.39.0
com.google.http-client:google-http-client:1.39.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.template:soy:2021-02-01
com.ibm.icu:icu4j:57.1
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
io.grpc:grpc-context:1.36.0
io.opencensus:opencensus-api:0.28.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-context:1.47.0
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-http-util:0.31.1
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.commons:commons-lang3:3.11
org.apache.commons:commons-text:1.9
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.14
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.8.0
org.apache.httpcomponents:httpcore:4.4.15
org.checkerframework:checker-qual:3.22.2
org.json:json:20160212
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.threeten:threetenbp:1.5.0
org.threeten:threetenbp:1.6.0

View File

@@ -3,61 +3,60 @@
# This file is expected to be part of source control.
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
com.fasterxml.jackson.core:jackson-core:2.12.1
com.fasterxml.jackson:jackson-bom:2.12.1
com.google.api-client:google-api-client:1.31.2
com.google.api.grpc:proto-google-common-protos:2.1.0
com.google.api.grpc:proto-google-iam-v1:1.0.10
com.google.api:api-common:1.10.1
com.google.api:gax-httpjson:0.79.0
com.google.api:gax:1.62.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.31.0
com.google.auth:google-auth-library-credentials:0.24.1
com.google.auth:google-auth-library-oauth2-http:0.24.1
com.google.auto.value:auto-value-annotations:1.7.4
com.google.cloud:google-cloud-core-http:1.94.1
com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.google.api-client:google-api-client:1.35.1
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.value:auto-value-annotations:1.9
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-storage:2.9.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
com.google.http-client:google-http-client-jackson2:1.39.0
com.google.http-client:google-http-client:1.39.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.template:soy:2021-02-01
com.ibm.icu:icu4j:57.1
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
io.grpc:grpc-context:1.36.0
io.opencensus:opencensus-api:0.28.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-context:1.47.0
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-http-util:0.31.1
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.commons:commons-lang3:3.11
org.apache.commons:commons-text:1.9
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.14
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.8.0
org.apache.httpcomponents:httpcore:4.4.15
org.checkerframework:checker-qual:3.22.2
org.json:json:20160212
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:7.0
org.threeten:threetenbp:1.5.0
org.threeten:threetenbp:1.6.0

View File

@@ -3,76 +3,74 @@
# This file is expected to be part of source control.
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
com.fasterxml.jackson.core:jackson-core:2.12.1
com.fasterxml.jackson:jackson-bom:2.12.1
com.google.api-client:google-api-client:1.31.2
com.google.api.grpc:proto-google-common-protos:2.1.0
com.google.api.grpc:proto-google-iam-v1:1.0.10
com.google.api:api-common:1.10.1
com.google.api:gax-httpjson:0.79.0
com.google.api:gax:1.62.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.31.0
com.google.auth:google-auth-library-credentials:0.24.1
com.google.auth:google-auth-library-oauth2-http:0.24.1
com.google.auto.value:auto-value-annotations:1.7.4
com.google.cloud:google-cloud-core-http:1.94.1
com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.google.api-client:google-api-client:1.35.1
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.value:auto-value-annotations:1.9
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-storage:2.9.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
com.google.http-client:google-http-client-jackson2:1.39.0
com.google.http-client:google-http-client:1.39.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.google.truth.extensions:truth-java8-extension:1.1.3
com.google.truth:truth:1.1.3
com.ibm.icu:icu4j:57.1
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
io.grpc:grpc-context:1.36.0
io.opencensus:opencensus-api:0.28.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-context:1.47.0
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-http-util:0.31.1
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
junit:junit:4.13.1
net.bytebuddy:byte-buddy-agent:1.10.19
net.bytebuddy:byte-buddy:1.10.19
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
junit:junit:4.13.2
net.bytebuddy:byte-buddy-agent:1.12.10
net.bytebuddy:byte-buddy:1.12.10
org.apache.commons:commons-lang3:3.11
org.apache.commons:commons-text:1.9
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.14
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.9.1
org.apache.httpcomponents:httpcore:4.4.15
org.checkerframework:checker-qual:3.22.2
org.hamcrest:hamcrest-core:1.3
org.json:json:20160212
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.mockito:mockito-core:3.7.7
org.objenesis:objenesis:3.1
org.junit.jupiter:junit-jupiter-api:5.9.0-RC1
org.junit.jupiter:junit-jupiter-engine:5.9.0-RC1
org.junit.platform:junit-platform-commons:1.9.0-RC1
org.junit.platform:junit-platform-engine:1.9.0-RC1
org.junit:junit-bom:5.9.0-RC1
org.mockito:mockito-core:4.6.1
org.objenesis:objenesis:3.2
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:9.0
org.threeten:threetenbp:1.5.0
org.ow2.asm:asm:9.1
org.threeten:threetenbp:1.6.0

View File

@@ -3,76 +3,74 @@
# This file is expected to be part of source control.
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
com.fasterxml.jackson.core:jackson-core:2.12.1
com.fasterxml.jackson:jackson-bom:2.12.1
com.google.api-client:google-api-client:1.31.2
com.google.api.grpc:proto-google-common-protos:2.1.0
com.google.api.grpc:proto-google-iam-v1:1.0.10
com.google.api:api-common:1.10.1
com.google.api:gax-httpjson:0.79.0
com.google.api:gax:1.62.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.31.0
com.google.auth:google-auth-library-credentials:0.24.1
com.google.auth:google-auth-library-oauth2-http:0.24.1
com.google.auto.value:auto-value-annotations:1.7.4
com.google.cloud:google-cloud-core-http:1.94.1
com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.google.api-client:google-api-client:1.35.1
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.value:auto-value-annotations:1.9
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-storage:2.9.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
com.google.http-client:google-http-client-jackson2:1.39.0
com.google.http-client:google-http-client:1.39.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.google.truth.extensions:truth-java8-extension:1.1.3
com.google.truth:truth:1.1.3
com.ibm.icu:icu4j:57.1
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
io.grpc:grpc-context:1.36.0
io.opencensus:opencensus-api:0.28.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-context:1.47.0
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-http-util:0.31.1
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
junit:junit:4.13.1
net.bytebuddy:byte-buddy-agent:1.10.19
net.bytebuddy:byte-buddy:1.10.19
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
junit:junit:4.13.2
net.bytebuddy:byte-buddy-agent:1.12.10
net.bytebuddy:byte-buddy:1.12.10
org.apache.commons:commons-lang3:3.11
org.apache.commons:commons-text:1.9
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.14
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.9.1
org.apache.httpcomponents:httpcore:4.4.15
org.apiguardian:apiguardian-api:1.1.2
org.checkerframework:checker-qual:3.22.2
org.hamcrest:hamcrest-core:1.3
org.json:json:20160212
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.mockito:mockito-core:3.7.7
org.objenesis:objenesis:3.1
org.junit.jupiter:junit-jupiter-api:5.9.0-RC1
org.junit.jupiter:junit-jupiter-engine:5.9.0-RC1
org.junit.platform:junit-platform-commons:1.9.0-RC1
org.junit.platform:junit-platform-engine:1.9.0-RC1
org.junit:junit-bom:5.9.0-RC1
org.mockito:mockito-core:4.6.1
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:9.0
org.threeten:threetenbp:1.5.0
org.ow2.asm:asm:9.1
org.threeten:threetenbp:1.6.0

View File

@@ -3,76 +3,74 @@
# This file is expected to be part of source control.
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
com.fasterxml.jackson.core:jackson-core:2.12.1
com.fasterxml.jackson:jackson-bom:2.12.1
com.google.api-client:google-api-client:1.31.2
com.google.api.grpc:proto-google-common-protos:2.1.0
com.google.api.grpc:proto-google-iam-v1:1.0.10
com.google.api:api-common:1.10.1
com.google.api:gax-httpjson:0.79.0
com.google.api:gax:1.62.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.31.0
com.google.auth:google-auth-library-credentials:0.24.1
com.google.auth:google-auth-library-oauth2-http:0.24.1
com.google.auto.value:auto-value-annotations:1.7.4
com.google.cloud:google-cloud-core-http:1.94.1
com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.google.api-client:google-api-client:1.35.1
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.value:auto-value-annotations:1.9
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-storage:2.9.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
com.google.http-client:google-http-client-jackson2:1.39.0
com.google.http-client:google-http-client:1.39.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.google.truth.extensions:truth-java8-extension:1.1.3
com.google.truth:truth:1.1.3
com.ibm.icu:icu4j:57.1
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
io.grpc:grpc-context:1.36.0
io.opencensus:opencensus-api:0.28.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-context:1.47.0
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-http-util:0.31.1
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
junit:junit:4.13.1
net.bytebuddy:byte-buddy-agent:1.10.19
net.bytebuddy:byte-buddy:1.10.19
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
junit:junit:4.13.2
net.bytebuddy:byte-buddy-agent:1.12.10
net.bytebuddy:byte-buddy:1.12.10
org.apache.commons:commons-lang3:3.11
org.apache.commons:commons-text:1.9
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.14
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.9.1
org.apache.httpcomponents:httpcore:4.4.15
org.checkerframework:checker-qual:3.22.2
org.hamcrest:hamcrest-core:1.3
org.json:json:20160212
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.mockito:mockito-core:3.7.7
org.objenesis:objenesis:3.1
org.junit.jupiter:junit-jupiter-api:5.9.0-RC1
org.junit.jupiter:junit-jupiter-engine:5.9.0-RC1
org.junit.platform:junit-platform-commons:1.9.0-RC1
org.junit.platform:junit-platform-engine:1.9.0-RC1
org.junit:junit-bom:5.9.0-RC1
org.mockito:mockito-core:4.6.1
org.objenesis:objenesis:3.2
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:9.0
org.threeten:threetenbp:1.5.0
org.ow2.asm:asm:9.1
org.threeten:threetenbp:1.6.0

View File

@@ -3,76 +3,74 @@
# This file is expected to be part of source control.
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
com.fasterxml.jackson.core:jackson-core:2.12.1
com.fasterxml.jackson:jackson-bom:2.12.1
com.google.api-client:google-api-client:1.31.2
com.google.api.grpc:proto-google-common-protos:2.1.0
com.google.api.grpc:proto-google-iam-v1:1.0.10
com.google.api:api-common:1.10.1
com.google.api:gax-httpjson:0.79.0
com.google.api:gax:1.62.0
com.google.apis:google-api-services-storage:v1-rev20210127-1.31.0
com.google.auth:google-auth-library-credentials:0.24.1
com.google.auth:google-auth-library-oauth2-http:0.24.1
com.google.auto.value:auto-value-annotations:1.7.4
com.google.cloud:google-cloud-core-http:1.94.1
com.google.cloud:google-cloud-core:1.94.3
com.google.cloud:google-cloud-storage:1.113.12
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.google.api-client:google-api-client:1.35.1
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.value:auto-value-annotations:1.9
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-storage:2.9.2
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.6
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.http-client:google-http-client-apache-v2:1.39.0
com.google.http-client:google-http-client-appengine:1.39.0
com.google.http-client:google-http-client-gson:1.39.0
com.google.http-client:google-http-client-jackson2:1.39.0
com.google.http-client:google-http-client:1.39.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.oauth-client:google-oauth-client:1.31.4
com.google.protobuf:protobuf-java-util:3.15.3
com.google.protobuf:protobuf-java:3.15.3
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.google.truth.extensions:truth-java8-extension:1.1.3
com.google.truth:truth:1.1.3
com.ibm.icu:icu4j:57.1
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
io.grpc:grpc-context:1.36.0
io.opencensus:opencensus-api:0.28.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-context:1.47.0
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-http-util:0.31.1
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
junit:junit:4.13.1
net.bytebuddy:byte-buddy-agent:1.10.19
net.bytebuddy:byte-buddy:1.10.19
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
junit:junit:4.13.2
net.bytebuddy:byte-buddy-agent:1.12.10
net.bytebuddy:byte-buddy:1.12.10
org.apache.commons:commons-lang3:3.11
org.apache.commons:commons-text:1.9
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.14
org.apiguardian:apiguardian-api:1.1.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.9.1
org.apache.httpcomponents:httpcore:4.4.15
org.checkerframework:checker-qual:3.22.2
org.hamcrest:hamcrest-core:1.3
org.json:json:20160212
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.mockito:mockito-core:3.7.7
org.objenesis:objenesis:3.1
org.junit.jupiter:junit-jupiter-api:5.9.0-RC1
org.junit.jupiter:junit-jupiter-engine:5.9.0-RC1
org.junit.platform:junit-platform-commons:1.9.0-RC1
org.junit.platform:junit-platform-engine:1.9.0-RC1
org.junit:junit-bom:5.9.0-RC1
org.mockito:mockito-core:4.6.1
org.objenesis:objenesis:3.2
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:7.0
org.ow2.asm:asm-commons:7.0
org.ow2.asm:asm-tree:7.0
org.ow2.asm:asm-util:7.0
org.ow2.asm:asm:9.0
org.threeten:threetenbp:1.5.0
org.ow2.asm:asm:9.1
org.threeten:threetenbp:1.6.0

View File

@@ -51,6 +51,7 @@ artifacts {
dependencies {
def deps = rootProject.dependencyMap
compile deps['com.github.ben-manes.caffeine:caffeine']
compile deps['com.google.code.findbugs:jsr305']
compile deps['com.google.guava:guava']
compile deps['javax.inject:javax.inject']

View File

@@ -1,12 +1,13 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.github.ben-manes.caffeine:caffeine:2.9.3
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:3.8.0
joda-time:joda-time:2.10.14
org.checkerframework:checker-qual:3.19.0

View File

@@ -1,12 +1,13 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.github.ben-manes.caffeine:caffeine:2.9.3
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:3.8.0
joda-time:joda-time:2.10.14
org.checkerframework:checker-qual:3.19.0

View File

@@ -1,12 +1,13 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.github.ben-manes.caffeine:caffeine:2.9.3
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:3.8.0
joda-time:joda-time:2.10.14
org.checkerframework:checker-qual:3.19.0

View File

@@ -1,12 +1,13 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.github.ben-manes.caffeine:caffeine:2.9.3
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:3.8.0
joda-time:joda-time:2.10.14
org.checkerframework:checker-qual:3.19.0

View File

@@ -1,12 +1,13 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.github.ben-manes.caffeine:caffeine:2.9.3
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:3.8.0
joda-time:joda-time:2.10.14
org.checkerframework:checker-qual:3.19.0

View File

@@ -1,12 +1,13 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.github.ben-manes.caffeine:caffeine:2.9.3
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
org.checkerframework:checker-qual:3.8.0
joda-time:joda-time:2.10.14
org.checkerframework:checker-qual:3.19.0

View File

@@ -1,27 +1,27 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.7.4
com.github.ben-manes.caffeine:caffeine:2.9.3
com.google.auto.value:auto-value-annotations:1.8.1
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.flogger:flogger:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.1.2
io.github.java-diff-utils:java-diff-utils:4.9
com.google.truth:truth:1.1.3
io.github.java-diff-utils:java-diff-utils:4.11
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.13.1
org.apiguardian:apiguardian-api:1.1.0
joda-time:joda-time:2.10.14
junit:junit:4.13.2
org.checkerframework:checker-compat-qual:2.5.3
org.checkerframework:checker-qual:3.9.1
org.checkerframework:checker-qual:3.19.0
org.hamcrest:hamcrest-core:1.3
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.junit.jupiter:junit-jupiter-api:5.9.0-RC1
org.junit.jupiter:junit-jupiter-engine:5.9.0-RC1
org.junit.platform:junit-platform-commons:1.9.0-RC1
org.junit.platform:junit-platform-engine:1.9.0-RC1
org.junit:junit-bom:5.9.0-RC1
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm:9.0
org.ow2.asm:asm:9.1

View File

@@ -1,27 +1,28 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.7.4
com.github.ben-manes.caffeine:caffeine:2.9.3
com.google.auto.value:auto-value-annotations:1.8.1
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.flogger:flogger:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.1.2
io.github.java-diff-utils:java-diff-utils:4.9
com.google.truth:truth:1.1.3
io.github.java-diff-utils:java-diff-utils:4.11
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.13.1
org.apiguardian:apiguardian-api:1.1.0
joda-time:joda-time:2.10.14
junit:junit:4.13.2
org.apiguardian:apiguardian-api:1.1.2
org.checkerframework:checker-compat-qual:2.5.3
org.checkerframework:checker-qual:3.9.1
org.checkerframework:checker-qual:3.19.0
org.hamcrest:hamcrest-core:1.3
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.junit.jupiter:junit-jupiter-api:5.9.0-RC1
org.junit.jupiter:junit-jupiter-engine:5.9.0-RC1
org.junit.platform:junit-platform-commons:1.9.0-RC1
org.junit.platform:junit-platform-engine:1.9.0-RC1
org.junit:junit-bom:5.9.0-RC1
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm:9.0
org.ow2.asm:asm:9.1

View File

@@ -1,28 +1,28 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.7.4
com.github.ben-manes.caffeine:caffeine:2.9.3
com.google.auto.value:auto-value-annotations:1.8.1
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.1.2
io.github.java-diff-utils:java-diff-utils:4.9
com.google.truth:truth:1.1.3
io.github.java-diff-utils:java-diff-utils:4.11
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.13.1
org.apiguardian:apiguardian-api:1.1.0
joda-time:joda-time:2.10.14
junit:junit:4.13.2
org.checkerframework:checker-compat-qual:2.5.3
org.checkerframework:checker-qual:3.9.1
org.checkerframework:checker-qual:3.19.0
org.hamcrest:hamcrest-core:1.3
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.junit.jupiter:junit-jupiter-api:5.9.0-RC1
org.junit.jupiter:junit-jupiter-engine:5.9.0-RC1
org.junit.platform:junit-platform-commons:1.9.0-RC1
org.junit.platform:junit-platform-engine:1.9.0-RC1
org.junit:junit-bom:5.9.0-RC1
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm:9.0
org.ow2.asm:asm:9.1

View File

@@ -1,28 +1,28 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.7.4
com.github.ben-manes.caffeine:caffeine:2.9.3
com.google.auto.value:auto-value-annotations:1.8.1
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.1.2
io.github.java-diff-utils:java-diff-utils:4.9
com.google.truth:truth:1.1.3
io.github.java-diff-utils:java-diff-utils:4.11
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.13.1
org.apiguardian:apiguardian-api:1.1.0
joda-time:joda-time:2.10.14
junit:junit:4.13.2
org.checkerframework:checker-compat-qual:2.5.3
org.checkerframework:checker-qual:3.9.1
org.checkerframework:checker-qual:3.19.0
org.hamcrest:hamcrest-core:1.3
org.junit.jupiter:junit-jupiter-api:5.6.2
org.junit.jupiter:junit-jupiter-engine:5.6.2
org.junit.platform:junit-platform-commons:1.6.2
org.junit.platform:junit-platform-engine:1.6.2
org.junit:junit-bom:5.6.2
org.junit.jupiter:junit-jupiter-api:5.9.0-RC1
org.junit.jupiter:junit-jupiter-engine:5.9.0-RC1
org.junit.platform:junit-platform-commons:1.9.0-RC1
org.junit.platform:junit-platform-engine:1.9.0-RC1
org.junit:junit-bom:5.9.0-RC1
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm:9.0
org.ow2.asm:asm:9.1

View File

@@ -1,20 +1,21 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.7.4
com.github.ben-manes.caffeine:caffeine:2.9.3
com.google.auto.value:auto-value-annotations:1.8.1
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.flogger:flogger:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.1.2
io.github.java-diff-utils:java-diff-utils:4.9
com.google.truth:truth:1.1.3
io.github.java-diff-utils:java-diff-utils:4.11
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.13.1
joda-time:joda-time:2.10.14
junit:junit:4.13.2
org.checkerframework:checker-compat-qual:2.5.3
org.checkerframework:checker-qual:3.9.1
org.checkerframework:checker-qual:3.19.0
org.hamcrest:hamcrest-core:1.3
org.ow2.asm:asm:9.0
org.ow2.asm:asm:9.1

View File

@@ -1,20 +1,21 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.7.4
com.github.ben-manes.caffeine:caffeine:2.9.3
com.google.auto.value:auto-value-annotations:1.8.1
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.flogger:flogger:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.1.2
io.github.java-diff-utils:java-diff-utils:4.9
com.google.truth:truth:1.1.3
io.github.java-diff-utils:java-diff-utils:4.11
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.13.1
joda-time:joda-time:2.10.14
junit:junit:4.13.2
org.checkerframework:checker-compat-qual:2.5.3
org.checkerframework:checker-qual:3.9.1
org.checkerframework:checker-qual:3.19.0
org.hamcrest:hamcrest-core:1.3
org.ow2.asm:asm:9.0
org.ow2.asm:asm:9.1

View File

@@ -1,21 +1,22 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.7.4
com.github.ben-manes.caffeine:caffeine:2.9.3
com.google.auto.value:auto-value-annotations:1.8.1
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.1.2
io.github.java-diff-utils:java-diff-utils:4.9
com.google.truth:truth:1.1.3
io.github.java-diff-utils:java-diff-utils:4.11
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.13.1
joda-time:joda-time:2.10.14
junit:junit:4.13.2
org.checkerframework:checker-compat-qual:2.5.3
org.checkerframework:checker-qual:3.9.1
org.checkerframework:checker-qual:3.19.0
org.hamcrest:hamcrest-core:1.3
org.ow2.asm:asm:9.0
org.ow2.asm:asm:9.1

View File

@@ -1,21 +1,22 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.7.4
com.github.ben-manes.caffeine:caffeine:2.9.3
com.google.auto.value:auto-value-annotations:1.8.1
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.11.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.1.2
io.github.java-diff-utils:java-diff-utils:4.9
com.google.truth:truth:1.1.3
io.github.java-diff-utils:java-diff-utils:4.11
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.13.1
joda-time:joda-time:2.10.14
junit:junit:4.13.2
org.checkerframework:checker-compat-qual:2.5.3
org.checkerframework:checker-qual:3.9.1
org.checkerframework:checker-qual:3.19.0
org.hamcrest:hamcrest-core:1.3
org.ow2.asm:asm:9.0
org.ow2.asm:asm:9.1

View File

@@ -1,3 +1,2 @@
This is a random file,
with three lines and terminates without a newline.
with three lines and terminates without a newline.

View File

@@ -1,5 +1,4 @@
|Expected |Actual |
|------------------------------------------------------|-----------------------------------------------------|
|This is a random file, |This is a random file, |
| |with **two** lines and terminates **with** a newline.|
|with ~three~ lines and terminates ~without~ a newline.| |
|with ~three~ lines and terminates ~without~ a newline.|with **two** lines and terminates **with** a newline.|

View File

@@ -1,6 +1,5 @@
--- expected
+++ actual
@@ -2,2 +2,1 @@
-
@@ -2,1 +2,1 @@
-with three lines and terminates without a newline.
+with two lines and terminates with a newline.

View File

@@ -14,9 +14,9 @@
package google.registry.testing;
import com.google.common.cache.CacheBuilder;
import com.google.common.cache.CacheLoader;
import com.google.common.cache.LoadingCache;
import com.github.benmanes.caffeine.cache.CacheLoader;
import com.github.benmanes.caffeine.cache.Caffeine;
import com.github.benmanes.caffeine.cache.LoadingCache;
import com.google.common.flogger.FluentLogger;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
@@ -29,7 +29,7 @@ public final class SystemInfo {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final LoadingCache<String, Boolean> hasCommandCache =
CacheBuilder.newBuilder()
Caffeine.newBuilder()
.build(
new CacheLoader<String, Boolean>() {
@Override

View File

@@ -69,6 +69,12 @@
{
"moduleLicense": "BSD 3-clause New License"
},
{
"moduleLicense": "BSD-3-Clause"
},
{
"moduleLicense": "BSD-3-Clause;link=\"https://raw.githubusercontent.com/dnsjava/dnsjava/master/LICENSE\""
},
{
"moduleLicense": "3-Clause BSD License"
},
@@ -111,6 +117,9 @@
{
"moduleLicense": "The PostgreSQL License"
},
{
"moduleLicense": "CC0"
},
{
"moduleLicense": "CC0 1.0 Universal License"
},
@@ -258,6 +267,12 @@
"moduleLicense": "Public Domain",
"moduleName": "org.tukaani:xz"
},
{
// "Apache License, Version 2.0".
"moduleLicense": null,
"moduleVersion": "2.10.0",
"moduleName": "com.google.gwt:gwt-user"
},
{
// "Apache License, Version 2.0". The plugin is able to parse up to
// 2.11.3 correctly but then something changed with 2.12.* and it no
@@ -265,6 +280,13 @@
"moduleLicense": null,
"moduleName": "com.fasterxml.jackson:jackson-bom"
},
{
// "Apache License, Version 2.0". The plugin is able to parse up to
// 2.0.33.Final but not this verson.
"moduleLicense": null,
"moduleVersion": "2.0.46.Final",
"moduleName": "io.netty:netty-tcnative-classes"
},
{
// Actually Eclipse Public License v2.0
"moduleLicense": null,

View File

@@ -57,6 +57,7 @@ PROPERTIES_HEADER = """\
#
# DO NOT EDIT THIS FILE BY HAND
org.gradle.jvmargs=-Xmx1024m
org.gradle.caching=true
"""
# Help text to be displayed (in addition to the synopsis and flag help, which

View File

@@ -37,7 +37,6 @@ def jsDir = "${project.projectDir}/src/main/javascript"
// TODO(weiminyu): identify cause and fix offending tests.
def outcastTestPatterns = [
// Problem seems to lie with AppEngine TaskQueue for test.
"google/registry/batch/DeleteContactsAndHostsActionTest.*",
"google/registry/batch/RefreshDnsOnHostRenameActionTest.*",
"google/registry/flows/CheckApiActionTest.*",
"google/registry/flows/EppLifecycleHostTest.*",
@@ -172,6 +171,7 @@ dependencies {
testRuntime files(sourceSets.test.resources.srcDirs)
compile deps['com.beust:jcommander']
compile deps['com.github.ben-manes.caffeine:caffeine']
compile deps['com.google.api:gax']
compile deps['com.google.api.grpc:proto-google-cloud-datastore-v1']
compile deps['com.google.api.grpc:proto-google-common-protos']
@@ -196,7 +196,6 @@ dependencies {
compile deps['com.google.apis:google-api-services-storage']
testCompile deps['com.google.appengine:appengine-api-stubs']
compile deps['com.google.appengine.tools:appengine-gcs-client']
compile deps['com.google.appengine.tools:appengine-mapreduce']
compile deps['com.google.appengine.tools:appengine-pipeline']
compile deps['com.google.appengine:appengine-remote-api']
compile deps['com.google.auth:google-auth-library-credentials']
@@ -257,6 +256,7 @@ dependencies {
compile deps['org.apache.beam:beam-sdks-java-core']
compile deps['org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core']
compile deps['org.apache.beam:beam-sdks-java-io-google-cloud-platform']
compile deps['org.apache.commons:commons-csv']
compile deps['org.apache.commons:commons-lang3']
testCompile deps['org.apache.commons:commons-text']
testCompile deps['org.apache.ftpserver:ftplet-api']
@@ -703,28 +703,9 @@ createToolTask(
'google.registry.tools.DevTool',
sourceSets.nonprod)
createToolTask(
'initSqlPipeline', 'google.registry.beam.initsql.InitSqlPipeline')
createToolTask(
'validateDatabasePipeline', 'google.registry.beam.comparedb.ValidateDatabasePipeline')
createToolTask(
'jpaDemoPipeline', 'google.registry.beam.common.JpaDemoPipeline')
createToolTask(
'createSyntheticHistoryEntries',
'google.registry.tools.javascrap.CreateSyntheticHistoryEntriesPipeline')
// Caller must provide projectId, GCP region, runner, and the kinds to delete
// (comma-separated kind names or '*' for all). E.g.:
// nom_build :core:bulkDeleteDatastore --args="--project=domain-registry-crash \
// --region=us-central1 --runner=DataflowRunner --kindsToDelete=*"
createToolTask(
'bulkDeleteDatastore',
'google.registry.beam.datastore.BulkDeleteDatastorePipeline')
project.tasks.create('generateSqlSchema', JavaExec) {
classpath = sourceSets.nonprod.runtimeClasspath
main = 'google.registry.tools.DevTool'
@@ -769,16 +750,6 @@ createUberJar(
// User should install gcloud and login to GCP before invoking this tasks.
if (environment == 'alpha') {
def pipelines = [
initSql :
[
mainClass: 'google.registry.beam.initsql.InitSqlPipeline',
metaData : 'google/registry/beam/init_sql_pipeline_metadata.json'
],
bulkDeleteDatastore:
[
mainClass: 'google.registry.beam.datastore.BulkDeleteDatastorePipeline',
metaData : 'google/registry/beam/bulk_delete_datastore_pipeline_metadata.json'
],
spec11 :
[
mainClass: 'google.registry.beam.spec11.Spec11Pipeline',
@@ -794,10 +765,10 @@ if (environment == 'alpha') {
mainClass: 'google.registry.beam.rde.RdePipeline',
metaData : 'google/registry/beam/rde_pipeline_metadata.json'
],
validateDatabase :
resaveAllEppResources:
[
mainClass: 'google.registry.beam.comparedb.ValidateDatabasePipeline',
metaData: 'google/registry/beam/validate_database_pipeline_metadata.json'
mainClass: 'google.registry.beam.resave.ResaveAllEppResourcesPipeline',
metaData: 'google/registry/beam/resave_all_epp_resources_pipeline_metadata.json'
],
]
project.tasks.create("stageBeamPipelines") {
@@ -892,48 +863,6 @@ task buildToolImage(dependsOn: nomulus, type: Exec) {
commandLine 'docker', 'build', '-t', 'nomulus-tool', '.'
}
task generateInitSqlPipelineGraph(type: FilteringTest) {
tests = ['InitSqlPipelineGraphTest.createPipeline_compareGraph']
ignoreFailures = true
}
task updateInitSqlPipelineGraph(type: Copy) {
def graphRelativePath = 'google/registry/beam/initsql/'
from ("${projectDir}/build/resources/test/${graphRelativePath}") {
include 'pipeline_curr.dot'
rename 'curr', 'golden'
}
into "src/test/resources/${graphRelativePath}"
dependsOn generateInitSqlPipelineGraph
doLast {
if (com.google.common.base.Strings.isNullOrEmpty(project.dot_path)) {
getLogger().info('Property dot_path is null. Not creating image for pipeline graph.')
}
def dotPath = project.dot_path
if (!new File(dotPath).exists()) {
throw new RuntimeException(
"""\
${dotPath} not found. Make sure graphviz is installed
and the dot_path property is set correctly."""
.stripIndent())
}
def goldenGraph = "src/test/resources/${graphRelativePath}/pipeline_golden.dot"
def goldenImage = "src/test/resources/${graphRelativePath}/pipeline_golden.png"
def cmd = "${dotPath} -Tpng -o \"${goldenImage}\" \"${goldenGraph}\""
try {
rootProject.ext.execInBash(cmd, projectDir)
} catch (Throwable throwable) {
throw new RuntimeException(
"""\
Failed to generate golden image with command ${cmd}
Error: ${throwable.getMessage()}
""")
}
}
}
// Build the devtool jar.
createUberJar(
'devtool',

View File

@@ -3,41 +3,43 @@
# This file is expected to be part of source control.
com.github.ben-manes.caffeine:caffeine:2.7.0
com.github.kevinstern:software-and-algorithms:1.0
com.google.auto.service:auto-service-annotations:1.0-rc7
com.google.auto.service:auto-service:1.0-rc7
com.google.auto.value:auto-value:1.7.4
com.google.auto:auto-common:0.10
com.google.auto.service:auto-service-annotations:1.0.1
com.google.auto.service:auto-service:1.0.1
com.google.auto.value:auto-value:1.9
com.google.auto:auto-common:1.2
com.google.code.findbugs:jFormatString:3.0.0
com.google.code.findbugs:jsr305:3.0.2
com.google.dagger:dagger-compiler:2.33
com.google.dagger:dagger-producers:2.33
com.google.dagger:dagger-spi:2.33
com.google.dagger:dagger:2.33
com.google.dagger:dagger-compiler:2.42
com.google.dagger:dagger-producers:2.42
com.google.dagger:dagger-spi:2.42
com.google.dagger:dagger:2.42
com.google.devtools.ksp:symbol-processing-api:1.5.30-1.0.0
com.google.errorprone:error_prone_annotation:2.3.4
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.7.1
com.google.errorprone:error_prone_check_api:2.3.4
com.google.errorprone:error_prone_core:2.3.4
com.google.errorprone:error_prone_type_annotations:2.3.4
com.google.errorprone:javac-shaded:9-dev-r4023-3
com.google.googlejavaformat:google-java-format:1.5
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.0.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.protobuf:protobuf-java:3.4.0
com.googlecode.java-diff-utils:diffutils:1.3.0
com.squareup:javapoet:1.13.0
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.persistence:javax.persistence-api:2.2
net.ltgt.gradle.incap:incap:0.2
org.checkerframework:checker-compat-qual:2.5.3
org.checkerframework:checker-qual:3.8.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.12.0
org.checkerframework:dataflow:3.0.0
org.checkerframework:javacutil:3.0.0
org.jetbrains.kotlin:kotlin-stdlib-common:1.4.20
org.jetbrains.kotlin:kotlin-stdlib:1.4.20
org.jetbrains.kotlinx:kotlinx-metadata-jvm:0.1.0
org.jetbrains.kotlin:kotlin-stdlib-common:1.6.10
org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.5.32
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.5.32
org.jetbrains.kotlin:kotlin-stdlib:1.6.10
org.jetbrains.kotlinx:kotlinx-metadata-jvm:0.4.2
org.jetbrains:annotations:13.0
org.pcollections:pcollections:2.1.2
org.plumelib:plume-util:1.0.6

View File

@@ -4,289 +4,299 @@
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.101tec:zkclient:0.10
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.13.0
com.fasterxml.jackson.core:jackson-core:2.13.0
com.fasterxml.jackson.core:jackson-databind:2.13.0
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
com.fasterxml.jackson:jackson-bom:2.13.0
com.fasterxml.jackson.core:jackson-annotations:2.13.3
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson.core:jackson-databind:2.13.3
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.fasterxml:classmate:1.5.1
com.github.docker-java:docker-java-api:3.2.7
com.github.docker-java:docker-java-transport-zerodep:3.2.7
com.github.docker-java:docker-java-transport:3.2.7
com.github.jnr:jffi:1.3.1
com.github.ben-manes.caffeine:caffeine:2.9.3
com.github.docker-java:docker-java-api:3.2.13
com.github.docker-java:docker-java-transport-zerodep:3.2.13
com.github.docker-java:docker-java-transport:3.2.13
com.github.jnr:jffi:1.3.9
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.10.1
com.github.jnr:jnr-enxio:0.32.3
com.github.jnr:jnr-ffi:2.2.1
com.github.jnr:jnr-posix:3.1.4
com.github.jnr:jnr-unixsocket:0.38.5
com.github.jnr:jnr-constants:0.10.3
com.github.jnr:jnr-enxio:0.32.13
com.github.jnr:jnr-ffi:2.2.11
com.github.jnr:jnr-posix:3.1.15
com.github.jnr:jnr-unixsocket:0.38.17
com.github.jnr:jnr-x86asm:1.0.2
com.google.android:annotations:4.1.1.4
com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-appengine:1.35.2
com.google.api-client:google-api-client-jackson2:1.32.2
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.32.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api-client:google-api-client-java6:1.35.2
com.google.api-client:google-api-client-servlet:1.35.2
com.google.api-client:google-api-client:1.35.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:grpc-google-common-protos:2.7.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
com.google.api.grpc:proto-google-common-protos:2.7.1
com.google.api.grpc:proto-google-iam-v1:1.2.0
com.google.api:api-common:2.1.2
com.google.api:gax-grpc:2.8.1
com.google.api:gax-httpjson:0.93.1
com.google.api:gax:2.8.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:grpc-google-common-protos:2.8.3
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.6.2
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.10
com.google.api.grpc:proto-google-cloud-firestore-v1:3.1.0
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.3.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.3.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:proto-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:2.3.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.93.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.93.0
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-grpc:2.18.2
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
com.google.apis:google-api-services-cloudkms:v1-rev20220617-1.32.1
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
com.google.apis:google-api-services-drive:v2-rev393-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev20210624-1.32.1
com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-monitoring:v3-rev20220626-1.32.1
com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220513-1.32.1
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.86
com.google.appengine:appengine-remote-api:1.9.86
com.google.appengine:appengine-api-1.0-sdk:2.0.5
com.google.appengine:appengine-remote-api:2.0.5
com.google.appengine:appengine-testing:1.9.86
com.google.auth:google-auth-library-credentials:1.3.0
com.google.auth:google-auth-library-oauth2-http:1.3.0
com.google.auto.service:auto-service-annotations:1.0-rc7
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.service:auto-service-annotations:1.0.1
com.google.auto.value:auto-value-annotations:1.9
com.google.auto.value:auto-value:1.7.4
com.google.cloud.bigdataoss:gcsio:2.2.4
com.google.cloud.bigdataoss:util:2.2.4
com.google.cloud.bigtable:bigtable-client-core:1.25.1
com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
com.google.cloud:google-cloud-bigquerystorage:2.8.1
com.google.cloud:google-cloud-bigtable:1.27.1
com.google.cloud:google-cloud-core-grpc:2.3.5
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:2.3.5
com.google.cloud:google-cloud-firestore:3.0.10
com.google.cloud:google-cloud-pubsub:1.115.1
com.google.cloud:google-cloud-pubsublite:1.4.8
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:6.17.4
com.google.cloud:google-cloud-storage:1.113.12
com.google.cloud:google-cloud-tasks:1.33.2
com.google.auto.value:auto-value:1.9
com.google.cloud.bigdataoss:gcsio:2.2.6
com.google.cloud.bigdataoss:util:2.2.6
com.google.cloud.bigtable:bigtable-client-core:1.26.3
com.google.cloud.bigtable:bigtable-metrics-api:1.26.3
com.google.cloud.datastore:datastore-v1-proto-client:2.2.10
com.google.cloud.sql:jdbc-socket-factory-core:1.6.1
com.google.cloud:google-cloud-bigquerystorage:2.12.2
com.google.cloud:google-cloud-bigtable:2.6.2
com.google.cloud:google-cloud-core-grpc:2.6.0
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-firestore:3.1.0
com.google.cloud:google-cloud-monitoring:1.82.0
com.google.cloud:google-cloud-pubsub:1.116.4
com.google.cloud:google-cloud-pubsublite:1.5.4
com.google.cloud:google-cloud-secretmanager:2.3.0
com.google.cloud:google-cloud-spanner:6.23.3
com.google.cloud:google-cloud-storage:2.9.2
com.google.cloud:google-cloud-tasks:2.3.0
com.google.cloud:grpc-gcp:1.1.0
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.1.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.9
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.10.0
com.google.dagger:dagger:2.42
com.google.errorprone:error_prone_annotations:2.14.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flatbuffers:flatbuffers-java:1.12.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.flogger:google-extensions:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:31.0.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.9.0
com.google.http-client:google-http-client-apache-v2:1.40.1
com.google.http-client:google-http-client-appengine:1.39.2
com.google.http-client:google-http-client-gson:1.41.0
com.google.http-client:google-http-client-jackson2:1.41.0
com.google.http-client:google-http-client-protobuf:1.40.1
com.google.http-client:google-http-client:1.41.0
com.google.gwt:gwt-user:2.10.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client-protobuf:1.41.7
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:2.0.0
com.google.monitoring-client:metrics:1.0.7
com.google.monitoring-client:stackdriver:1.0.7
com.google.oauth-client:google-oauth-client-appengine:1.31.4
com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.32.1
com.google.protobuf:protobuf-java-util:3.19.2
com.google.protobuf:protobuf-java:3.19.2
com.google.re2j:re2j:1.6
com.google.oauth-client:google-oauth-client-appengine:1.34.1
com.google.oauth-client:google-oauth-client-java6:1.34.1
com.google.oauth-client:google-oauth-client-jetty:1.34.1
com.google.oauth-client:google-oauth-client-servlet:1.34.1
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.re2j:re2j:1.7
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
com.ibm.icu:icu4j:71.1
com.jcraft:jsch:0.1.55
com.lmax:disruptor:3.4.2
com.sun.istack:istack-commons-runtime:3.0.7
com.sun.xml.fastinfoset:FastInfoset:1.2.15
com.thoughtworks.paranamer:paranamer:2.7
com.zaxxer:HikariCP:3.4.5
commons-codec:commons-codec:1.15
commons-logging:commons-logging:1.2
dnsjava:dnsjava:3.3.1
dnsjava:dnsjava:3.5.1
io.confluent:common-config:5.3.2
io.confluent:common-utils:5.3.2
io.confluent:kafka-avro-serializer:5.3.2
io.confluent:kafka-schema-registry-client:5.3.2
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.104
io.grpc:grpc-alts:1.43.2
io.grpc:grpc-api:1.43.2
io.grpc:grpc-auth:1.43.2
io.grpc:grpc-context:1.43.2
io.grpc:grpc-core:1.43.2
io.grpc:grpc-grpclb:1.43.2
io.grpc:grpc-netty-shaded:1.43.2
io.grpc:grpc-netty:1.43.2
io.grpc:grpc-protobuf-lite:1.43.2
io.grpc:grpc-protobuf:1.43.2
io.grpc:grpc-services:1.43.2
io.grpc:grpc-stub:1.43.2
io.grpc:grpc-xds:1.43.2
io.netty:netty-buffer:4.1.63.Final
io.netty:netty-codec-http2:4.1.63.Final
io.netty:netty-codec-http:4.1.63.Final
io.netty:netty-codec-socks:4.1.63.Final
io.netty:netty-codec:4.1.63.Final
io.netty:netty-common:4.1.63.Final
io.netty:netty-handler-proxy:4.1.63.Final
io.netty:netty-handler:4.1.63.Final
io.netty:netty-resolver:4.1.63.Final
io.netty:netty-tcnative-boringssl-static:2.0.33.Final
io.netty:netty-transport:4.1.63.Final
io.opencensus:opencensus-api:0.30.0
io.opencensus:opencensus-contrib-grpc-util:0.30.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-alts:1.47.0
io.grpc:grpc-api:1.47.0
io.grpc:grpc-auth:1.47.0
io.grpc:grpc-census:1.45.1
io.grpc:grpc-context:1.47.0
io.grpc:grpc-core:1.47.0
io.grpc:grpc-googleapis:1.47.0
io.grpc:grpc-grpclb:1.47.0
io.grpc:grpc-netty-shaded:1.47.0
io.grpc:grpc-netty:1.45.1
io.grpc:grpc-protobuf-lite:1.47.0
io.grpc:grpc-protobuf:1.47.0
io.grpc:grpc-services:1.47.0
io.grpc:grpc-stub:1.47.0
io.grpc:grpc-xds:1.47.0
io.netty:netty-buffer:4.1.72.Final
io.netty:netty-codec-http2:4.1.72.Final
io.netty:netty-codec-http:4.1.72.Final
io.netty:netty-codec-socks:4.1.72.Final
io.netty:netty-codec:4.1.72.Final
io.netty:netty-common:4.1.72.Final
io.netty:netty-handler-proxy:4.1.72.Final
io.netty:netty-handler:4.1.72.Final
io.netty:netty-resolver:4.1.72.Final
io.netty:netty-tcnative-boringssl-static:2.0.46.Final
io.netty:netty-tcnative-classes:2.0.46.Final
io.netty:netty-transport:4.1.72.Final
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-exemplar-util:0.31.0
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0
io.opencensus:opencensus-contrib-grpc-util:0.31.0
io.opencensus:opencensus-contrib-http-util:0.31.1
io.opencensus:opencensus-contrib-resource-util:0.31.0
io.opencensus:opencensus-exporter-metrics-util:0.31.0
io.opencensus:opencensus-exporter-stats-stackdriver:0.31.0
io.opencensus:opencensus-impl-core:0.31.0
io.opencensus:opencensus-impl:0.31.0
io.opencensus:opencensus-proto:0.2.0
io.perfmark:perfmark-api:0.23.0
it.unimi.dsi:fastutil:6.5.16
io.perfmark:perfmark-api:0.25.0
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.jdo:jdo2-api:2.3-20090302111651
javax.mail:mail:1.5.0-b01
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.10
junit:junit:4.13.1
net.bytebuddy:byte-buddy:1.10.17
net.java.dev.jna:jna:5.5.0
junit:junit:4.13.2
net.bytebuddy:byte-buddy:1.12.9
net.java.dev.jna:jna:5.8.0
org.apache.arrow:arrow-format:5.0.0
org.apache.arrow:arrow-memory-core:5.0.0
org.apache.arrow:arrow-vector:5.0.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-fn-execution:2.37.0
org.apache.beam:beam-model-job-management:2.37.0
org.apache.beam:beam-model-pipeline:2.37.0
org.apache.beam:beam-runners-core-construction-java:2.37.0
org.apache.beam:beam-runners-core-java:2.37.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
org.apache.beam:beam-runners-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-core:2.37.0
org.apache.beam:beam-sdks-java-expansion-service:2.37.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
org.apache.beam:beam-sdks-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
org.apache.beam:beam-sdks-java-io-kafka:2.37.0
org.apache.beam:beam-model-fn-execution:2.40.0
org.apache.beam:beam-model-job-management:2.40.0
org.apache.beam:beam-model-pipeline:2.40.0
org.apache.beam:beam-runners-core-construction-java:2.40.0
org.apache.beam:beam-runners-core-java:2.40.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.40.0
org.apache.beam:beam-runners-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-core:2.40.0
org.apache.beam:beam-sdks-java-expansion-service:2.40.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.40.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.40.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.40.0
org.apache.beam:beam-sdks-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0
org.apache.beam:beam-sdks-java-io-kafka:2.40.0
org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
org.apache.beam:beam-vendor-grpc-1_43_2:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-compress:1.21
org.apache.commons:commons-csv:1.9.0
org.apache.commons:commons-lang3:3.12.0
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.15
org.bouncycastle:bcpg-jdk15on:1.67
org.bouncycastle:bcpkix-jdk15on:1.67
org.bouncycastle:bcprov-jdk15on:1.67
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.21.0
org.checkerframework:checker-qual:3.22.2
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.20
org.codehaus.mojo:animal-sniffer-annotations:1.21
org.conscrypt:conscrypt-openjdk-uber:2.5.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.flywaydb:flyway-core:8.5.13
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.gwtproject:gwt-user:2.10.0
org.hamcrest:hamcrest-core:1.3
org.hamcrest:hamcrest:2.1
org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
org.hibernate:hibernate-core:5.4.23.Final
org.hibernate:hibernate-hikaricp:5.4.23.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.4.1.Final
org.hibernate:hibernate-core:5.6.10.Final
org.hibernate:hibernate-hikaricp:5.6.10.Final
org.jboss.logging:jboss-logging:3.4.3.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.1.3.Final
org.jboss:jandex:2.4.2.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20200518
org.jsoup:jsoup:1.13.1
org.jsoup:jsoup:1.15.2
org.jvnet.staxex:stax-ex:1.8
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:9.1
org.ow2.asm:asm-commons:9.0
org.ow2.asm:asm-tree:9.1
org.ow2.asm:asm-util:9.1
org.ow2.asm:asm:9.1
org.postgresql:postgresql:42.2.18
org.ow2.asm:asm-analysis:9.3
org.ow2.asm:asm-commons:9.2
org.ow2.asm:asm-tree:9.3
org.ow2.asm:asm-util:9.3
org.ow2.asm:asm:9.3
org.postgresql:postgresql:42.4.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.slf4j:slf4j-api:1.7.30
org.springframework:spring-core:4.3.18.RELEASE
org.springframework:spring-expression:4.3.18.RELEASE
org.testcontainers:database-commons:1.15.2
org.testcontainers:jdbc:1.15.2
org.testcontainers:postgresql:1.15.2
org.testcontainers:testcontainers:1.15.2
org.threeten:threetenbp:1.5.2
org.slf4j:slf4j-api:1.7.36
org.springframework:spring-core:5.3.18
org.springframework:spring-expression:5.3.18
org.springframework:spring-jcl:5.3.18
org.testcontainers:database-commons:1.17.3
org.testcontainers:jdbc:1.17.3
org.testcontainers:postgresql:1.17.3
org.testcontainers:testcontainers:1.17.3
org.threeten:threetenbp:1.6.0
org.tukaani:xz:1.5
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.8.4
org.yaml:snakeyaml:1.28
org.yaml:snakeyaml:1.30
us.fatehi:schemacrawler-api:16.10.1
us.fatehi:schemacrawler-diagram:16.10.1
us.fatehi:schemacrawler-tools:16.10.1

View File

@@ -4,279 +4,291 @@
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.101tec:zkclient:0.10
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.13.0
com.fasterxml.jackson.core:jackson-core:2.13.0
com.fasterxml.jackson.core:jackson-databind:2.13.0
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
com.fasterxml.jackson:jackson-bom:2.13.0
com.fasterxml.jackson.core:jackson-annotations:2.13.3
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson.core:jackson-databind:2.13.3
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.fasterxml:classmate:1.5.1
com.github.docker-java:docker-java-api:3.2.7
com.github.docker-java:docker-java-transport-zerodep:3.2.7
com.github.docker-java:docker-java-transport:3.2.7
com.github.jnr:jffi:1.3.1
com.github.ben-manes.caffeine:caffeine:2.9.3
com.github.docker-java:docker-java-api:3.2.13
com.github.docker-java:docker-java-transport-zerodep:3.2.13
com.github.docker-java:docker-java-transport:3.2.13
com.github.jnr:jffi:1.3.9
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.10.1
com.github.jnr:jnr-enxio:0.32.3
com.github.jnr:jnr-ffi:2.2.1
com.github.jnr:jnr-posix:3.1.4
com.github.jnr:jnr-unixsocket:0.38.5
com.github.jnr:jnr-constants:0.10.3
com.github.jnr:jnr-enxio:0.32.13
com.github.jnr:jnr-ffi:2.2.11
com.github.jnr:jnr-posix:3.1.15
com.github.jnr:jnr-unixsocket:0.38.17
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-appengine:1.35.2
com.google.api-client:google-api-client-jackson2:1.32.2
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.32.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api-client:google-api-client-java6:1.35.2
com.google.api-client:google-api-client-servlet:1.35.2
com.google.api-client:google-api-client:1.35.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:grpc-google-common-protos:2.7.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
com.google.api.grpc:proto-google-common-protos:2.7.1
com.google.api.grpc:proto-google-iam-v1:1.2.0
com.google.api:api-common:2.1.2
com.google.api:gax-grpc:2.8.1
com.google.api:gax-httpjson:0.93.1
com.google.api:gax:2.8.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:grpc-google-common-protos:2.8.3
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.6.2
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.10
com.google.api.grpc:proto-google-cloud-firestore-v1:3.1.0
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.3.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.3.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:proto-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:2.3.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.93.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.93.0
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-grpc:2.18.2
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
com.google.apis:google-api-services-cloudkms:v1-rev20220617-1.32.1
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
com.google.apis:google-api-services-drive:v2-rev393-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev20210624-1.32.1
com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-monitoring:v3-rev20220626-1.32.1
com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220513-1.32.1
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.86
com.google.appengine:appengine-remote-api:1.9.86
com.google.appengine:appengine-api-1.0-sdk:2.0.5
com.google.appengine:appengine-remote-api:2.0.5
com.google.appengine:appengine-testing:1.9.86
com.google.auth:google-auth-library-credentials:1.3.0
com.google.auth:google-auth-library-oauth2-http:1.3.0
com.google.auto.service:auto-service-annotations:1.0-rc7
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.service:auto-service-annotations:1.0.1
com.google.auto.value:auto-value-annotations:1.9
com.google.auto.value:auto-value:1.7.4
com.google.cloud.bigdataoss:gcsio:2.2.4
com.google.cloud.bigdataoss:util:2.2.4
com.google.cloud.bigtable:bigtable-client-core:1.25.1
com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
com.google.cloud:google-cloud-bigquerystorage:2.8.1
com.google.cloud:google-cloud-bigtable:1.27.1
com.google.cloud:google-cloud-core-grpc:2.3.5
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:2.3.5
com.google.cloud:google-cloud-firestore:3.0.10
com.google.cloud:google-cloud-pubsub:1.115.1
com.google.cloud:google-cloud-pubsublite:1.4.8
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:6.17.4
com.google.cloud:google-cloud-storage:1.113.12
com.google.cloud:google-cloud-tasks:1.33.2
com.google.auto.value:auto-value:1.9
com.google.cloud.bigdataoss:gcsio:2.2.6
com.google.cloud.bigdataoss:util:2.2.6
com.google.cloud.bigtable:bigtable-client-core:1.26.3
com.google.cloud.bigtable:bigtable-metrics-api:1.26.3
com.google.cloud.datastore:datastore-v1-proto-client:2.2.10
com.google.cloud.sql:jdbc-socket-factory-core:1.6.1
com.google.cloud:google-cloud-bigquerystorage:2.12.2
com.google.cloud:google-cloud-bigtable:2.6.2
com.google.cloud:google-cloud-core-grpc:2.6.0
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-firestore:3.1.0
com.google.cloud:google-cloud-monitoring:1.82.0
com.google.cloud:google-cloud-pubsub:1.116.4
com.google.cloud:google-cloud-pubsublite:1.5.4
com.google.cloud:google-cloud-secretmanager:2.3.0
com.google.cloud:google-cloud-spanner:6.23.3
com.google.cloud:google-cloud-storage:2.9.2
com.google.cloud:google-cloud-tasks:2.3.0
com.google.cloud:grpc-gcp:1.1.0
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.1.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.9
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.10.0
com.google.dagger:dagger:2.42
com.google.errorprone:error_prone_annotations:2.14.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flatbuffers:flatbuffers-java:1.12.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.flogger:google-extensions:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:31.0.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.9.0
com.google.http-client:google-http-client-apache-v2:1.40.1
com.google.http-client:google-http-client-appengine:1.39.2
com.google.http-client:google-http-client-gson:1.41.0
com.google.http-client:google-http-client-jackson2:1.41.0
com.google.http-client:google-http-client-protobuf:1.40.1
com.google.http-client:google-http-client:1.41.0
com.google.gwt:gwt-user:2.10.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client-protobuf:1.41.7
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:2.0.0
com.google.monitoring-client:metrics:1.0.7
com.google.monitoring-client:stackdriver:1.0.7
com.google.oauth-client:google-oauth-client-appengine:1.31.4
com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.32.1
com.google.protobuf:protobuf-java-util:3.19.2
com.google.protobuf:protobuf-java:3.19.2
com.google.re2j:re2j:1.6
com.google.oauth-client:google-oauth-client-appengine:1.34.1
com.google.oauth-client:google-oauth-client-java6:1.34.1
com.google.oauth-client:google-oauth-client-jetty:1.34.1
com.google.oauth-client:google-oauth-client-servlet:1.34.1
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.re2j:re2j:1.7
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
com.ibm.icu:icu4j:71.1
com.jcraft:jsch:0.1.55
com.lmax:disruptor:3.4.2
com.sun.istack:istack-commons-runtime:3.0.7
com.sun.xml.fastinfoset:FastInfoset:1.2.15
com.thoughtworks.paranamer:paranamer:2.7
com.zaxxer:HikariCP:3.4.5
commons-codec:commons-codec:1.15
commons-logging:commons-logging:1.2
dnsjava:dnsjava:3.3.1
dnsjava:dnsjava:3.5.1
io.confluent:common-config:5.3.2
io.confluent:common-utils:5.3.2
io.confluent:kafka-avro-serializer:5.3.2
io.confluent:kafka-schema-registry-client:5.3.2
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.104
io.grpc:grpc-alts:1.43.2
io.grpc:grpc-api:1.43.2
io.grpc:grpc-auth:1.43.2
io.grpc:grpc-context:1.43.2
io.grpc:grpc-core:1.43.2
io.grpc:grpc-grpclb:1.43.2
io.grpc:grpc-netty-shaded:1.43.2
io.grpc:grpc-netty:1.43.2
io.grpc:grpc-protobuf-lite:1.43.2
io.grpc:grpc-protobuf:1.43.2
io.grpc:grpc-stub:1.43.2
io.netty:netty-buffer:4.1.63.Final
io.netty:netty-codec-http2:4.1.63.Final
io.netty:netty-codec-http:4.1.63.Final
io.netty:netty-codec:4.1.63.Final
io.netty:netty-common:4.1.63.Final
io.netty:netty-handler:4.1.63.Final
io.netty:netty-resolver:4.1.63.Final
io.netty:netty-tcnative-boringssl-static:2.0.33.Final
io.netty:netty-transport:4.1.63.Final
io.opencensus:opencensus-api:0.30.0
io.opencensus:opencensus-contrib-grpc-util:0.30.0
io.opencensus:opencensus-contrib-http-util:0.28.0
it.unimi.dsi:fastutil:6.5.16
io.grpc:grpc-alts:1.45.1
io.grpc:grpc-api:1.47.0
io.grpc:grpc-auth:1.45.1
io.grpc:grpc-census:1.45.1
io.grpc:grpc-context:1.47.0
io.grpc:grpc-core:1.45.1
io.grpc:grpc-grpclb:1.45.1
io.grpc:grpc-netty-shaded:1.45.1
io.grpc:grpc-netty:1.45.1
io.grpc:grpc-protobuf-lite:1.47.0
io.grpc:grpc-protobuf:1.47.0
io.grpc:grpc-services:1.45.1
io.grpc:grpc-stub:1.47.0
io.grpc:grpc-xds:1.45.1
io.netty:netty-buffer:4.1.72.Final
io.netty:netty-codec-http2:4.1.72.Final
io.netty:netty-codec-http:4.1.72.Final
io.netty:netty-codec:4.1.72.Final
io.netty:netty-common:4.1.72.Final
io.netty:netty-handler:4.1.72.Final
io.netty:netty-resolver:4.1.72.Final
io.netty:netty-tcnative-boringssl-static:2.0.46.Final
io.netty:netty-tcnative-classes:2.0.46.Final
io.netty:netty-transport:4.1.72.Final
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-exemplar-util:0.31.0
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0
io.opencensus:opencensus-contrib-grpc-util:0.31.0
io.opencensus:opencensus-contrib-http-util:0.31.1
io.opencensus:opencensus-contrib-resource-util:0.31.0
io.opencensus:opencensus-exporter-metrics-util:0.31.0
io.opencensus:opencensus-exporter-stats-stackdriver:0.31.0
io.opencensus:opencensus-impl-core:0.31.0
io.opencensus:opencensus-impl:0.31.0
io.opencensus:opencensus-proto:0.2.0
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.jdo:jdo2-api:2.3-20090302111651
javax.mail:mail:1.5.0-b01
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.10
net.bytebuddy:byte-buddy:1.10.17
net.java.dev.jna:jna:5.5.0
net.bytebuddy:byte-buddy:1.12.9
net.java.dev.jna:jna:5.8.0
org.apache.arrow:arrow-format:5.0.0
org.apache.arrow:arrow-memory-core:5.0.0
org.apache.arrow:arrow-vector:5.0.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-fn-execution:2.37.0
org.apache.beam:beam-model-job-management:2.37.0
org.apache.beam:beam-model-pipeline:2.37.0
org.apache.beam:beam-runners-core-construction-java:2.37.0
org.apache.beam:beam-runners-core-java:2.37.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
org.apache.beam:beam-runners-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-core:2.37.0
org.apache.beam:beam-sdks-java-expansion-service:2.37.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
org.apache.beam:beam-sdks-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
org.apache.beam:beam-sdks-java-io-kafka:2.37.0
org.apache.beam:beam-model-fn-execution:2.40.0
org.apache.beam:beam-model-job-management:2.40.0
org.apache.beam:beam-model-pipeline:2.40.0
org.apache.beam:beam-runners-core-construction-java:2.40.0
org.apache.beam:beam-runners-core-java:2.40.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.40.0
org.apache.beam:beam-runners-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-core:2.40.0
org.apache.beam:beam-sdks-java-expansion-service:2.40.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.40.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.40.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.40.0
org.apache.beam:beam-sdks-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0
org.apache.beam:beam-sdks-java-io-kafka:2.40.0
org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
org.apache.beam:beam-vendor-grpc-1_43_2:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-compress:1.21
org.apache.commons:commons-csv:1.9.0
org.apache.commons:commons-lang3:3.12.0
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.15
org.bouncycastle:bcpg-jdk15on:1.67
org.bouncycastle:bcpkix-jdk15on:1.67
org.bouncycastle:bcprov-jdk15on:1.67
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.21.0
org.checkerframework:checker-qual:3.22.2
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.conscrypt:conscrypt-openjdk-uber:2.5.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.flywaydb:flyway-core:8.5.13
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.gwtproject:gwt-user:2.10.0
org.hamcrest:hamcrest:2.1
org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
org.hibernate:hibernate-core:5.4.23.Final
org.hibernate:hibernate-hikaricp:5.4.23.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.4.1.Final
org.hibernate:hibernate-core:5.6.10.Final
org.hibernate:hibernate-hikaricp:5.6.10.Final
org.jboss.logging:jboss-logging:3.4.3.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.1.3.Final
org.jboss:jandex:2.4.2.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20200518
org.jsoup:jsoup:1.13.1
org.jsoup:jsoup:1.15.2
org.jvnet.staxex:stax-ex:1.8
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:9.1
org.ow2.asm:asm-commons:9.0
org.ow2.asm:asm-tree:9.1
org.ow2.asm:asm-util:9.1
org.ow2.asm:asm:9.1
org.postgresql:postgresql:42.2.18
org.ow2.asm:asm-analysis:9.3
org.ow2.asm:asm-commons:9.2
org.ow2.asm:asm-tree:9.3
org.ow2.asm:asm-util:9.3
org.ow2.asm:asm:9.3
org.postgresql:postgresql:42.4.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.slf4j:slf4j-api:1.7.30
org.springframework:spring-core:4.3.18.RELEASE
org.springframework:spring-expression:4.3.18.RELEASE
org.testcontainers:database-commons:1.15.2
org.testcontainers:jdbc:1.15.2
org.testcontainers:postgresql:1.15.2
org.testcontainers:testcontainers:1.15.2
org.threeten:threetenbp:1.5.2
org.slf4j:slf4j-api:1.7.36
org.springframework:spring-core:5.3.18
org.springframework:spring-expression:5.3.18
org.springframework:spring-jcl:5.3.18
org.testcontainers:database-commons:1.17.3
org.testcontainers:jdbc:1.17.3
org.testcontainers:postgresql:1.17.3
org.testcontainers:testcontainers:1.17.3
org.threeten:threetenbp:1.6.0
org.tukaani:xz:1.5
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.8.4
org.yaml:snakeyaml:1.28
org.yaml:snakeyaml:1.30
us.fatehi:schemacrawler-api:16.10.1
us.fatehi:schemacrawler-diagram:16.10.1
us.fatehi:schemacrawler-tools:16.10.1

View File

@@ -4,303 +4,313 @@
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.101tec:zkclient:0.10
com.beust:jcommander:1.60
com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.13.0
com.fasterxml.jackson.core:jackson-core:2.13.0
com.fasterxml.jackson.core:jackson-databind:2.13.0
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
com.fasterxml.jackson:jackson-bom:2.13.0
com.fasterxml.jackson.core:jackson-annotations:2.13.3
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson.core:jackson-databind:2.13.3
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.fasterxml:classmate:1.5.1
com.github.docker-java:docker-java-api:3.2.7
com.github.docker-java:docker-java-transport-zerodep:3.2.7
com.github.docker-java:docker-java-transport:3.2.7
com.github.jnr:jffi:1.3.1
com.github.ben-manes.caffeine:caffeine:2.9.3
com.github.docker-java:docker-java-api:3.2.13
com.github.docker-java:docker-java-transport-zerodep:3.2.13
com.github.docker-java:docker-java-transport:3.2.13
com.github.jnr:jffi:1.3.9
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.10.1
com.github.jnr:jnr-enxio:0.32.3
com.github.jnr:jnr-ffi:2.2.1
com.github.jnr:jnr-posix:3.1.4
com.github.jnr:jnr-unixsocket:0.38.5
com.github.jnr:jnr-constants:0.10.3
com.github.jnr:jnr-enxio:0.32.13
com.github.jnr:jnr-ffi:2.2.11
com.github.jnr:jnr-posix:3.1.15
com.github.jnr:jnr-unixsocket:0.38.17
com.github.jnr:jnr-x86asm:1.0.2
com.google.android:annotations:4.1.1.4
com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-appengine:1.35.2
com.google.api-client:google-api-client-jackson2:1.32.2
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.32.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api-client:google-api-client-java6:1.35.2
com.google.api-client:google-api-client-servlet:1.35.2
com.google.api-client:google-api-client:1.35.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:grpc-google-common-protos:2.7.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
com.google.api.grpc:proto-google-common-protos:2.7.1
com.google.api.grpc:proto-google-iam-v1:1.2.0
com.google.api:api-common:2.1.2
com.google.api:gax-grpc:2.8.1
com.google.api:gax-httpjson:0.93.1
com.google.api:gax:2.8.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:grpc-google-common-protos:2.8.3
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.6.2
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.10
com.google.api.grpc:proto-google-cloud-firestore-v1:3.1.0
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.3.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.3.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:proto-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:2.3.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.93.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.93.0
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-grpc:2.18.2
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
com.google.apis:google-api-services-cloudkms:v1-rev20220617-1.32.1
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
com.google.apis:google-api-services-drive:v2-rev393-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev20210624-1.32.1
com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-monitoring:v3-rev20220626-1.32.1
com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220513-1.32.1
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.86
com.google.appengine:appengine-remote-api:1.9.86
com.google.appengine:appengine-api-1.0-sdk:2.0.5
com.google.appengine:appengine-remote-api:2.0.5
com.google.appengine:appengine-testing:1.9.86
com.google.auth:google-auth-library-credentials:1.3.0
com.google.auth:google-auth-library-oauth2-http:1.3.0
com.google.auto.service:auto-service-annotations:1.0-rc7
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.service:auto-service-annotations:1.0.1
com.google.auto.value:auto-value-annotations:1.9
com.google.auto.value:auto-value:1.7.4
com.google.cloud.bigdataoss:gcsio:2.2.4
com.google.cloud.bigdataoss:util:2.2.4
com.google.cloud.bigtable:bigtable-client-core:1.25.1
com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
com.google.cloud.sql:postgres-socket-factory:1.2.1
com.google.cloud:google-cloud-bigquerystorage:2.8.1
com.google.cloud:google-cloud-bigtable:1.27.1
com.google.cloud:google-cloud-core-grpc:2.3.5
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:2.3.5
com.google.cloud:google-cloud-firestore:3.0.10
com.google.cloud:google-cloud-pubsub:1.115.1
com.google.cloud:google-cloud-pubsublite:1.4.8
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:6.17.4
com.google.cloud:google-cloud-storage:1.113.12
com.google.cloud:google-cloud-tasks:1.33.2
com.google.auto.value:auto-value:1.9
com.google.cloud.bigdataoss:gcsio:2.2.6
com.google.cloud.bigdataoss:util:2.2.6
com.google.cloud.bigtable:bigtable-client-core:1.26.3
com.google.cloud.bigtable:bigtable-metrics-api:1.26.3
com.google.cloud.datastore:datastore-v1-proto-client:2.2.10
com.google.cloud.sql:jdbc-socket-factory-core:1.6.1
com.google.cloud.sql:postgres-socket-factory:1.6.1
com.google.cloud:google-cloud-bigquerystorage:2.12.2
com.google.cloud:google-cloud-bigtable:2.6.2
com.google.cloud:google-cloud-core-grpc:2.6.0
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-firestore:3.1.0
com.google.cloud:google-cloud-monitoring:1.82.0
com.google.cloud:google-cloud-pubsub:1.116.4
com.google.cloud:google-cloud-pubsublite:1.5.4
com.google.cloud:google-cloud-secretmanager:2.3.0
com.google.cloud:google-cloud-spanner:6.23.3
com.google.cloud:google-cloud-storage:2.9.2
com.google.cloud:google-cloud-tasks:2.3.0
com.google.cloud:grpc-gcp:1.1.0
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.1.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.9
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.10.0
com.google.dagger:dagger:2.42
com.google.errorprone:error_prone_annotations:2.14.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flatbuffers:flatbuffers-java:1.12.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.flogger:google-extensions:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:31.0.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.9.0
com.google.http-client:google-http-client-apache-v2:1.40.1
com.google.http-client:google-http-client-appengine:1.39.2
com.google.http-client:google-http-client-gson:1.41.0
com.google.http-client:google-http-client-jackson2:1.41.0
com.google.http-client:google-http-client-protobuf:1.40.1
com.google.http-client:google-http-client:1.41.0
com.google.gwt:gwt-user:2.10.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client-protobuf:1.41.7
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:2.0.0
com.google.monitoring-client:metrics:1.0.7
com.google.monitoring-client:stackdriver:1.0.7
com.google.oauth-client:google-oauth-client-appengine:1.31.4
com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.32.1
com.google.protobuf:protobuf-java-util:3.19.2
com.google.protobuf:protobuf-java:3.19.2
com.google.re2j:re2j:1.6
com.google.oauth-client:google-oauth-client-appengine:1.34.1
com.google.oauth-client:google-oauth-client-java6:1.34.1
com.google.oauth-client:google-oauth-client-jetty:1.34.1
com.google.oauth-client:google-oauth-client-servlet:1.34.1
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.re2j:re2j:1.7
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
com.ibm.icu:icu4j:71.1
com.jcraft:jsch:0.1.55
com.lmax:disruptor:3.4.2
com.sun.istack:istack-commons-runtime:3.0.7
com.sun.xml.fastinfoset:FastInfoset:1.2.15
com.thoughtworks.paranamer:paranamer:2.7
com.zaxxer:HikariCP:3.4.5
commons-codec:commons-codec:1.15
commons-logging:commons-logging:1.2
dnsjava:dnsjava:3.3.1
dnsjava:dnsjava:3.5.1
guru.nidi.com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
guru.nidi.com.kitfox:svgSalamander:1.1.3
guru.nidi:graphviz-java-all-j2v8:0.17.0
guru.nidi:graphviz-java:0.17.0
guru.nidi:graphviz-java-all-j2v8:0.18.1
guru.nidi:graphviz-java:0.18.1
io.confluent:common-config:5.3.2
io.confluent:common-utils:5.3.2
io.confluent:kafka-avro-serializer:5.3.2
io.confluent:kafka-schema-registry-client:5.3.2
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.104
io.grpc:grpc-alts:1.43.2
io.grpc:grpc-api:1.43.2
io.grpc:grpc-auth:1.43.2
io.grpc:grpc-context:1.43.2
io.grpc:grpc-core:1.43.2
io.grpc:grpc-grpclb:1.43.2
io.grpc:grpc-netty-shaded:1.43.2
io.grpc:grpc-netty:1.43.2
io.grpc:grpc-protobuf-lite:1.43.2
io.grpc:grpc-protobuf:1.43.2
io.grpc:grpc-services:1.43.2
io.grpc:grpc-stub:1.43.2
io.grpc:grpc-xds:1.43.2
io.netty:netty-buffer:4.1.63.Final
io.netty:netty-codec-http2:4.1.63.Final
io.netty:netty-codec-http:4.1.63.Final
io.netty:netty-codec-socks:4.1.63.Final
io.netty:netty-codec:4.1.63.Final
io.netty:netty-common:4.1.63.Final
io.netty:netty-handler-proxy:4.1.63.Final
io.netty:netty-handler:4.1.63.Final
io.netty:netty-resolver:4.1.63.Final
io.netty:netty-tcnative-boringssl-static:2.0.33.Final
io.netty:netty-transport:4.1.63.Final
io.opencensus:opencensus-api:0.30.0
io.opencensus:opencensus-contrib-grpc-util:0.30.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-alts:1.47.0
io.grpc:grpc-api:1.47.0
io.grpc:grpc-auth:1.47.0
io.grpc:grpc-census:1.45.1
io.grpc:grpc-context:1.47.0
io.grpc:grpc-core:1.47.0
io.grpc:grpc-googleapis:1.47.0
io.grpc:grpc-grpclb:1.47.0
io.grpc:grpc-netty-shaded:1.47.0
io.grpc:grpc-netty:1.45.1
io.grpc:grpc-protobuf-lite:1.47.0
io.grpc:grpc-protobuf:1.47.0
io.grpc:grpc-services:1.47.0
io.grpc:grpc-stub:1.47.0
io.grpc:grpc-xds:1.47.0
io.netty:netty-buffer:4.1.72.Final
io.netty:netty-codec-http2:4.1.72.Final
io.netty:netty-codec-http:4.1.72.Final
io.netty:netty-codec-socks:4.1.72.Final
io.netty:netty-codec:4.1.72.Final
io.netty:netty-common:4.1.72.Final
io.netty:netty-handler-proxy:4.1.72.Final
io.netty:netty-handler:4.1.72.Final
io.netty:netty-resolver:4.1.72.Final
io.netty:netty-tcnative-boringssl-static:2.0.46.Final
io.netty:netty-tcnative-classes:2.0.46.Final
io.netty:netty-transport:4.1.72.Final
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-exemplar-util:0.31.0
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0
io.opencensus:opencensus-contrib-grpc-util:0.31.0
io.opencensus:opencensus-contrib-http-util:0.31.1
io.opencensus:opencensus-contrib-resource-util:0.31.0
io.opencensus:opencensus-exporter-metrics-util:0.31.0
io.opencensus:opencensus-exporter-stats-stackdriver:0.31.0
io.opencensus:opencensus-impl-core:0.31.0
io.opencensus:opencensus-impl:0.31.0
io.opencensus:opencensus-proto:0.2.0
io.perfmark:perfmark-api:0.23.0
it.unimi.dsi:fastutil:6.5.16
io.perfmark:perfmark-api:0.25.0
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.jdo:jdo2-api:2.3-20090302111651
javax.mail:mail:1.5.0-b01
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.10
junit:junit:4.13.1
junit:junit:4.13.2
net.arnx:nashorn-promise:0.1.1
net.bytebuddy:byte-buddy:1.10.17
net.java.dev.jna:jna:5.5.0
net.bytebuddy:byte-buddy:1.12.9
net.java.dev.jna:jna:5.8.0
org.apache.arrow:arrow-format:5.0.0
org.apache.arrow:arrow-memory-core:5.0.0
org.apache.arrow:arrow-vector:5.0.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-fn-execution:2.37.0
org.apache.beam:beam-model-job-management:2.37.0
org.apache.beam:beam-model-pipeline:2.37.0
org.apache.beam:beam-runners-core-construction-java:2.37.0
org.apache.beam:beam-runners-core-java:2.37.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
org.apache.beam:beam-runners-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-core:2.37.0
org.apache.beam:beam-sdks-java-expansion-service:2.37.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
org.apache.beam:beam-sdks-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
org.apache.beam:beam-sdks-java-io-kafka:2.37.0
org.apache.beam:beam-model-fn-execution:2.40.0
org.apache.beam:beam-model-job-management:2.40.0
org.apache.beam:beam-model-pipeline:2.40.0
org.apache.beam:beam-runners-core-construction-java:2.40.0
org.apache.beam:beam-runners-core-java:2.40.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.40.0
org.apache.beam:beam-runners-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-core:2.40.0
org.apache.beam:beam-sdks-java-expansion-service:2.40.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.40.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.40.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.40.0
org.apache.beam:beam-sdks-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0
org.apache.beam:beam-sdks-java-io-kafka:2.40.0
org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
org.apache.beam:beam-vendor-grpc-1_43_2:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-compress:1.21
org.apache.commons:commons-csv:1.9.0
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-lang3:3.12.0
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.15
org.bouncycastle:bcpg-jdk15on:1.67
org.bouncycastle:bcpkix-jdk15on:1.67
org.bouncycastle:bcprov-jdk15on:1.67
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.21.0
org.checkerframework:checker-qual:3.22.2
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.20
org.codehaus.mojo:animal-sniffer-annotations:1.21
org.conscrypt:conscrypt-openjdk-uber:2.5.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.flywaydb:flyway-core:8.5.13
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.gwtproject:gwt-user:2.10.0
org.hamcrest:hamcrest-core:1.3
org.hamcrest:hamcrest:2.1
org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
org.hibernate:hibernate-core:5.4.23.Final
org.hibernate:hibernate-hikaricp:5.4.23.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.4.1.Final
org.hibernate:hibernate-core:5.6.10.Final
org.hibernate:hibernate-hikaricp:5.6.10.Final
org.jboss.logging:jboss-logging:3.4.3.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.1.3.Final
org.jboss:jandex:2.4.2.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20200518
org.jsoup:jsoup:1.13.1
org.jsoup:jsoup:1.15.2
org.jvnet.staxex:stax-ex:1.8
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:9.1
org.ow2.asm:asm-commons:9.0
org.ow2.asm:asm-tree:9.1
org.ow2.asm:asm-util:9.1
org.ow2.asm:asm:9.1
org.postgresql:postgresql:42.2.18
org.ow2.asm:asm-analysis:9.3
org.ow2.asm:asm-commons:9.2
org.ow2.asm:asm-tree:9.3
org.ow2.asm:asm-util:9.3
org.ow2.asm:asm:9.3
org.postgresql:postgresql:42.4.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.slf4j:jcl-over-slf4j:1.7.30
org.slf4j:jul-to-slf4j:1.7.30
org.slf4j:slf4j-api:1.7.30
org.slf4j:slf4j-jdk14:1.7.28
org.springframework:spring-core:4.3.18.RELEASE
org.springframework:spring-expression:4.3.18.RELEASE
org.testcontainers:database-commons:1.15.2
org.testcontainers:jdbc:1.15.2
org.testcontainers:postgresql:1.15.2
org.testcontainers:testcontainers:1.15.2
org.threeten:threetenbp:1.5.2
org.slf4j:slf4j-api:2.0.0-alpha7
org.slf4j:slf4j-jdk14:2.0.0-alpha7
org.springframework:spring-core:5.3.18
org.springframework:spring-expression:5.3.18
org.springframework:spring-jcl:5.3.18
org.testcontainers:database-commons:1.17.3
org.testcontainers:jdbc:1.17.3
org.testcontainers:postgresql:1.17.3
org.testcontainers:testcontainers:1.17.3
org.threeten:threetenbp:1.6.0
org.tukaani:xz:1.5
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3
org.webjars.npm:viz.js-graphviz-java:2.1.3
org.xerial.snappy:snappy-java:1.1.8.4
org.yaml:snakeyaml:1.28
org.yaml:snakeyaml:1.30
us.fatehi:schemacrawler-api:16.10.1
us.fatehi:schemacrawler-diagram:16.10.1
us.fatehi:schemacrawler-tools:16.10.1

View File

@@ -4,301 +4,311 @@
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.101tec:zkclient:0.10
com.beust:jcommander:1.60
com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.13.0
com.fasterxml.jackson.core:jackson-core:2.13.0
com.fasterxml.jackson.core:jackson-databind:2.13.0
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
com.fasterxml.jackson:jackson-bom:2.13.0
com.fasterxml.jackson.core:jackson-annotations:2.13.3
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson.core:jackson-databind:2.13.3
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.fasterxml:classmate:1.5.1
com.github.docker-java:docker-java-api:3.2.7
com.github.docker-java:docker-java-transport-zerodep:3.2.7
com.github.docker-java:docker-java-transport:3.2.7
com.github.jnr:jffi:1.3.1
com.github.ben-manes.caffeine:caffeine:2.9.3
com.github.docker-java:docker-java-api:3.2.13
com.github.docker-java:docker-java-transport-zerodep:3.2.13
com.github.docker-java:docker-java-transport:3.2.13
com.github.jnr:jffi:1.3.9
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.10.1
com.github.jnr:jnr-enxio:0.32.3
com.github.jnr:jnr-ffi:2.2.1
com.github.jnr:jnr-posix:3.1.4
com.github.jnr:jnr-unixsocket:0.38.5
com.github.jnr:jnr-constants:0.10.3
com.github.jnr:jnr-enxio:0.32.13
com.github.jnr:jnr-ffi:2.2.11
com.github.jnr:jnr-posix:3.1.15
com.github.jnr:jnr-unixsocket:0.38.17
com.github.jnr:jnr-x86asm:1.0.2
com.google.android:annotations:4.1.1.4
com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-appengine:1.35.2
com.google.api-client:google-api-client-jackson2:1.32.2
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.32.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api-client:google-api-client-java6:1.35.2
com.google.api-client:google-api-client-servlet:1.35.2
com.google.api-client:google-api-client:1.35.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:grpc-google-common-protos:2.7.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
com.google.api.grpc:proto-google-common-protos:2.7.1
com.google.api.grpc:proto-google-iam-v1:1.2.0
com.google.api:api-common:2.1.2
com.google.api:gax-grpc:2.8.1
com.google.api:gax-httpjson:0.93.1
com.google.api:gax:2.8.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:grpc-google-common-protos:2.8.3
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.6.2
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.10
com.google.api.grpc:proto-google-cloud-firestore-v1:3.1.0
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.3.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.3.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:proto-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:2.3.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.93.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.93.0
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-grpc:2.18.2
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
com.google.apis:google-api-services-cloudkms:v1-rev20220617-1.32.1
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
com.google.apis:google-api-services-drive:v2-rev393-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev20210624-1.32.1
com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-monitoring:v3-rev20220626-1.32.1
com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220513-1.32.1
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.86
com.google.appengine:appengine-remote-api:1.9.86
com.google.appengine:appengine-api-1.0-sdk:2.0.5
com.google.appengine:appengine-remote-api:2.0.5
com.google.appengine:appengine-testing:1.9.86
com.google.auth:google-auth-library-credentials:1.3.0
com.google.auth:google-auth-library-oauth2-http:1.3.0
com.google.auto.service:auto-service-annotations:1.0-rc7
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.service:auto-service-annotations:1.0.1
com.google.auto.value:auto-value-annotations:1.9
com.google.auto.value:auto-value:1.7.4
com.google.cloud.bigdataoss:gcsio:2.2.4
com.google.cloud.bigdataoss:util:2.2.4
com.google.cloud.bigtable:bigtable-client-core:1.25.1
com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
com.google.cloud.sql:postgres-socket-factory:1.2.1
com.google.cloud:google-cloud-bigquerystorage:2.8.1
com.google.cloud:google-cloud-bigtable:1.27.1
com.google.cloud:google-cloud-core-grpc:2.3.5
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:2.3.5
com.google.cloud:google-cloud-firestore:3.0.10
com.google.cloud:google-cloud-pubsub:1.115.1
com.google.cloud:google-cloud-pubsublite:1.4.8
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:6.17.4
com.google.cloud:google-cloud-storage:1.113.12
com.google.cloud:google-cloud-tasks:1.33.2
com.google.auto.value:auto-value:1.9
com.google.cloud.bigdataoss:gcsio:2.2.6
com.google.cloud.bigdataoss:util:2.2.6
com.google.cloud.bigtable:bigtable-client-core:1.26.3
com.google.cloud.bigtable:bigtable-metrics-api:1.26.3
com.google.cloud.datastore:datastore-v1-proto-client:2.2.10
com.google.cloud.sql:jdbc-socket-factory-core:1.6.1
com.google.cloud.sql:postgres-socket-factory:1.6.1
com.google.cloud:google-cloud-bigquerystorage:2.12.2
com.google.cloud:google-cloud-bigtable:2.6.2
com.google.cloud:google-cloud-core-grpc:2.6.0
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-firestore:3.1.0
com.google.cloud:google-cloud-monitoring:1.82.0
com.google.cloud:google-cloud-pubsub:1.116.4
com.google.cloud:google-cloud-pubsublite:1.5.4
com.google.cloud:google-cloud-secretmanager:2.3.0
com.google.cloud:google-cloud-spanner:6.23.3
com.google.cloud:google-cloud-storage:2.9.2
com.google.cloud:google-cloud-tasks:2.3.0
com.google.cloud:grpc-gcp:1.1.0
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.1.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.9
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.10.0
com.google.dagger:dagger:2.42
com.google.errorprone:error_prone_annotations:2.14.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flatbuffers:flatbuffers-java:1.12.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.flogger:google-extensions:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:31.0.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.9.0
com.google.http-client:google-http-client-apache-v2:1.40.1
com.google.http-client:google-http-client-appengine:1.39.2
com.google.http-client:google-http-client-gson:1.41.0
com.google.http-client:google-http-client-jackson2:1.41.0
com.google.http-client:google-http-client-protobuf:1.40.1
com.google.http-client:google-http-client:1.41.0
com.google.gwt:gwt-user:2.10.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client-protobuf:1.41.7
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:2.0.0
com.google.monitoring-client:metrics:1.0.7
com.google.monitoring-client:stackdriver:1.0.7
com.google.oauth-client:google-oauth-client-appengine:1.31.4
com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.32.1
com.google.protobuf:protobuf-java-util:3.19.2
com.google.protobuf:protobuf-java:3.19.2
com.google.re2j:re2j:1.6
com.google.oauth-client:google-oauth-client-appengine:1.34.1
com.google.oauth-client:google-oauth-client-java6:1.34.1
com.google.oauth-client:google-oauth-client-jetty:1.34.1
com.google.oauth-client:google-oauth-client-servlet:1.34.1
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.re2j:re2j:1.7
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
com.ibm.icu:icu4j:71.1
com.jcraft:jsch:0.1.55
com.lmax:disruptor:3.4.2
com.sun.istack:istack-commons-runtime:3.0.7
com.sun.xml.fastinfoset:FastInfoset:1.2.15
com.thoughtworks.paranamer:paranamer:2.7
com.zaxxer:HikariCP:3.4.5
commons-codec:commons-codec:1.15
commons-logging:commons-logging:1.2
dnsjava:dnsjava:3.3.1
dnsjava:dnsjava:3.5.1
guru.nidi.com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
guru.nidi.com.kitfox:svgSalamander:1.1.3
guru.nidi:graphviz-java-all-j2v8:0.17.0
guru.nidi:graphviz-java:0.17.0
guru.nidi:graphviz-java-all-j2v8:0.18.1
guru.nidi:graphviz-java:0.18.1
io.confluent:common-config:5.3.2
io.confluent:common-utils:5.3.2
io.confluent:kafka-avro-serializer:5.3.2
io.confluent:kafka-schema-registry-client:5.3.2
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.104
io.grpc:grpc-alts:1.43.2
io.grpc:grpc-api:1.43.2
io.grpc:grpc-auth:1.43.2
io.grpc:grpc-context:1.43.2
io.grpc:grpc-core:1.43.2
io.grpc:grpc-grpclb:1.43.2
io.grpc:grpc-netty-shaded:1.43.2
io.grpc:grpc-netty:1.43.2
io.grpc:grpc-protobuf-lite:1.43.2
io.grpc:grpc-protobuf:1.43.2
io.grpc:grpc-services:1.43.2
io.grpc:grpc-stub:1.43.2
io.grpc:grpc-xds:1.43.2
io.netty:netty-buffer:4.1.63.Final
io.netty:netty-codec-http2:4.1.63.Final
io.netty:netty-codec-http:4.1.63.Final
io.netty:netty-codec-socks:4.1.63.Final
io.netty:netty-codec:4.1.63.Final
io.netty:netty-common:4.1.63.Final
io.netty:netty-handler-proxy:4.1.63.Final
io.netty:netty-handler:4.1.63.Final
io.netty:netty-resolver:4.1.63.Final
io.netty:netty-tcnative-boringssl-static:2.0.33.Final
io.netty:netty-transport:4.1.63.Final
io.opencensus:opencensus-api:0.30.0
io.opencensus:opencensus-contrib-grpc-util:0.30.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-alts:1.47.0
io.grpc:grpc-api:1.47.0
io.grpc:grpc-auth:1.47.0
io.grpc:grpc-census:1.45.1
io.grpc:grpc-context:1.47.0
io.grpc:grpc-core:1.47.0
io.grpc:grpc-googleapis:1.47.0
io.grpc:grpc-grpclb:1.47.0
io.grpc:grpc-netty-shaded:1.47.0
io.grpc:grpc-netty:1.45.1
io.grpc:grpc-protobuf-lite:1.47.0
io.grpc:grpc-protobuf:1.47.0
io.grpc:grpc-services:1.47.0
io.grpc:grpc-stub:1.47.0
io.grpc:grpc-xds:1.47.0
io.netty:netty-buffer:4.1.72.Final
io.netty:netty-codec-http2:4.1.72.Final
io.netty:netty-codec-http:4.1.72.Final
io.netty:netty-codec-socks:4.1.72.Final
io.netty:netty-codec:4.1.72.Final
io.netty:netty-common:4.1.72.Final
io.netty:netty-handler-proxy:4.1.72.Final
io.netty:netty-handler:4.1.72.Final
io.netty:netty-resolver:4.1.72.Final
io.netty:netty-tcnative-boringssl-static:2.0.46.Final
io.netty:netty-tcnative-classes:2.0.46.Final
io.netty:netty-transport:4.1.72.Final
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-exemplar-util:0.31.0
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0
io.opencensus:opencensus-contrib-grpc-util:0.31.0
io.opencensus:opencensus-contrib-http-util:0.31.1
io.opencensus:opencensus-contrib-resource-util:0.31.0
io.opencensus:opencensus-exporter-metrics-util:0.31.0
io.opencensus:opencensus-exporter-stats-stackdriver:0.31.0
io.opencensus:opencensus-impl-core:0.31.0
io.opencensus:opencensus-impl:0.31.0
io.opencensus:opencensus-proto:0.2.0
io.perfmark:perfmark-api:0.23.0
it.unimi.dsi:fastutil:6.5.16
io.perfmark:perfmark-api:0.25.0
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.jdo:jdo2-api:2.3-20090302111651
javax.mail:mail:1.5.0-b01
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.10
net.arnx:nashorn-promise:0.1.1
net.bytebuddy:byte-buddy:1.10.17
net.java.dev.jna:jna:5.5.0
net.bytebuddy:byte-buddy:1.12.9
net.java.dev.jna:jna:5.8.0
org.apache.arrow:arrow-format:5.0.0
org.apache.arrow:arrow-memory-core:5.0.0
org.apache.arrow:arrow-vector:5.0.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-fn-execution:2.37.0
org.apache.beam:beam-model-job-management:2.37.0
org.apache.beam:beam-model-pipeline:2.37.0
org.apache.beam:beam-runners-core-construction-java:2.37.0
org.apache.beam:beam-runners-core-java:2.37.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
org.apache.beam:beam-runners-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-core:2.37.0
org.apache.beam:beam-sdks-java-expansion-service:2.37.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
org.apache.beam:beam-sdks-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
org.apache.beam:beam-sdks-java-io-kafka:2.37.0
org.apache.beam:beam-model-fn-execution:2.40.0
org.apache.beam:beam-model-job-management:2.40.0
org.apache.beam:beam-model-pipeline:2.40.0
org.apache.beam:beam-runners-core-construction-java:2.40.0
org.apache.beam:beam-runners-core-java:2.40.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.40.0
org.apache.beam:beam-runners-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-core:2.40.0
org.apache.beam:beam-sdks-java-expansion-service:2.40.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.40.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.40.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.40.0
org.apache.beam:beam-sdks-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0
org.apache.beam:beam-sdks-java-io-kafka:2.40.0
org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
org.apache.beam:beam-vendor-grpc-1_43_2:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-compress:1.21
org.apache.commons:commons-csv:1.9.0
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-lang3:3.12.0
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.15
org.bouncycastle:bcpg-jdk15on:1.67
org.bouncycastle:bcpkix-jdk15on:1.67
org.bouncycastle:bcprov-jdk15on:1.67
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.21.0
org.checkerframework:checker-qual:3.22.2
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.20
org.codehaus.mojo:animal-sniffer-annotations:1.21
org.conscrypt:conscrypt-openjdk-uber:2.5.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.flywaydb:flyway-core:8.5.13
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.gwtproject:gwt-user:2.10.0
org.hamcrest:hamcrest:2.1
org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
org.hibernate:hibernate-core:5.4.23.Final
org.hibernate:hibernate-hikaricp:5.4.23.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.4.1.Final
org.hibernate:hibernate-core:5.6.10.Final
org.hibernate:hibernate-hikaricp:5.6.10.Final
org.jboss.logging:jboss-logging:3.4.3.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.1.3.Final
org.jboss:jandex:2.4.2.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20200518
org.jsoup:jsoup:1.13.1
org.jsoup:jsoup:1.15.2
org.jvnet.staxex:stax-ex:1.8
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:9.1
org.ow2.asm:asm-commons:9.0
org.ow2.asm:asm-tree:9.1
org.ow2.asm:asm-util:9.1
org.ow2.asm:asm:9.1
org.postgresql:postgresql:42.2.18
org.ow2.asm:asm-analysis:9.3
org.ow2.asm:asm-commons:9.2
org.ow2.asm:asm-tree:9.3
org.ow2.asm:asm-util:9.3
org.ow2.asm:asm:9.3
org.postgresql:postgresql:42.4.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.slf4j:jcl-over-slf4j:1.7.30
org.slf4j:jul-to-slf4j:1.7.30
org.slf4j:slf4j-api:1.7.30
org.slf4j:slf4j-jdk14:1.7.28
org.springframework:spring-core:4.3.18.RELEASE
org.springframework:spring-expression:4.3.18.RELEASE
org.testcontainers:database-commons:1.15.2
org.testcontainers:jdbc:1.15.2
org.testcontainers:postgresql:1.15.2
org.testcontainers:testcontainers:1.15.2
org.threeten:threetenbp:1.5.2
org.slf4j:slf4j-api:2.0.0-alpha7
org.slf4j:slf4j-jdk14:2.0.0-alpha7
org.springframework:spring-core:5.3.18
org.springframework:spring-expression:5.3.18
org.springframework:spring-jcl:5.3.18
org.testcontainers:database-commons:1.17.3
org.testcontainers:jdbc:1.17.3
org.testcontainers:postgresql:1.17.3
org.testcontainers:testcontainers:1.17.3
org.threeten:threetenbp:1.6.0
org.tukaani:xz:1.5
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3
org.webjars.npm:viz.js-graphviz-java:2.1.3
org.xerial.snappy:snappy-java:1.1.8.4
org.yaml:snakeyaml:1.28
org.yaml:snakeyaml:1.30
us.fatehi:schemacrawler-api:16.10.1
us.fatehi:schemacrawler-diagram:16.10.1
us.fatehi:schemacrawler-tools:16.10.1

View File

@@ -4,7 +4,9 @@
com.sun.activation:jakarta.activation:1.2.2
com.sun.activation:javax.activation:1.2.0
com.sun.xml.bind:jaxb-impl:2.3.3
com.sun.xml.bind:jaxb-osgi:2.3.3
com.sun.xml.bind:jaxb-osgi:4.0.0
com.sun.xml.bind:jaxb-xjc:2.3.3
jakarta.xml.bind:jakarta.xml.bind-api:2.3.3
javax.xml.bind:jaxb-api:2.3.0
jakarta.activation:jakarta.activation-api:2.1.0
jakarta.xml.bind:jakarta.xml.bind-api:4.0.0
javax.activation:javax.activation-api:1.2.0
javax.xml.bind:jaxb-api:2.4.0-b180830.0359

View File

@@ -4,289 +4,299 @@
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.101tec:zkclient:0.10
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.13.0
com.fasterxml.jackson.core:jackson-core:2.13.0
com.fasterxml.jackson.core:jackson-databind:2.13.0
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
com.fasterxml.jackson:jackson-bom:2.13.0
com.fasterxml.jackson.core:jackson-annotations:2.13.3
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson.core:jackson-databind:2.13.3
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.fasterxml:classmate:1.5.1
com.github.docker-java:docker-java-api:3.2.7
com.github.docker-java:docker-java-transport-zerodep:3.2.7
com.github.docker-java:docker-java-transport:3.2.7
com.github.jnr:jffi:1.3.1
com.github.ben-manes.caffeine:caffeine:2.9.3
com.github.docker-java:docker-java-api:3.2.13
com.github.docker-java:docker-java-transport-zerodep:3.2.13
com.github.docker-java:docker-java-transport:3.2.13
com.github.jnr:jffi:1.3.9
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.10.1
com.github.jnr:jnr-enxio:0.32.3
com.github.jnr:jnr-ffi:2.2.1
com.github.jnr:jnr-posix:3.1.4
com.github.jnr:jnr-unixsocket:0.38.5
com.github.jnr:jnr-constants:0.10.3
com.github.jnr:jnr-enxio:0.32.13
com.github.jnr:jnr-ffi:2.2.11
com.github.jnr:jnr-posix:3.1.15
com.github.jnr:jnr-unixsocket:0.38.17
com.github.jnr:jnr-x86asm:1.0.2
com.google.android:annotations:4.1.1.4
com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-appengine:1.35.2
com.google.api-client:google-api-client-jackson2:1.32.2
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.32.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api-client:google-api-client-java6:1.35.2
com.google.api-client:google-api-client-servlet:1.35.2
com.google.api-client:google-api-client:1.35.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:grpc-google-common-protos:2.7.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
com.google.api.grpc:proto-google-common-protos:2.7.1
com.google.api.grpc:proto-google-iam-v1:1.2.0
com.google.api:api-common:2.1.2
com.google.api:gax-grpc:2.8.1
com.google.api:gax-httpjson:0.93.1
com.google.api:gax:2.8.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:grpc-google-common-protos:2.8.3
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.6.2
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.10
com.google.api.grpc:proto-google-cloud-firestore-v1:3.1.0
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.3.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.3.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:proto-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:2.3.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.93.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.93.0
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-grpc:2.18.2
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
com.google.apis:google-api-services-cloudkms:v1-rev20220617-1.32.1
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
com.google.apis:google-api-services-drive:v2-rev393-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev20210624-1.32.1
com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-monitoring:v3-rev20220626-1.32.1
com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220513-1.32.1
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.86
com.google.appengine:appengine-remote-api:1.9.86
com.google.appengine:appengine-api-1.0-sdk:2.0.5
com.google.appengine:appengine-remote-api:2.0.5
com.google.appengine:appengine-testing:1.9.86
com.google.auth:google-auth-library-credentials:1.3.0
com.google.auth:google-auth-library-oauth2-http:1.3.0
com.google.auto.service:auto-service-annotations:1.0-rc7
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.service:auto-service-annotations:1.0.1
com.google.auto.value:auto-value-annotations:1.9
com.google.auto.value:auto-value:1.7.4
com.google.cloud.bigdataoss:gcsio:2.2.4
com.google.cloud.bigdataoss:util:2.2.4
com.google.cloud.bigtable:bigtable-client-core:1.25.1
com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
com.google.cloud:google-cloud-bigquerystorage:2.8.1
com.google.cloud:google-cloud-bigtable:1.27.1
com.google.cloud:google-cloud-core-grpc:2.3.5
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:2.3.5
com.google.cloud:google-cloud-firestore:3.0.10
com.google.cloud:google-cloud-pubsub:1.115.1
com.google.cloud:google-cloud-pubsublite:1.4.8
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:6.17.4
com.google.cloud:google-cloud-storage:1.113.12
com.google.cloud:google-cloud-tasks:1.33.2
com.google.auto.value:auto-value:1.9
com.google.cloud.bigdataoss:gcsio:2.2.6
com.google.cloud.bigdataoss:util:2.2.6
com.google.cloud.bigtable:bigtable-client-core:1.26.3
com.google.cloud.bigtable:bigtable-metrics-api:1.26.3
com.google.cloud.datastore:datastore-v1-proto-client:2.2.10
com.google.cloud.sql:jdbc-socket-factory-core:1.6.1
com.google.cloud:google-cloud-bigquerystorage:2.12.2
com.google.cloud:google-cloud-bigtable:2.6.2
com.google.cloud:google-cloud-core-grpc:2.6.0
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-firestore:3.1.0
com.google.cloud:google-cloud-monitoring:1.82.0
com.google.cloud:google-cloud-pubsub:1.116.4
com.google.cloud:google-cloud-pubsublite:1.5.4
com.google.cloud:google-cloud-secretmanager:2.3.0
com.google.cloud:google-cloud-spanner:6.23.3
com.google.cloud:google-cloud-storage:2.9.2
com.google.cloud:google-cloud-tasks:2.3.0
com.google.cloud:grpc-gcp:1.1.0
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.1.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.9
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.10.0
com.google.dagger:dagger:2.42
com.google.errorprone:error_prone_annotations:2.14.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flatbuffers:flatbuffers-java:1.12.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.flogger:google-extensions:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:31.0.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.9.0
com.google.http-client:google-http-client-apache-v2:1.40.1
com.google.http-client:google-http-client-appengine:1.39.2
com.google.http-client:google-http-client-gson:1.41.0
com.google.http-client:google-http-client-jackson2:1.41.0
com.google.http-client:google-http-client-protobuf:1.40.1
com.google.http-client:google-http-client:1.41.0
com.google.gwt:gwt-user:2.10.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client-protobuf:1.41.7
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:2.0.0
com.google.monitoring-client:metrics:1.0.7
com.google.monitoring-client:stackdriver:1.0.7
com.google.oauth-client:google-oauth-client-appengine:1.31.4
com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.32.1
com.google.protobuf:protobuf-java-util:3.19.2
com.google.protobuf:protobuf-java:3.19.2
com.google.re2j:re2j:1.6
com.google.oauth-client:google-oauth-client-appengine:1.34.1
com.google.oauth-client:google-oauth-client-java6:1.34.1
com.google.oauth-client:google-oauth-client-jetty:1.34.1
com.google.oauth-client:google-oauth-client-servlet:1.34.1
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.re2j:re2j:1.7
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
com.ibm.icu:icu4j:71.1
com.jcraft:jsch:0.1.55
com.lmax:disruptor:3.4.2
com.sun.istack:istack-commons-runtime:3.0.7
com.sun.xml.fastinfoset:FastInfoset:1.2.15
com.thoughtworks.paranamer:paranamer:2.7
com.zaxxer:HikariCP:3.4.5
commons-codec:commons-codec:1.15
commons-logging:commons-logging:1.2
dnsjava:dnsjava:3.3.1
dnsjava:dnsjava:3.5.1
io.confluent:common-config:5.3.2
io.confluent:common-utils:5.3.2
io.confluent:kafka-avro-serializer:5.3.2
io.confluent:kafka-schema-registry-client:5.3.2
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.104
io.grpc:grpc-alts:1.43.2
io.grpc:grpc-api:1.43.2
io.grpc:grpc-auth:1.43.2
io.grpc:grpc-context:1.43.2
io.grpc:grpc-core:1.43.2
io.grpc:grpc-grpclb:1.43.2
io.grpc:grpc-netty-shaded:1.43.2
io.grpc:grpc-netty:1.43.2
io.grpc:grpc-protobuf-lite:1.43.2
io.grpc:grpc-protobuf:1.43.2
io.grpc:grpc-services:1.43.2
io.grpc:grpc-stub:1.43.2
io.grpc:grpc-xds:1.43.2
io.netty:netty-buffer:4.1.63.Final
io.netty:netty-codec-http2:4.1.63.Final
io.netty:netty-codec-http:4.1.63.Final
io.netty:netty-codec-socks:4.1.63.Final
io.netty:netty-codec:4.1.63.Final
io.netty:netty-common:4.1.63.Final
io.netty:netty-handler-proxy:4.1.63.Final
io.netty:netty-handler:4.1.63.Final
io.netty:netty-resolver:4.1.63.Final
io.netty:netty-tcnative-boringssl-static:2.0.33.Final
io.netty:netty-transport:4.1.63.Final
io.opencensus:opencensus-api:0.30.0
io.opencensus:opencensus-contrib-grpc-util:0.30.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-alts:1.47.0
io.grpc:grpc-api:1.47.0
io.grpc:grpc-auth:1.47.0
io.grpc:grpc-census:1.45.1
io.grpc:grpc-context:1.47.0
io.grpc:grpc-core:1.47.0
io.grpc:grpc-googleapis:1.47.0
io.grpc:grpc-grpclb:1.47.0
io.grpc:grpc-netty-shaded:1.47.0
io.grpc:grpc-netty:1.45.1
io.grpc:grpc-protobuf-lite:1.47.0
io.grpc:grpc-protobuf:1.47.0
io.grpc:grpc-services:1.47.0
io.grpc:grpc-stub:1.47.0
io.grpc:grpc-xds:1.47.0
io.netty:netty-buffer:4.1.72.Final
io.netty:netty-codec-http2:4.1.72.Final
io.netty:netty-codec-http:4.1.72.Final
io.netty:netty-codec-socks:4.1.72.Final
io.netty:netty-codec:4.1.72.Final
io.netty:netty-common:4.1.72.Final
io.netty:netty-handler-proxy:4.1.72.Final
io.netty:netty-handler:4.1.72.Final
io.netty:netty-resolver:4.1.72.Final
io.netty:netty-tcnative-boringssl-static:2.0.46.Final
io.netty:netty-tcnative-classes:2.0.46.Final
io.netty:netty-transport:4.1.72.Final
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-exemplar-util:0.31.0
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0
io.opencensus:opencensus-contrib-grpc-util:0.31.0
io.opencensus:opencensus-contrib-http-util:0.31.1
io.opencensus:opencensus-contrib-resource-util:0.31.0
io.opencensus:opencensus-exporter-metrics-util:0.31.0
io.opencensus:opencensus-exporter-stats-stackdriver:0.31.0
io.opencensus:opencensus-impl-core:0.31.0
io.opencensus:opencensus-impl:0.31.0
io.opencensus:opencensus-proto:0.2.0
io.perfmark:perfmark-api:0.23.0
it.unimi.dsi:fastutil:6.5.16
io.perfmark:perfmark-api:0.25.0
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.jdo:jdo2-api:2.3-20090302111651
javax.mail:mail:1.5.0-b01
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.10
junit:junit:4.13.1
net.bytebuddy:byte-buddy:1.10.17
net.java.dev.jna:jna:5.5.0
junit:junit:4.13.2
net.bytebuddy:byte-buddy:1.12.9
net.java.dev.jna:jna:5.8.0
org.apache.arrow:arrow-format:5.0.0
org.apache.arrow:arrow-memory-core:5.0.0
org.apache.arrow:arrow-vector:5.0.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-fn-execution:2.37.0
org.apache.beam:beam-model-job-management:2.37.0
org.apache.beam:beam-model-pipeline:2.37.0
org.apache.beam:beam-runners-core-construction-java:2.37.0
org.apache.beam:beam-runners-core-java:2.37.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
org.apache.beam:beam-runners-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-core:2.37.0
org.apache.beam:beam-sdks-java-expansion-service:2.37.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
org.apache.beam:beam-sdks-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
org.apache.beam:beam-sdks-java-io-kafka:2.37.0
org.apache.beam:beam-model-fn-execution:2.40.0
org.apache.beam:beam-model-job-management:2.40.0
org.apache.beam:beam-model-pipeline:2.40.0
org.apache.beam:beam-runners-core-construction-java:2.40.0
org.apache.beam:beam-runners-core-java:2.40.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.40.0
org.apache.beam:beam-runners-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-core:2.40.0
org.apache.beam:beam-sdks-java-expansion-service:2.40.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.40.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.40.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.40.0
org.apache.beam:beam-sdks-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0
org.apache.beam:beam-sdks-java-io-kafka:2.40.0
org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
org.apache.beam:beam-vendor-grpc-1_43_2:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-compress:1.21
org.apache.commons:commons-csv:1.9.0
org.apache.commons:commons-lang3:3.12.0
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.15
org.bouncycastle:bcpg-jdk15on:1.67
org.bouncycastle:bcpkix-jdk15on:1.67
org.bouncycastle:bcprov-jdk15on:1.67
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.21.0
org.checkerframework:checker-qual:3.22.2
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.20
org.codehaus.mojo:animal-sniffer-annotations:1.21
org.conscrypt:conscrypt-openjdk-uber:2.5.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.flywaydb:flyway-core:8.5.13
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.gwtproject:gwt-user:2.10.0
org.hamcrest:hamcrest-core:1.3
org.hamcrest:hamcrest:2.1
org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
org.hibernate:hibernate-core:5.4.23.Final
org.hibernate:hibernate-hikaricp:5.4.23.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.4.1.Final
org.hibernate:hibernate-core:5.6.10.Final
org.hibernate:hibernate-hikaricp:5.6.10.Final
org.jboss.logging:jboss-logging:3.4.3.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.1.3.Final
org.jboss:jandex:2.4.2.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20200518
org.jsoup:jsoup:1.13.1
org.jsoup:jsoup:1.15.2
org.jvnet.staxex:stax-ex:1.8
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:9.1
org.ow2.asm:asm-commons:9.0
org.ow2.asm:asm-tree:9.1
org.ow2.asm:asm-util:9.1
org.ow2.asm:asm:9.1
org.postgresql:postgresql:42.2.18
org.ow2.asm:asm-analysis:9.3
org.ow2.asm:asm-commons:9.2
org.ow2.asm:asm-tree:9.3
org.ow2.asm:asm-util:9.3
org.ow2.asm:asm:9.3
org.postgresql:postgresql:42.4.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.slf4j:slf4j-api:1.7.30
org.springframework:spring-core:4.3.18.RELEASE
org.springframework:spring-expression:4.3.18.RELEASE
org.testcontainers:database-commons:1.15.2
org.testcontainers:jdbc:1.15.2
org.testcontainers:postgresql:1.15.2
org.testcontainers:testcontainers:1.15.2
org.threeten:threetenbp:1.5.2
org.slf4j:slf4j-api:1.7.36
org.springframework:spring-core:5.3.18
org.springframework:spring-expression:5.3.18
org.springframework:spring-jcl:5.3.18
org.testcontainers:database-commons:1.17.3
org.testcontainers:jdbc:1.17.3
org.testcontainers:postgresql:1.17.3
org.testcontainers:testcontainers:1.17.3
org.threeten:threetenbp:1.6.0
org.tukaani:xz:1.5
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.8.4
org.yaml:snakeyaml:1.28
org.yaml:snakeyaml:1.30
us.fatehi:schemacrawler-api:16.10.1
us.fatehi:schemacrawler-diagram:16.10.1
us.fatehi:schemacrawler-tools:16.10.1

View File

@@ -4,281 +4,293 @@
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.101tec:zkclient:0.10
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.13.0
com.fasterxml.jackson.core:jackson-core:2.13.0
com.fasterxml.jackson.core:jackson-databind:2.13.0
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
com.fasterxml.jackson:jackson-bom:2.13.0
com.fasterxml.jackson.core:jackson-annotations:2.13.3
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson.core:jackson-databind:2.13.3
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.fasterxml:classmate:1.5.1
com.github.docker-java:docker-java-api:3.2.7
com.github.docker-java:docker-java-transport-zerodep:3.2.7
com.github.docker-java:docker-java-transport:3.2.7
com.github.jnr:jffi:1.3.1
com.github.ben-manes.caffeine:caffeine:2.9.3
com.github.docker-java:docker-java-api:3.2.13
com.github.docker-java:docker-java-transport-zerodep:3.2.13
com.github.docker-java:docker-java-transport:3.2.13
com.github.jnr:jffi:1.3.9
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.10.1
com.github.jnr:jnr-enxio:0.32.3
com.github.jnr:jnr-ffi:2.2.1
com.github.jnr:jnr-posix:3.1.4
com.github.jnr:jnr-unixsocket:0.38.5
com.github.jnr:jnr-constants:0.10.3
com.github.jnr:jnr-enxio:0.32.13
com.github.jnr:jnr-ffi:2.2.11
com.github.jnr:jnr-posix:3.1.15
com.github.jnr:jnr-unixsocket:0.38.17
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-appengine:1.35.2
com.google.api-client:google-api-client-jackson2:1.32.2
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.32.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api-client:google-api-client-java6:1.35.2
com.google.api-client:google-api-client-servlet:1.35.2
com.google.api-client:google-api-client:1.35.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:grpc-google-common-protos:2.7.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
com.google.api.grpc:proto-google-common-protos:2.7.1
com.google.api.grpc:proto-google-iam-v1:1.2.0
com.google.api:api-common:2.1.2
com.google.api:gax-grpc:2.8.1
com.google.api:gax-httpjson:0.93.1
com.google.api:gax:2.8.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:grpc-google-common-protos:2.8.3
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.6.2
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.10
com.google.api.grpc:proto-google-cloud-firestore-v1:3.1.0
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.3.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.3.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:proto-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:2.3.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.93.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.93.0
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-grpc:2.18.2
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
com.google.apis:google-api-services-cloudkms:v1-rev20220617-1.32.1
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
com.google.apis:google-api-services-drive:v2-rev393-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev20210624-1.32.1
com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-monitoring:v3-rev20220626-1.32.1
com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220513-1.32.1
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.86
com.google.appengine:appengine-remote-api:1.9.86
com.google.appengine:appengine-api-1.0-sdk:2.0.5
com.google.appengine:appengine-remote-api:2.0.5
com.google.appengine:appengine-testing:1.9.86
com.google.auth:google-auth-library-credentials:1.3.0
com.google.auth:google-auth-library-oauth2-http:1.3.0
com.google.auto.service:auto-service-annotations:1.0-rc7
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.service:auto-service-annotations:1.0.1
com.google.auto.value:auto-value-annotations:1.9
com.google.auto.value:auto-value:1.7.4
com.google.cloud.bigdataoss:gcsio:2.2.4
com.google.cloud.bigdataoss:util:2.2.4
com.google.cloud.bigtable:bigtable-client-core:1.25.1
com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
com.google.cloud:google-cloud-bigquerystorage:2.8.1
com.google.cloud:google-cloud-bigtable:1.27.1
com.google.cloud:google-cloud-core-grpc:2.3.5
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:2.3.5
com.google.cloud:google-cloud-firestore:3.0.10
com.google.cloud:google-cloud-pubsub:1.115.1
com.google.cloud:google-cloud-pubsublite:1.4.8
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:6.17.4
com.google.cloud:google-cloud-storage:1.113.12
com.google.cloud:google-cloud-tasks:1.33.2
com.google.auto.value:auto-value:1.9
com.google.cloud.bigdataoss:gcsio:2.2.6
com.google.cloud.bigdataoss:util:2.2.6
com.google.cloud.bigtable:bigtable-client-core:1.26.3
com.google.cloud.bigtable:bigtable-metrics-api:1.26.3
com.google.cloud.datastore:datastore-v1-proto-client:2.2.10
com.google.cloud.sql:jdbc-socket-factory-core:1.6.1
com.google.cloud:google-cloud-bigquerystorage:2.12.2
com.google.cloud:google-cloud-bigtable:2.6.2
com.google.cloud:google-cloud-core-grpc:2.6.0
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-firestore:3.1.0
com.google.cloud:google-cloud-monitoring:1.82.0
com.google.cloud:google-cloud-pubsub:1.116.4
com.google.cloud:google-cloud-pubsublite:1.5.4
com.google.cloud:google-cloud-secretmanager:2.3.0
com.google.cloud:google-cloud-spanner:6.23.3
com.google.cloud:google-cloud-storage:2.9.2
com.google.cloud:google-cloud-tasks:2.3.0
com.google.cloud:grpc-gcp:1.1.0
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.1.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.9
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.10.0
com.google.dagger:dagger:2.42
com.google.errorprone:error_prone_annotations:2.14.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flatbuffers:flatbuffers-java:1.12.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.flogger:google-extensions:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:31.0.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.9.0
com.google.http-client:google-http-client-apache-v2:1.40.1
com.google.http-client:google-http-client-appengine:1.39.2
com.google.http-client:google-http-client-gson:1.41.0
com.google.http-client:google-http-client-jackson2:1.41.0
com.google.http-client:google-http-client-protobuf:1.40.1
com.google.http-client:google-http-client:1.41.0
com.google.gwt:gwt-user:2.10.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client-protobuf:1.41.7
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:2.0.0
com.google.monitoring-client:metrics:1.0.7
com.google.monitoring-client:stackdriver:1.0.7
com.google.oauth-client:google-oauth-client-appengine:1.31.4
com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.32.1
com.google.protobuf:protobuf-java-util:3.19.2
com.google.protobuf:protobuf-java:3.19.2
com.google.re2j:re2j:1.6
com.google.oauth-client:google-oauth-client-appengine:1.34.1
com.google.oauth-client:google-oauth-client-java6:1.34.1
com.google.oauth-client:google-oauth-client-jetty:1.34.1
com.google.oauth-client:google-oauth-client-servlet:1.34.1
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.re2j:re2j:1.7
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
com.ibm.icu:icu4j:71.1
com.jcraft:jsch:0.1.55
com.lmax:disruptor:3.4.2
com.sun.istack:istack-commons-runtime:3.0.7
com.sun.xml.fastinfoset:FastInfoset:1.2.15
com.thoughtworks.paranamer:paranamer:2.7
com.zaxxer:HikariCP:3.4.5
commons-codec:commons-codec:1.15
commons-logging:commons-logging:1.2
dnsjava:dnsjava:3.3.1
dnsjava:dnsjava:3.5.1
io.confluent:common-config:5.3.2
io.confluent:common-utils:5.3.2
io.confluent:kafka-avro-serializer:5.3.2
io.confluent:kafka-schema-registry-client:5.3.2
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.104
io.grpc:grpc-alts:1.43.2
io.grpc:grpc-api:1.43.2
io.grpc:grpc-auth:1.43.2
io.grpc:grpc-context:1.43.2
io.grpc:grpc-core:1.43.2
io.grpc:grpc-grpclb:1.43.2
io.grpc:grpc-netty-shaded:1.43.2
io.grpc:grpc-netty:1.43.2
io.grpc:grpc-protobuf-lite:1.43.2
io.grpc:grpc-protobuf:1.43.2
io.grpc:grpc-stub:1.43.2
io.netty:netty-buffer:4.1.63.Final
io.netty:netty-codec-http2:4.1.63.Final
io.netty:netty-codec-http:4.1.63.Final
io.netty:netty-codec:4.1.63.Final
io.netty:netty-common:4.1.63.Final
io.netty:netty-handler:4.1.63.Final
io.netty:netty-resolver:4.1.63.Final
io.netty:netty-tcnative-boringssl-static:2.0.33.Final
io.netty:netty-transport:4.1.63.Final
io.opencensus:opencensus-api:0.30.0
io.opencensus:opencensus-contrib-grpc-util:0.30.0
io.opencensus:opencensus-contrib-http-util:0.28.0
it.unimi.dsi:fastutil:6.5.16
io.grpc:grpc-alts:1.45.1
io.grpc:grpc-api:1.47.0
io.grpc:grpc-auth:1.45.1
io.grpc:grpc-census:1.45.1
io.grpc:grpc-context:1.47.0
io.grpc:grpc-core:1.45.1
io.grpc:grpc-grpclb:1.45.1
io.grpc:grpc-netty-shaded:1.45.1
io.grpc:grpc-netty:1.45.1
io.grpc:grpc-protobuf-lite:1.47.0
io.grpc:grpc-protobuf:1.47.0
io.grpc:grpc-services:1.45.1
io.grpc:grpc-stub:1.47.0
io.grpc:grpc-xds:1.45.1
io.netty:netty-buffer:4.1.72.Final
io.netty:netty-codec-http2:4.1.72.Final
io.netty:netty-codec-http:4.1.72.Final
io.netty:netty-codec:4.1.72.Final
io.netty:netty-common:4.1.72.Final
io.netty:netty-handler:4.1.72.Final
io.netty:netty-resolver:4.1.72.Final
io.netty:netty-tcnative-boringssl-static:2.0.46.Final
io.netty:netty-tcnative-classes:2.0.46.Final
io.netty:netty-transport:4.1.72.Final
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-exemplar-util:0.31.0
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0
io.opencensus:opencensus-contrib-grpc-util:0.31.0
io.opencensus:opencensus-contrib-http-util:0.31.1
io.opencensus:opencensus-contrib-resource-util:0.31.0
io.opencensus:opencensus-exporter-metrics-util:0.31.0
io.opencensus:opencensus-exporter-stats-stackdriver:0.31.0
io.opencensus:opencensus-impl-core:0.31.0
io.opencensus:opencensus-impl:0.31.0
io.opencensus:opencensus-proto:0.2.0
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.jdo:jdo2-api:2.3-20090302111651
javax.mail:mail:1.5.0-b01
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.10
junit:junit:4.13.1
net.bytebuddy:byte-buddy:1.10.17
net.java.dev.jna:jna:5.5.0
junit:junit:4.13.2
net.bytebuddy:byte-buddy:1.12.9
net.java.dev.jna:jna:5.8.0
org.apache.arrow:arrow-format:5.0.0
org.apache.arrow:arrow-memory-core:5.0.0
org.apache.arrow:arrow-vector:5.0.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-fn-execution:2.37.0
org.apache.beam:beam-model-job-management:2.37.0
org.apache.beam:beam-model-pipeline:2.37.0
org.apache.beam:beam-runners-core-construction-java:2.37.0
org.apache.beam:beam-runners-core-java:2.37.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
org.apache.beam:beam-runners-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-core:2.37.0
org.apache.beam:beam-sdks-java-expansion-service:2.37.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
org.apache.beam:beam-sdks-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
org.apache.beam:beam-sdks-java-io-kafka:2.37.0
org.apache.beam:beam-model-fn-execution:2.40.0
org.apache.beam:beam-model-job-management:2.40.0
org.apache.beam:beam-model-pipeline:2.40.0
org.apache.beam:beam-runners-core-construction-java:2.40.0
org.apache.beam:beam-runners-core-java:2.40.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.40.0
org.apache.beam:beam-runners-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-core:2.40.0
org.apache.beam:beam-sdks-java-expansion-service:2.40.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.40.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.40.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.40.0
org.apache.beam:beam-sdks-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0
org.apache.beam:beam-sdks-java-io-kafka:2.40.0
org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
org.apache.beam:beam-vendor-grpc-1_43_2:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-compress:1.21
org.apache.commons:commons-csv:1.9.0
org.apache.commons:commons-lang3:3.12.0
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.15
org.bouncycastle:bcpg-jdk15on:1.67
org.bouncycastle:bcpkix-jdk15on:1.67
org.bouncycastle:bcprov-jdk15on:1.67
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.21.0
org.checkerframework:checker-qual:3.22.2
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.conscrypt:conscrypt-openjdk-uber:2.5.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.flywaydb:flyway-core:8.5.13
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.gwtproject:gwt-user:2.10.0
org.hamcrest:hamcrest-core:1.3
org.hamcrest:hamcrest:2.1
org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
org.hibernate:hibernate-core:5.4.23.Final
org.hibernate:hibernate-hikaricp:5.4.23.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.4.1.Final
org.hibernate:hibernate-core:5.6.10.Final
org.hibernate:hibernate-hikaricp:5.6.10.Final
org.jboss.logging:jboss-logging:3.4.3.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.1.3.Final
org.jboss:jandex:2.4.2.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20200518
org.jsoup:jsoup:1.13.1
org.jsoup:jsoup:1.15.2
org.jvnet.staxex:stax-ex:1.8
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:9.1
org.ow2.asm:asm-commons:9.0
org.ow2.asm:asm-tree:9.1
org.ow2.asm:asm-util:9.1
org.ow2.asm:asm:9.1
org.postgresql:postgresql:42.2.18
org.ow2.asm:asm-analysis:9.3
org.ow2.asm:asm-commons:9.2
org.ow2.asm:asm-tree:9.3
org.ow2.asm:asm-util:9.3
org.ow2.asm:asm:9.3
org.postgresql:postgresql:42.4.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.slf4j:slf4j-api:1.7.30
org.springframework:spring-core:4.3.18.RELEASE
org.springframework:spring-expression:4.3.18.RELEASE
org.testcontainers:database-commons:1.15.2
org.testcontainers:jdbc:1.15.2
org.testcontainers:postgresql:1.15.2
org.testcontainers:testcontainers:1.15.2
org.threeten:threetenbp:1.5.2
org.slf4j:slf4j-api:1.7.36
org.springframework:spring-core:5.3.18
org.springframework:spring-expression:5.3.18
org.springframework:spring-jcl:5.3.18
org.testcontainers:database-commons:1.17.3
org.testcontainers:jdbc:1.17.3
org.testcontainers:postgresql:1.17.3
org.testcontainers:testcontainers:1.17.3
org.threeten:threetenbp:1.6.0
org.tukaani:xz:1.5
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.8.4
org.yaml:snakeyaml:1.28
org.yaml:snakeyaml:1.30
us.fatehi:schemacrawler-api:16.10.1
us.fatehi:schemacrawler-diagram:16.10.1
us.fatehi:schemacrawler-tools:16.10.1

View File

@@ -4,301 +4,311 @@
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.101tec:zkclient:0.10
com.beust:jcommander:1.60
com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.13.0
com.fasterxml.jackson.core:jackson-core:2.13.0
com.fasterxml.jackson.core:jackson-databind:2.13.0
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
com.fasterxml.jackson:jackson-bom:2.13.0
com.fasterxml.jackson.core:jackson-annotations:2.13.3
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson.core:jackson-databind:2.13.3
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.fasterxml:classmate:1.5.1
com.github.docker-java:docker-java-api:3.2.7
com.github.docker-java:docker-java-transport-zerodep:3.2.7
com.github.docker-java:docker-java-transport:3.2.7
com.github.jnr:jffi:1.3.1
com.github.ben-manes.caffeine:caffeine:2.9.3
com.github.docker-java:docker-java-api:3.2.13
com.github.docker-java:docker-java-transport-zerodep:3.2.13
com.github.docker-java:docker-java-transport:3.2.13
com.github.jnr:jffi:1.3.9
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.10.1
com.github.jnr:jnr-enxio:0.32.3
com.github.jnr:jnr-ffi:2.2.1
com.github.jnr:jnr-posix:3.1.4
com.github.jnr:jnr-unixsocket:0.38.5
com.github.jnr:jnr-constants:0.10.3
com.github.jnr:jnr-enxio:0.32.13
com.github.jnr:jnr-ffi:2.2.11
com.github.jnr:jnr-posix:3.1.15
com.github.jnr:jnr-unixsocket:0.38.17
com.github.jnr:jnr-x86asm:1.0.2
com.google.android:annotations:4.1.1.4
com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-appengine:1.35.2
com.google.api-client:google-api-client-jackson2:1.32.2
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.32.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api-client:google-api-client-java6:1.35.2
com.google.api-client:google-api-client-servlet:1.35.2
com.google.api-client:google-api-client:1.35.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:grpc-google-common-protos:2.7.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
com.google.api.grpc:proto-google-common-protos:2.7.1
com.google.api.grpc:proto-google-iam-v1:1.2.0
com.google.api:api-common:2.1.2
com.google.api:gax-grpc:2.8.1
com.google.api:gax-httpjson:0.93.1
com.google.api:gax:2.8.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:grpc-google-common-protos:2.8.3
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.6.2
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.10
com.google.api.grpc:proto-google-cloud-firestore-v1:3.1.0
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.3.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.3.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:proto-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:2.3.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.93.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.93.0
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-grpc:2.18.2
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
com.google.apis:google-api-services-cloudkms:v1-rev20220617-1.32.1
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
com.google.apis:google-api-services-drive:v2-rev393-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev20210624-1.32.1
com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-monitoring:v3-rev20220626-1.32.1
com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220513-1.32.1
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.86
com.google.appengine:appengine-remote-api:1.9.86
com.google.appengine:appengine-api-1.0-sdk:2.0.5
com.google.appengine:appengine-remote-api:2.0.5
com.google.appengine:appengine-testing:1.9.86
com.google.auth:google-auth-library-credentials:1.3.0
com.google.auth:google-auth-library-oauth2-http:1.3.0
com.google.auto.service:auto-service-annotations:1.0-rc7
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.service:auto-service-annotations:1.0.1
com.google.auto.value:auto-value-annotations:1.9
com.google.auto.value:auto-value:1.7.4
com.google.cloud.bigdataoss:gcsio:2.2.4
com.google.cloud.bigdataoss:util:2.2.4
com.google.cloud.bigtable:bigtable-client-core:1.25.1
com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
com.google.cloud:google-cloud-bigquerystorage:2.8.1
com.google.cloud:google-cloud-bigtable:1.27.1
com.google.cloud:google-cloud-core-grpc:2.3.5
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:2.3.5
com.google.cloud:google-cloud-firestore:3.0.10
com.google.cloud:google-cloud-pubsub:1.115.1
com.google.cloud:google-cloud-pubsublite:1.4.8
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:6.17.4
com.google.cloud:google-cloud-storage:1.113.12
com.google.cloud:google-cloud-tasks:1.33.2
com.google.auto.value:auto-value:1.9
com.google.cloud.bigdataoss:gcsio:2.2.6
com.google.cloud.bigdataoss:util:2.2.6
com.google.cloud.bigtable:bigtable-client-core:1.26.3
com.google.cloud.bigtable:bigtable-metrics-api:1.26.3
com.google.cloud.datastore:datastore-v1-proto-client:2.2.10
com.google.cloud.sql:jdbc-socket-factory-core:1.6.1
com.google.cloud:google-cloud-bigquerystorage:2.12.2
com.google.cloud:google-cloud-bigtable:2.6.2
com.google.cloud:google-cloud-core-grpc:2.6.0
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-firestore:3.1.0
com.google.cloud:google-cloud-monitoring:1.82.0
com.google.cloud:google-cloud-pubsub:1.116.4
com.google.cloud:google-cloud-pubsublite:1.5.4
com.google.cloud:google-cloud-secretmanager:2.3.0
com.google.cloud:google-cloud-spanner:6.23.3
com.google.cloud:google-cloud-storage:2.9.2
com.google.cloud:google-cloud-tasks:2.3.0
com.google.cloud:grpc-gcp:1.1.0
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.1.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.9
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.10.0
com.google.dagger:dagger:2.42
com.google.errorprone:error_prone_annotations:2.14.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flatbuffers:flatbuffers-java:1.12.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.flogger:google-extensions:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:31.0.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.9.0
com.google.http-client:google-http-client-apache-v2:1.40.1
com.google.http-client:google-http-client-appengine:1.39.2
com.google.http-client:google-http-client-gson:1.41.0
com.google.http-client:google-http-client-jackson2:1.41.0
com.google.http-client:google-http-client-protobuf:1.40.1
com.google.http-client:google-http-client:1.41.0
com.google.gwt:gwt-user:2.10.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client-protobuf:1.41.7
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:2.0.0
com.google.monitoring-client:metrics:1.0.7
com.google.monitoring-client:stackdriver:1.0.7
com.google.oauth-client:google-oauth-client-appengine:1.31.4
com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.32.1
com.google.protobuf:protobuf-java-util:3.19.2
com.google.protobuf:protobuf-java:3.19.2
com.google.re2j:re2j:1.6
com.google.oauth-client:google-oauth-client-appengine:1.34.1
com.google.oauth-client:google-oauth-client-java6:1.34.1
com.google.oauth-client:google-oauth-client-jetty:1.34.1
com.google.oauth-client:google-oauth-client-servlet:1.34.1
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.re2j:re2j:1.7
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
com.ibm.icu:icu4j:71.1
com.jcraft:jsch:0.1.55
com.lmax:disruptor:3.4.2
com.sun.istack:istack-commons-runtime:3.0.7
com.sun.xml.fastinfoset:FastInfoset:1.2.15
com.thoughtworks.paranamer:paranamer:2.7
com.zaxxer:HikariCP:3.4.5
commons-codec:commons-codec:1.15
commons-logging:commons-logging:1.2
dnsjava:dnsjava:3.3.1
dnsjava:dnsjava:3.5.1
guru.nidi.com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
guru.nidi.com.kitfox:svgSalamander:1.1.3
guru.nidi:graphviz-java-all-j2v8:0.17.0
guru.nidi:graphviz-java:0.17.0
guru.nidi:graphviz-java-all-j2v8:0.18.1
guru.nidi:graphviz-java:0.18.1
io.confluent:common-config:5.3.2
io.confluent:common-utils:5.3.2
io.confluent:kafka-avro-serializer:5.3.2
io.confluent:kafka-schema-registry-client:5.3.2
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.104
io.grpc:grpc-alts:1.43.2
io.grpc:grpc-api:1.43.2
io.grpc:grpc-auth:1.43.2
io.grpc:grpc-context:1.43.2
io.grpc:grpc-core:1.43.2
io.grpc:grpc-grpclb:1.43.2
io.grpc:grpc-netty-shaded:1.43.2
io.grpc:grpc-netty:1.43.2
io.grpc:grpc-protobuf-lite:1.43.2
io.grpc:grpc-protobuf:1.43.2
io.grpc:grpc-services:1.43.2
io.grpc:grpc-stub:1.43.2
io.grpc:grpc-xds:1.43.2
io.netty:netty-buffer:4.1.63.Final
io.netty:netty-codec-http2:4.1.63.Final
io.netty:netty-codec-http:4.1.63.Final
io.netty:netty-codec-socks:4.1.63.Final
io.netty:netty-codec:4.1.63.Final
io.netty:netty-common:4.1.63.Final
io.netty:netty-handler-proxy:4.1.63.Final
io.netty:netty-handler:4.1.63.Final
io.netty:netty-resolver:4.1.63.Final
io.netty:netty-tcnative-boringssl-static:2.0.33.Final
io.netty:netty-transport:4.1.63.Final
io.opencensus:opencensus-api:0.30.0
io.opencensus:opencensus-contrib-grpc-util:0.30.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-alts:1.47.0
io.grpc:grpc-api:1.47.0
io.grpc:grpc-auth:1.47.0
io.grpc:grpc-census:1.45.1
io.grpc:grpc-context:1.47.0
io.grpc:grpc-core:1.47.0
io.grpc:grpc-googleapis:1.47.0
io.grpc:grpc-grpclb:1.47.0
io.grpc:grpc-netty-shaded:1.47.0
io.grpc:grpc-netty:1.45.1
io.grpc:grpc-protobuf-lite:1.47.0
io.grpc:grpc-protobuf:1.47.0
io.grpc:grpc-services:1.47.0
io.grpc:grpc-stub:1.47.0
io.grpc:grpc-xds:1.47.0
io.netty:netty-buffer:4.1.72.Final
io.netty:netty-codec-http2:4.1.72.Final
io.netty:netty-codec-http:4.1.72.Final
io.netty:netty-codec-socks:4.1.72.Final
io.netty:netty-codec:4.1.72.Final
io.netty:netty-common:4.1.72.Final
io.netty:netty-handler-proxy:4.1.72.Final
io.netty:netty-handler:4.1.72.Final
io.netty:netty-resolver:4.1.72.Final
io.netty:netty-tcnative-boringssl-static:2.0.46.Final
io.netty:netty-tcnative-classes:2.0.46.Final
io.netty:netty-transport:4.1.72.Final
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-exemplar-util:0.31.0
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0
io.opencensus:opencensus-contrib-grpc-util:0.31.0
io.opencensus:opencensus-contrib-http-util:0.31.1
io.opencensus:opencensus-contrib-resource-util:0.31.0
io.opencensus:opencensus-exporter-metrics-util:0.31.0
io.opencensus:opencensus-exporter-stats-stackdriver:0.31.0
io.opencensus:opencensus-impl-core:0.31.0
io.opencensus:opencensus-impl:0.31.0
io.opencensus:opencensus-proto:0.2.0
io.perfmark:perfmark-api:0.23.0
it.unimi.dsi:fastutil:6.5.16
io.perfmark:perfmark-api:0.25.0
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.jdo:jdo2-api:2.3-20090302111651
javax.mail:mail:1.5.0-b01
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.10
junit:junit:4.13.1
junit:junit:4.13.2
net.arnx:nashorn-promise:0.1.1
net.bytebuddy:byte-buddy:1.10.17
net.java.dev.jna:jna:5.5.0
net.bytebuddy:byte-buddy:1.12.9
net.java.dev.jna:jna:5.8.0
org.apache.arrow:arrow-format:5.0.0
org.apache.arrow:arrow-memory-core:5.0.0
org.apache.arrow:arrow-vector:5.0.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-fn-execution:2.37.0
org.apache.beam:beam-model-job-management:2.37.0
org.apache.beam:beam-model-pipeline:2.37.0
org.apache.beam:beam-runners-core-construction-java:2.37.0
org.apache.beam:beam-runners-core-java:2.37.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
org.apache.beam:beam-runners-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-core:2.37.0
org.apache.beam:beam-sdks-java-expansion-service:2.37.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
org.apache.beam:beam-sdks-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
org.apache.beam:beam-sdks-java-io-kafka:2.37.0
org.apache.beam:beam-model-fn-execution:2.40.0
org.apache.beam:beam-model-job-management:2.40.0
org.apache.beam:beam-model-pipeline:2.40.0
org.apache.beam:beam-runners-core-construction-java:2.40.0
org.apache.beam:beam-runners-core-java:2.40.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.40.0
org.apache.beam:beam-runners-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-core:2.40.0
org.apache.beam:beam-sdks-java-expansion-service:2.40.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.40.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.40.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.40.0
org.apache.beam:beam-sdks-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0
org.apache.beam:beam-sdks-java-io-kafka:2.40.0
org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
org.apache.beam:beam-vendor-grpc-1_43_2:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-compress:1.21
org.apache.commons:commons-csv:1.9.0
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-lang3:3.12.0
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.15
org.bouncycastle:bcpg-jdk15on:1.67
org.bouncycastle:bcpkix-jdk15on:1.67
org.bouncycastle:bcprov-jdk15on:1.67
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.21.0
org.checkerframework:checker-qual:3.22.2
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.20
org.codehaus.mojo:animal-sniffer-annotations:1.21
org.conscrypt:conscrypt-openjdk-uber:2.5.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.flywaydb:flyway-core:8.5.13
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.gwtproject:gwt-user:2.10.0
org.hamcrest:hamcrest-core:1.3
org.hamcrest:hamcrest:2.1
org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
org.hibernate:hibernate-core:5.4.23.Final
org.hibernate:hibernate-hikaricp:5.4.23.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.4.1.Final
org.hibernate:hibernate-core:5.6.10.Final
org.hibernate:hibernate-hikaricp:5.6.10.Final
org.jboss.logging:jboss-logging:3.4.3.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.1.3.Final
org.jboss:jandex:2.4.2.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20200518
org.jsoup:jsoup:1.13.1
org.jsoup:jsoup:1.15.2
org.jvnet.staxex:stax-ex:1.8
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:9.1
org.ow2.asm:asm-commons:9.0
org.ow2.asm:asm-tree:9.1
org.ow2.asm:asm-util:9.1
org.ow2.asm:asm:9.1
org.postgresql:postgresql:42.2.18
org.ow2.asm:asm-analysis:9.3
org.ow2.asm:asm-commons:9.2
org.ow2.asm:asm-tree:9.3
org.ow2.asm:asm-util:9.3
org.ow2.asm:asm:9.3
org.postgresql:postgresql:42.4.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.slf4j:jcl-over-slf4j:1.7.30
org.slf4j:jul-to-slf4j:1.7.30
org.slf4j:slf4j-api:1.7.30
org.springframework:spring-core:4.3.18.RELEASE
org.springframework:spring-expression:4.3.18.RELEASE
org.testcontainers:database-commons:1.15.2
org.testcontainers:jdbc:1.15.2
org.testcontainers:postgresql:1.15.2
org.testcontainers:testcontainers:1.15.2
org.threeten:threetenbp:1.5.2
org.slf4j:slf4j-api:1.7.36
org.springframework:spring-core:5.3.18
org.springframework:spring-expression:5.3.18
org.springframework:spring-jcl:5.3.18
org.testcontainers:database-commons:1.17.3
org.testcontainers:jdbc:1.17.3
org.testcontainers:postgresql:1.17.3
org.testcontainers:testcontainers:1.17.3
org.threeten:threetenbp:1.6.0
org.tukaani:xz:1.5
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3
org.webjars.npm:viz.js-graphviz-java:2.1.3
org.xerial.snappy:snappy-java:1.1.8.4
org.yaml:snakeyaml:1.28
org.yaml:snakeyaml:1.30
us.fatehi:schemacrawler-api:16.10.1
us.fatehi:schemacrawler-diagram:16.10.1
us.fatehi:schemacrawler-tools:16.10.1

View File

@@ -4,301 +4,311 @@
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.101tec:zkclient:0.10
com.beust:jcommander:1.60
com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.13.0
com.fasterxml.jackson.core:jackson-core:2.13.0
com.fasterxml.jackson.core:jackson-databind:2.13.0
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
com.fasterxml.jackson:jackson-bom:2.13.0
com.fasterxml.jackson.core:jackson-annotations:2.13.3
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson.core:jackson-databind:2.13.3
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.fasterxml:classmate:1.5.1
com.github.docker-java:docker-java-api:3.2.7
com.github.docker-java:docker-java-transport-zerodep:3.2.7
com.github.docker-java:docker-java-transport:3.2.7
com.github.jnr:jffi:1.3.1
com.github.ben-manes.caffeine:caffeine:2.9.3
com.github.docker-java:docker-java-api:3.2.13
com.github.docker-java:docker-java-transport-zerodep:3.2.13
com.github.docker-java:docker-java-transport:3.2.13
com.github.jnr:jffi:1.3.9
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.10.1
com.github.jnr:jnr-enxio:0.32.3
com.github.jnr:jnr-ffi:2.2.1
com.github.jnr:jnr-posix:3.1.4
com.github.jnr:jnr-unixsocket:0.38.5
com.github.jnr:jnr-constants:0.10.3
com.github.jnr:jnr-enxio:0.32.13
com.github.jnr:jnr-ffi:2.2.11
com.github.jnr:jnr-posix:3.1.15
com.github.jnr:jnr-unixsocket:0.38.17
com.github.jnr:jnr-x86asm:1.0.2
com.google.android:annotations:4.1.1.4
com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-appengine:1.35.2
com.google.api-client:google-api-client-jackson2:1.32.2
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.32.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api-client:google-api-client-java6:1.35.2
com.google.api-client:google-api-client-servlet:1.35.2
com.google.api-client:google-api-client:1.35.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:grpc-google-common-protos:2.7.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
com.google.api.grpc:proto-google-common-protos:2.7.1
com.google.api.grpc:proto-google-iam-v1:1.2.0
com.google.api:api-common:2.1.2
com.google.api:gax-grpc:2.8.1
com.google.api:gax-httpjson:0.93.1
com.google.api:gax:2.8.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:grpc-google-common-protos:2.8.3
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.6.2
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.10
com.google.api.grpc:proto-google-cloud-firestore-v1:3.1.0
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.3.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.3.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:proto-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:2.3.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.93.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.93.0
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-grpc:2.18.2
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
com.google.apis:google-api-services-cloudkms:v1-rev20220617-1.32.1
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
com.google.apis:google-api-services-drive:v2-rev393-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev20210624-1.32.1
com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-monitoring:v3-rev20220626-1.32.1
com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220513-1.32.1
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.86
com.google.appengine:appengine-remote-api:1.9.86
com.google.appengine:appengine-api-1.0-sdk:2.0.5
com.google.appengine:appengine-remote-api:2.0.5
com.google.appengine:appengine-testing:1.9.86
com.google.auth:google-auth-library-credentials:1.3.0
com.google.auth:google-auth-library-oauth2-http:1.3.0
com.google.auto.service:auto-service-annotations:1.0-rc7
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.service:auto-service-annotations:1.0.1
com.google.auto.value:auto-value-annotations:1.9
com.google.auto.value:auto-value:1.7.4
com.google.cloud.bigdataoss:gcsio:2.2.4
com.google.cloud.bigdataoss:util:2.2.4
com.google.cloud.bigtable:bigtable-client-core:1.25.1
com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
com.google.cloud:google-cloud-bigquerystorage:2.8.1
com.google.cloud:google-cloud-bigtable:1.27.1
com.google.cloud:google-cloud-core-grpc:2.3.5
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:2.3.5
com.google.cloud:google-cloud-firestore:3.0.10
com.google.cloud:google-cloud-pubsub:1.115.1
com.google.cloud:google-cloud-pubsublite:1.4.8
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:6.17.4
com.google.cloud:google-cloud-storage:1.113.12
com.google.cloud:google-cloud-tasks:1.33.2
com.google.auto.value:auto-value:1.9
com.google.cloud.bigdataoss:gcsio:2.2.6
com.google.cloud.bigdataoss:util:2.2.6
com.google.cloud.bigtable:bigtable-client-core:1.26.3
com.google.cloud.bigtable:bigtable-metrics-api:1.26.3
com.google.cloud.datastore:datastore-v1-proto-client:2.2.10
com.google.cloud.sql:jdbc-socket-factory-core:1.6.1
com.google.cloud:google-cloud-bigquerystorage:2.12.2
com.google.cloud:google-cloud-bigtable:2.6.2
com.google.cloud:google-cloud-core-grpc:2.6.0
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-firestore:3.1.0
com.google.cloud:google-cloud-monitoring:1.82.0
com.google.cloud:google-cloud-pubsub:1.116.4
com.google.cloud:google-cloud-pubsublite:1.5.4
com.google.cloud:google-cloud-secretmanager:2.3.0
com.google.cloud:google-cloud-spanner:6.23.3
com.google.cloud:google-cloud-storage:2.9.2
com.google.cloud:google-cloud-tasks:2.3.0
com.google.cloud:grpc-gcp:1.1.0
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.1.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.9
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.10.0
com.google.dagger:dagger:2.42
com.google.errorprone:error_prone_annotations:2.14.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flatbuffers:flatbuffers-java:1.12.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.flogger:google-extensions:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:31.0.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.9.0
com.google.http-client:google-http-client-apache-v2:1.40.1
com.google.http-client:google-http-client-appengine:1.39.2
com.google.http-client:google-http-client-gson:1.41.0
com.google.http-client:google-http-client-jackson2:1.41.0
com.google.http-client:google-http-client-protobuf:1.40.1
com.google.http-client:google-http-client:1.41.0
com.google.gwt:gwt-user:2.10.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client-protobuf:1.41.7
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:2.0.0
com.google.monitoring-client:metrics:1.0.7
com.google.monitoring-client:stackdriver:1.0.7
com.google.oauth-client:google-oauth-client-appengine:1.31.4
com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.32.1
com.google.protobuf:protobuf-java-util:3.19.2
com.google.protobuf:protobuf-java:3.19.2
com.google.re2j:re2j:1.6
com.google.oauth-client:google-oauth-client-appengine:1.34.1
com.google.oauth-client:google-oauth-client-java6:1.34.1
com.google.oauth-client:google-oauth-client-jetty:1.34.1
com.google.oauth-client:google-oauth-client-servlet:1.34.1
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.re2j:re2j:1.7
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
com.ibm.icu:icu4j:71.1
com.jcraft:jsch:0.1.55
com.lmax:disruptor:3.4.2
com.sun.istack:istack-commons-runtime:3.0.7
com.sun.xml.fastinfoset:FastInfoset:1.2.15
com.thoughtworks.paranamer:paranamer:2.7
com.zaxxer:HikariCP:3.4.5
commons-codec:commons-codec:1.15
commons-logging:commons-logging:1.2
dnsjava:dnsjava:3.3.1
dnsjava:dnsjava:3.5.1
guru.nidi.com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
guru.nidi.com.kitfox:svgSalamander:1.1.3
guru.nidi:graphviz-java-all-j2v8:0.17.0
guru.nidi:graphviz-java:0.17.0
guru.nidi:graphviz-java-all-j2v8:0.18.1
guru.nidi:graphviz-java:0.18.1
io.confluent:common-config:5.3.2
io.confluent:common-utils:5.3.2
io.confluent:kafka-avro-serializer:5.3.2
io.confluent:kafka-schema-registry-client:5.3.2
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.104
io.grpc:grpc-alts:1.43.2
io.grpc:grpc-api:1.43.2
io.grpc:grpc-auth:1.43.2
io.grpc:grpc-context:1.43.2
io.grpc:grpc-core:1.43.2
io.grpc:grpc-grpclb:1.43.2
io.grpc:grpc-netty-shaded:1.43.2
io.grpc:grpc-netty:1.43.2
io.grpc:grpc-protobuf-lite:1.43.2
io.grpc:grpc-protobuf:1.43.2
io.grpc:grpc-services:1.43.2
io.grpc:grpc-stub:1.43.2
io.grpc:grpc-xds:1.43.2
io.netty:netty-buffer:4.1.63.Final
io.netty:netty-codec-http2:4.1.63.Final
io.netty:netty-codec-http:4.1.63.Final
io.netty:netty-codec-socks:4.1.63.Final
io.netty:netty-codec:4.1.63.Final
io.netty:netty-common:4.1.63.Final
io.netty:netty-handler-proxy:4.1.63.Final
io.netty:netty-handler:4.1.63.Final
io.netty:netty-resolver:4.1.63.Final
io.netty:netty-tcnative-boringssl-static:2.0.33.Final
io.netty:netty-transport:4.1.63.Final
io.opencensus:opencensus-api:0.30.0
io.opencensus:opencensus-contrib-grpc-util:0.30.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-alts:1.47.0
io.grpc:grpc-api:1.47.0
io.grpc:grpc-auth:1.47.0
io.grpc:grpc-census:1.45.1
io.grpc:grpc-context:1.47.0
io.grpc:grpc-core:1.47.0
io.grpc:grpc-googleapis:1.47.0
io.grpc:grpc-grpclb:1.47.0
io.grpc:grpc-netty-shaded:1.47.0
io.grpc:grpc-netty:1.45.1
io.grpc:grpc-protobuf-lite:1.47.0
io.grpc:grpc-protobuf:1.47.0
io.grpc:grpc-services:1.47.0
io.grpc:grpc-stub:1.47.0
io.grpc:grpc-xds:1.47.0
io.netty:netty-buffer:4.1.72.Final
io.netty:netty-codec-http2:4.1.72.Final
io.netty:netty-codec-http:4.1.72.Final
io.netty:netty-codec-socks:4.1.72.Final
io.netty:netty-codec:4.1.72.Final
io.netty:netty-common:4.1.72.Final
io.netty:netty-handler-proxy:4.1.72.Final
io.netty:netty-handler:4.1.72.Final
io.netty:netty-resolver:4.1.72.Final
io.netty:netty-tcnative-boringssl-static:2.0.46.Final
io.netty:netty-tcnative-classes:2.0.46.Final
io.netty:netty-transport:4.1.72.Final
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-exemplar-util:0.31.0
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0
io.opencensus:opencensus-contrib-grpc-util:0.31.0
io.opencensus:opencensus-contrib-http-util:0.31.1
io.opencensus:opencensus-contrib-resource-util:0.31.0
io.opencensus:opencensus-exporter-metrics-util:0.31.0
io.opencensus:opencensus-exporter-stats-stackdriver:0.31.0
io.opencensus:opencensus-impl-core:0.31.0
io.opencensus:opencensus-impl:0.31.0
io.opencensus:opencensus-proto:0.2.0
io.perfmark:perfmark-api:0.23.0
it.unimi.dsi:fastutil:6.5.16
io.perfmark:perfmark-api:0.25.0
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.jdo:jdo2-api:2.3-20090302111651
javax.mail:mail:1.5.0-b01
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.10
junit:junit:4.13.1
junit:junit:4.13.2
net.arnx:nashorn-promise:0.1.1
net.bytebuddy:byte-buddy:1.10.17
net.java.dev.jna:jna:5.5.0
net.bytebuddy:byte-buddy:1.12.9
net.java.dev.jna:jna:5.8.0
org.apache.arrow:arrow-format:5.0.0
org.apache.arrow:arrow-memory-core:5.0.0
org.apache.arrow:arrow-vector:5.0.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-fn-execution:2.37.0
org.apache.beam:beam-model-job-management:2.37.0
org.apache.beam:beam-model-pipeline:2.37.0
org.apache.beam:beam-runners-core-construction-java:2.37.0
org.apache.beam:beam-runners-core-java:2.37.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
org.apache.beam:beam-runners-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-core:2.37.0
org.apache.beam:beam-sdks-java-expansion-service:2.37.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
org.apache.beam:beam-sdks-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
org.apache.beam:beam-sdks-java-io-kafka:2.37.0
org.apache.beam:beam-model-fn-execution:2.40.0
org.apache.beam:beam-model-job-management:2.40.0
org.apache.beam:beam-model-pipeline:2.40.0
org.apache.beam:beam-runners-core-construction-java:2.40.0
org.apache.beam:beam-runners-core-java:2.40.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.40.0
org.apache.beam:beam-runners-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-core:2.40.0
org.apache.beam:beam-sdks-java-expansion-service:2.40.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.40.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.40.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.40.0
org.apache.beam:beam-sdks-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0
org.apache.beam:beam-sdks-java-io-kafka:2.40.0
org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
org.apache.beam:beam-vendor-grpc-1_43_2:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-compress:1.21
org.apache.commons:commons-csv:1.9.0
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-lang3:3.12.0
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.15
org.bouncycastle:bcpg-jdk15on:1.67
org.bouncycastle:bcpkix-jdk15on:1.67
org.bouncycastle:bcprov-jdk15on:1.67
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.21.0
org.checkerframework:checker-qual:3.22.2
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.20
org.codehaus.mojo:animal-sniffer-annotations:1.21
org.conscrypt:conscrypt-openjdk-uber:2.5.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.flywaydb:flyway-core:8.5.13
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.gwtproject:gwt-user:2.10.0
org.hamcrest:hamcrest-core:1.3
org.hamcrest:hamcrest:2.1
org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
org.hibernate:hibernate-core:5.4.23.Final
org.hibernate:hibernate-hikaricp:5.4.23.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.4.1.Final
org.hibernate:hibernate-core:5.6.10.Final
org.hibernate:hibernate-hikaricp:5.6.10.Final
org.jboss.logging:jboss-logging:3.4.3.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.1.3.Final
org.jboss:jandex:2.4.2.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20200518
org.jsoup:jsoup:1.13.1
org.jsoup:jsoup:1.15.2
org.jvnet.staxex:stax-ex:1.8
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:9.1
org.ow2.asm:asm-commons:9.0
org.ow2.asm:asm-tree:9.1
org.ow2.asm:asm-util:9.1
org.ow2.asm:asm:9.1
org.postgresql:postgresql:42.2.18
org.ow2.asm:asm-analysis:9.3
org.ow2.asm:asm-commons:9.2
org.ow2.asm:asm-tree:9.3
org.ow2.asm:asm-util:9.3
org.ow2.asm:asm:9.3
org.postgresql:postgresql:42.4.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.slf4j:jcl-over-slf4j:1.7.30
org.slf4j:jul-to-slf4j:1.7.30
org.slf4j:slf4j-api:1.7.30
org.springframework:spring-core:4.3.18.RELEASE
org.springframework:spring-expression:4.3.18.RELEASE
org.testcontainers:database-commons:1.15.2
org.testcontainers:jdbc:1.15.2
org.testcontainers:postgresql:1.15.2
org.testcontainers:testcontainers:1.15.2
org.threeten:threetenbp:1.5.2
org.slf4j:slf4j-api:1.7.36
org.springframework:spring-core:5.3.18
org.springframework:spring-expression:5.3.18
org.springframework:spring-jcl:5.3.18
org.testcontainers:database-commons:1.17.3
org.testcontainers:jdbc:1.17.3
org.testcontainers:postgresql:1.17.3
org.testcontainers:testcontainers:1.17.3
org.threeten:threetenbp:1.6.0
org.tukaani:xz:1.5
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3
org.webjars.npm:viz.js-graphviz-java:2.1.3
org.xerial.snappy:snappy-java:1.1.8.4
org.yaml:snakeyaml:1.28
org.yaml:snakeyaml:1.30
us.fatehi:schemacrawler-api:16.10.1
us.fatehi:schemacrawler-diagram:16.10.1
us.fatehi:schemacrawler-tools:16.10.1

View File

@@ -4,301 +4,311 @@
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.101tec:zkclient:0.10
com.beust:jcommander:1.60
com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.13.0
com.fasterxml.jackson.core:jackson-core:2.13.0
com.fasterxml.jackson.core:jackson-databind:2.13.0
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
com.fasterxml.jackson:jackson-bom:2.13.0
com.fasterxml.jackson.core:jackson-annotations:2.13.3
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson.core:jackson-databind:2.13.3
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.fasterxml:classmate:1.5.1
com.github.docker-java:docker-java-api:3.2.7
com.github.docker-java:docker-java-transport-zerodep:3.2.7
com.github.docker-java:docker-java-transport:3.2.7
com.github.jnr:jffi:1.3.1
com.github.ben-manes.caffeine:caffeine:2.9.3
com.github.docker-java:docker-java-api:3.2.13
com.github.docker-java:docker-java-transport-zerodep:3.2.13
com.github.docker-java:docker-java-transport:3.2.13
com.github.jnr:jffi:1.3.9
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.10.1
com.github.jnr:jnr-enxio:0.32.3
com.github.jnr:jnr-ffi:2.2.1
com.github.jnr:jnr-posix:3.1.4
com.github.jnr:jnr-unixsocket:0.38.5
com.github.jnr:jnr-constants:0.10.3
com.github.jnr:jnr-enxio:0.32.13
com.github.jnr:jnr-ffi:2.2.11
com.github.jnr:jnr-posix:3.1.15
com.github.jnr:jnr-unixsocket:0.38.17
com.github.jnr:jnr-x86asm:1.0.2
com.google.android:annotations:4.1.1.4
com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-appengine:1.35.2
com.google.api-client:google-api-client-jackson2:1.32.2
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.32.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api-client:google-api-client-java6:1.35.2
com.google.api-client:google-api-client-servlet:1.35.2
com.google.api-client:google-api-client:1.35.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:grpc-google-common-protos:2.7.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
com.google.api.grpc:proto-google-common-protos:2.7.1
com.google.api.grpc:proto-google-iam-v1:1.2.0
com.google.api:api-common:2.1.2
com.google.api:gax-grpc:2.8.1
com.google.api:gax-httpjson:0.93.1
com.google.api:gax:2.8.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:grpc-google-common-protos:2.8.3
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.6.2
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.10
com.google.api.grpc:proto-google-cloud-firestore-v1:3.1.0
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.3.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.3.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:proto-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:2.3.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.93.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.93.0
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-grpc:2.18.2
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
com.google.apis:google-api-services-cloudkms:v1-rev20220617-1.32.1
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
com.google.apis:google-api-services-drive:v2-rev393-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev20210624-1.32.1
com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-monitoring:v3-rev20220626-1.32.1
com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220513-1.32.1
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.86
com.google.appengine:appengine-remote-api:1.9.86
com.google.appengine:appengine-api-1.0-sdk:2.0.5
com.google.appengine:appengine-remote-api:2.0.5
com.google.appengine:appengine-testing:1.9.86
com.google.auth:google-auth-library-credentials:1.3.0
com.google.auth:google-auth-library-oauth2-http:1.3.0
com.google.auto.service:auto-service-annotations:1.0-rc7
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.service:auto-service-annotations:1.0.1
com.google.auto.value:auto-value-annotations:1.9
com.google.auto.value:auto-value:1.7.4
com.google.cloud.bigdataoss:gcsio:2.2.4
com.google.cloud.bigdataoss:util:2.2.4
com.google.cloud.bigtable:bigtable-client-core:1.25.1
com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
com.google.cloud:google-cloud-bigquerystorage:2.8.1
com.google.cloud:google-cloud-bigtable:1.27.1
com.google.cloud:google-cloud-core-grpc:2.3.5
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:2.3.5
com.google.cloud:google-cloud-firestore:3.0.10
com.google.cloud:google-cloud-pubsub:1.115.1
com.google.cloud:google-cloud-pubsublite:1.4.8
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:6.17.4
com.google.cloud:google-cloud-storage:1.113.12
com.google.cloud:google-cloud-tasks:1.33.2
com.google.auto.value:auto-value:1.9
com.google.cloud.bigdataoss:gcsio:2.2.6
com.google.cloud.bigdataoss:util:2.2.6
com.google.cloud.bigtable:bigtable-client-core:1.26.3
com.google.cloud.bigtable:bigtable-metrics-api:1.26.3
com.google.cloud.datastore:datastore-v1-proto-client:2.2.10
com.google.cloud.sql:jdbc-socket-factory-core:1.6.1
com.google.cloud:google-cloud-bigquerystorage:2.12.2
com.google.cloud:google-cloud-bigtable:2.6.2
com.google.cloud:google-cloud-core-grpc:2.6.0
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-firestore:3.1.0
com.google.cloud:google-cloud-monitoring:1.82.0
com.google.cloud:google-cloud-pubsub:1.116.4
com.google.cloud:google-cloud-pubsublite:1.5.4
com.google.cloud:google-cloud-secretmanager:2.3.0
com.google.cloud:google-cloud-spanner:6.23.3
com.google.cloud:google-cloud-storage:2.9.2
com.google.cloud:google-cloud-tasks:2.3.0
com.google.cloud:grpc-gcp:1.1.0
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.1.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.9
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.10.0
com.google.dagger:dagger:2.42
com.google.errorprone:error_prone_annotations:2.14.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flatbuffers:flatbuffers-java:1.12.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.flogger:google-extensions:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:31.0.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.9.0
com.google.http-client:google-http-client-apache-v2:1.40.1
com.google.http-client:google-http-client-appengine:1.39.2
com.google.http-client:google-http-client-gson:1.41.0
com.google.http-client:google-http-client-jackson2:1.41.0
com.google.http-client:google-http-client-protobuf:1.40.1
com.google.http-client:google-http-client:1.41.0
com.google.gwt:gwt-user:2.10.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client-protobuf:1.41.7
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:2.0.0
com.google.monitoring-client:metrics:1.0.7
com.google.monitoring-client:stackdriver:1.0.7
com.google.oauth-client:google-oauth-client-appengine:1.31.4
com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.32.1
com.google.protobuf:protobuf-java-util:3.19.2
com.google.protobuf:protobuf-java:3.19.2
com.google.re2j:re2j:1.6
com.google.oauth-client:google-oauth-client-appengine:1.34.1
com.google.oauth-client:google-oauth-client-java6:1.34.1
com.google.oauth-client:google-oauth-client-jetty:1.34.1
com.google.oauth-client:google-oauth-client-servlet:1.34.1
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.re2j:re2j:1.7
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
com.ibm.icu:icu4j:71.1
com.jcraft:jsch:0.1.55
com.lmax:disruptor:3.4.2
com.sun.istack:istack-commons-runtime:3.0.7
com.sun.xml.fastinfoset:FastInfoset:1.2.15
com.thoughtworks.paranamer:paranamer:2.7
com.zaxxer:HikariCP:3.4.5
commons-codec:commons-codec:1.15
commons-logging:commons-logging:1.2
dnsjava:dnsjava:3.3.1
dnsjava:dnsjava:3.5.1
guru.nidi.com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
guru.nidi.com.kitfox:svgSalamander:1.1.3
guru.nidi:graphviz-java-all-j2v8:0.17.0
guru.nidi:graphviz-java:0.17.0
guru.nidi:graphviz-java-all-j2v8:0.18.1
guru.nidi:graphviz-java:0.18.1
io.confluent:common-config:5.3.2
io.confluent:common-utils:5.3.2
io.confluent:kafka-avro-serializer:5.3.2
io.confluent:kafka-schema-registry-client:5.3.2
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.104
io.grpc:grpc-alts:1.43.2
io.grpc:grpc-api:1.43.2
io.grpc:grpc-auth:1.43.2
io.grpc:grpc-context:1.43.2
io.grpc:grpc-core:1.43.2
io.grpc:grpc-grpclb:1.43.2
io.grpc:grpc-netty-shaded:1.43.2
io.grpc:grpc-netty:1.43.2
io.grpc:grpc-protobuf-lite:1.43.2
io.grpc:grpc-protobuf:1.43.2
io.grpc:grpc-services:1.43.2
io.grpc:grpc-stub:1.43.2
io.grpc:grpc-xds:1.43.2
io.netty:netty-buffer:4.1.63.Final
io.netty:netty-codec-http2:4.1.63.Final
io.netty:netty-codec-http:4.1.63.Final
io.netty:netty-codec-socks:4.1.63.Final
io.netty:netty-codec:4.1.63.Final
io.netty:netty-common:4.1.63.Final
io.netty:netty-handler-proxy:4.1.63.Final
io.netty:netty-handler:4.1.63.Final
io.netty:netty-resolver:4.1.63.Final
io.netty:netty-tcnative-boringssl-static:2.0.33.Final
io.netty:netty-transport:4.1.63.Final
io.opencensus:opencensus-api:0.30.0
io.opencensus:opencensus-contrib-grpc-util:0.30.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-alts:1.47.0
io.grpc:grpc-api:1.47.0
io.grpc:grpc-auth:1.47.0
io.grpc:grpc-census:1.45.1
io.grpc:grpc-context:1.47.0
io.grpc:grpc-core:1.47.0
io.grpc:grpc-googleapis:1.47.0
io.grpc:grpc-grpclb:1.47.0
io.grpc:grpc-netty-shaded:1.47.0
io.grpc:grpc-netty:1.45.1
io.grpc:grpc-protobuf-lite:1.47.0
io.grpc:grpc-protobuf:1.47.0
io.grpc:grpc-services:1.47.0
io.grpc:grpc-stub:1.47.0
io.grpc:grpc-xds:1.47.0
io.netty:netty-buffer:4.1.72.Final
io.netty:netty-codec-http2:4.1.72.Final
io.netty:netty-codec-http:4.1.72.Final
io.netty:netty-codec-socks:4.1.72.Final
io.netty:netty-codec:4.1.72.Final
io.netty:netty-common:4.1.72.Final
io.netty:netty-handler-proxy:4.1.72.Final
io.netty:netty-handler:4.1.72.Final
io.netty:netty-resolver:4.1.72.Final
io.netty:netty-tcnative-boringssl-static:2.0.46.Final
io.netty:netty-tcnative-classes:2.0.46.Final
io.netty:netty-transport:4.1.72.Final
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-exemplar-util:0.31.0
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0
io.opencensus:opencensus-contrib-grpc-util:0.31.0
io.opencensus:opencensus-contrib-http-util:0.31.1
io.opencensus:opencensus-contrib-resource-util:0.31.0
io.opencensus:opencensus-exporter-metrics-util:0.31.0
io.opencensus:opencensus-exporter-stats-stackdriver:0.31.0
io.opencensus:opencensus-impl-core:0.31.0
io.opencensus:opencensus-impl:0.31.0
io.opencensus:opencensus-proto:0.2.0
io.perfmark:perfmark-api:0.23.0
it.unimi.dsi:fastutil:6.5.16
io.perfmark:perfmark-api:0.25.0
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.jdo:jdo2-api:2.3-20090302111651
javax.mail:mail:1.5.0-b01
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.10
junit:junit:4.13.1
junit:junit:4.13.2
net.arnx:nashorn-promise:0.1.1
net.bytebuddy:byte-buddy:1.10.17
net.java.dev.jna:jna:5.5.0
net.bytebuddy:byte-buddy:1.12.9
net.java.dev.jna:jna:5.8.0
org.apache.arrow:arrow-format:5.0.0
org.apache.arrow:arrow-memory-core:5.0.0
org.apache.arrow:arrow-vector:5.0.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-fn-execution:2.37.0
org.apache.beam:beam-model-job-management:2.37.0
org.apache.beam:beam-model-pipeline:2.37.0
org.apache.beam:beam-runners-core-construction-java:2.37.0
org.apache.beam:beam-runners-core-java:2.37.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
org.apache.beam:beam-runners-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-core:2.37.0
org.apache.beam:beam-sdks-java-expansion-service:2.37.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
org.apache.beam:beam-sdks-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
org.apache.beam:beam-sdks-java-io-kafka:2.37.0
org.apache.beam:beam-model-fn-execution:2.40.0
org.apache.beam:beam-model-job-management:2.40.0
org.apache.beam:beam-model-pipeline:2.40.0
org.apache.beam:beam-runners-core-construction-java:2.40.0
org.apache.beam:beam-runners-core-java:2.40.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.40.0
org.apache.beam:beam-runners-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-core:2.40.0
org.apache.beam:beam-sdks-java-expansion-service:2.40.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.40.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.40.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.40.0
org.apache.beam:beam-sdks-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0
org.apache.beam:beam-sdks-java-io-kafka:2.40.0
org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
org.apache.beam:beam-vendor-grpc-1_43_2:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-compress:1.21
org.apache.commons:commons-csv:1.9.0
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-lang3:3.12.0
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.15
org.bouncycastle:bcpg-jdk15on:1.67
org.bouncycastle:bcpkix-jdk15on:1.67
org.bouncycastle:bcprov-jdk15on:1.67
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.21.0
org.checkerframework:checker-qual:3.22.2
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.20
org.codehaus.mojo:animal-sniffer-annotations:1.21
org.conscrypt:conscrypt-openjdk-uber:2.5.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.flywaydb:flyway-core:8.5.13
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.gwtproject:gwt-user:2.10.0
org.hamcrest:hamcrest-core:1.3
org.hamcrest:hamcrest:2.1
org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
org.hibernate:hibernate-core:5.4.23.Final
org.hibernate:hibernate-hikaricp:5.4.23.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.4.1.Final
org.hibernate:hibernate-core:5.6.10.Final
org.hibernate:hibernate-hikaricp:5.6.10.Final
org.jboss.logging:jboss-logging:3.4.3.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.1.3.Final
org.jboss:jandex:2.4.2.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20200518
org.jsoup:jsoup:1.13.1
org.jsoup:jsoup:1.15.2
org.jvnet.staxex:stax-ex:1.8
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:9.1
org.ow2.asm:asm-commons:9.0
org.ow2.asm:asm-tree:9.1
org.ow2.asm:asm-util:9.1
org.ow2.asm:asm:9.1
org.postgresql:postgresql:42.2.18
org.ow2.asm:asm-analysis:9.3
org.ow2.asm:asm-commons:9.2
org.ow2.asm:asm-tree:9.3
org.ow2.asm:asm-util:9.3
org.ow2.asm:asm:9.3
org.postgresql:postgresql:42.4.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.slf4j:jcl-over-slf4j:1.7.30
org.slf4j:jul-to-slf4j:1.7.30
org.slf4j:slf4j-api:1.7.30
org.springframework:spring-core:4.3.18.RELEASE
org.springframework:spring-expression:4.3.18.RELEASE
org.testcontainers:database-commons:1.15.2
org.testcontainers:jdbc:1.15.2
org.testcontainers:postgresql:1.15.2
org.testcontainers:testcontainers:1.15.2
org.threeten:threetenbp:1.5.2
org.slf4j:slf4j-api:1.7.36
org.springframework:spring-core:5.3.18
org.springframework:spring-expression:5.3.18
org.springframework:spring-jcl:5.3.18
org.testcontainers:database-commons:1.17.3
org.testcontainers:jdbc:1.17.3
org.testcontainers:postgresql:1.17.3
org.testcontainers:testcontainers:1.17.3
org.threeten:threetenbp:1.6.0
org.tukaani:xz:1.5
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3
org.webjars.npm:viz.js-graphviz-java:2.1.3
org.xerial.snappy:snappy-java:1.1.8.4
org.yaml:snakeyaml:1.28
org.yaml:snakeyaml:1.30
us.fatehi:schemacrawler-api:16.10.1
us.fatehi:schemacrawler-diagram:16.10.1
us.fatehi:schemacrawler-tools:16.10.1

View File

@@ -4,301 +4,311 @@
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.101tec:zkclient:0.10
com.beust:jcommander:1.60
com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.13.0
com.fasterxml.jackson.core:jackson-core:2.13.0
com.fasterxml.jackson.core:jackson-databind:2.13.0
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
com.fasterxml.jackson:jackson-bom:2.13.0
com.fasterxml.jackson.core:jackson-annotations:2.13.3
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson.core:jackson-databind:2.13.3
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.fasterxml:classmate:1.5.1
com.github.docker-java:docker-java-api:3.2.7
com.github.docker-java:docker-java-transport-zerodep:3.2.7
com.github.docker-java:docker-java-transport:3.2.7
com.github.jnr:jffi:1.3.1
com.github.ben-manes.caffeine:caffeine:2.9.3
com.github.docker-java:docker-java-api:3.2.13
com.github.docker-java:docker-java-transport-zerodep:3.2.13
com.github.docker-java:docker-java-transport:3.2.13
com.github.jnr:jffi:1.3.9
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.10.1
com.github.jnr:jnr-enxio:0.32.3
com.github.jnr:jnr-ffi:2.2.1
com.github.jnr:jnr-posix:3.1.4
com.github.jnr:jnr-unixsocket:0.38.5
com.github.jnr:jnr-constants:0.10.3
com.github.jnr:jnr-enxio:0.32.13
com.github.jnr:jnr-ffi:2.2.11
com.github.jnr:jnr-posix:3.1.15
com.github.jnr:jnr-unixsocket:0.38.17
com.github.jnr:jnr-x86asm:1.0.2
com.google.android:annotations:4.1.1.4
com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-appengine:1.35.2
com.google.api-client:google-api-client-jackson2:1.32.2
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.32.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api-client:google-api-client-java6:1.35.2
com.google.api-client:google-api-client-servlet:1.35.2
com.google.api-client:google-api-client:1.35.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:grpc-google-common-protos:2.7.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
com.google.api.grpc:proto-google-common-protos:2.7.1
com.google.api.grpc:proto-google-iam-v1:1.2.0
com.google.api:api-common:2.1.2
com.google.api:gax-grpc:2.8.1
com.google.api:gax-httpjson:0.93.1
com.google.api:gax:2.8.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:grpc-google-common-protos:2.8.3
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.6.2
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.10
com.google.api.grpc:proto-google-cloud-firestore-v1:3.1.0
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.3.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.3.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:proto-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:2.3.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.93.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.93.0
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-grpc:2.18.2
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
com.google.apis:google-api-services-cloudkms:v1-rev20220617-1.32.1
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
com.google.apis:google-api-services-drive:v2-rev393-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev20210624-1.32.1
com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-monitoring:v3-rev20220626-1.32.1
com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220513-1.32.1
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.86
com.google.appengine:appengine-remote-api:1.9.86
com.google.appengine:appengine-api-1.0-sdk:2.0.5
com.google.appengine:appengine-remote-api:2.0.5
com.google.appengine:appengine-testing:1.9.86
com.google.auth:google-auth-library-credentials:1.3.0
com.google.auth:google-auth-library-oauth2-http:1.3.0
com.google.auto.service:auto-service-annotations:1.0-rc7
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.service:auto-service-annotations:1.0.1
com.google.auto.value:auto-value-annotations:1.9
com.google.auto.value:auto-value:1.7.4
com.google.cloud.bigdataoss:gcsio:2.2.4
com.google.cloud.bigdataoss:util:2.2.4
com.google.cloud.bigtable:bigtable-client-core:1.25.1
com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
com.google.cloud.sql:postgres-socket-factory:1.2.1
com.google.cloud:google-cloud-bigquerystorage:2.8.1
com.google.cloud:google-cloud-bigtable:1.27.1
com.google.cloud:google-cloud-core-grpc:2.3.5
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:2.3.5
com.google.cloud:google-cloud-firestore:3.0.10
com.google.cloud:google-cloud-pubsub:1.115.1
com.google.cloud:google-cloud-pubsublite:1.4.8
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:6.17.4
com.google.cloud:google-cloud-storage:1.113.12
com.google.cloud:google-cloud-tasks:1.33.2
com.google.auto.value:auto-value:1.9
com.google.cloud.bigdataoss:gcsio:2.2.6
com.google.cloud.bigdataoss:util:2.2.6
com.google.cloud.bigtable:bigtable-client-core:1.26.3
com.google.cloud.bigtable:bigtable-metrics-api:1.26.3
com.google.cloud.datastore:datastore-v1-proto-client:2.2.10
com.google.cloud.sql:jdbc-socket-factory-core:1.6.1
com.google.cloud.sql:postgres-socket-factory:1.6.1
com.google.cloud:google-cloud-bigquerystorage:2.12.2
com.google.cloud:google-cloud-bigtable:2.6.2
com.google.cloud:google-cloud-core-grpc:2.6.0
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-firestore:3.1.0
com.google.cloud:google-cloud-monitoring:1.82.0
com.google.cloud:google-cloud-pubsub:1.116.4
com.google.cloud:google-cloud-pubsublite:1.5.4
com.google.cloud:google-cloud-secretmanager:2.3.0
com.google.cloud:google-cloud-spanner:6.23.3
com.google.cloud:google-cloud-storage:2.9.2
com.google.cloud:google-cloud-tasks:2.3.0
com.google.cloud:grpc-gcp:1.1.0
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.1.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.9
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.10.0
com.google.dagger:dagger:2.42
com.google.errorprone:error_prone_annotations:2.14.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flatbuffers:flatbuffers-java:1.12.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.flogger:google-extensions:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:31.0.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.9.0
com.google.http-client:google-http-client-apache-v2:1.40.1
com.google.http-client:google-http-client-appengine:1.39.2
com.google.http-client:google-http-client-gson:1.41.0
com.google.http-client:google-http-client-jackson2:1.41.0
com.google.http-client:google-http-client-protobuf:1.40.1
com.google.http-client:google-http-client:1.41.0
com.google.gwt:gwt-user:2.10.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client-protobuf:1.41.7
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:2.0.0
com.google.monitoring-client:metrics:1.0.7
com.google.monitoring-client:stackdriver:1.0.7
com.google.oauth-client:google-oauth-client-appengine:1.31.4
com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.32.1
com.google.protobuf:protobuf-java-util:3.19.2
com.google.protobuf:protobuf-java:3.19.2
com.google.re2j:re2j:1.6
com.google.oauth-client:google-oauth-client-appengine:1.34.1
com.google.oauth-client:google-oauth-client-java6:1.34.1
com.google.oauth-client:google-oauth-client-jetty:1.34.1
com.google.oauth-client:google-oauth-client-servlet:1.34.1
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.re2j:re2j:1.7
com.google.template:soy:2021-02-01
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
com.ibm.icu:icu4j:71.1
com.jcraft:jsch:0.1.55
com.lmax:disruptor:3.4.2
com.sun.istack:istack-commons-runtime:3.0.7
com.sun.xml.fastinfoset:FastInfoset:1.2.15
com.thoughtworks.paranamer:paranamer:2.7
com.zaxxer:HikariCP:3.4.5
commons-codec:commons-codec:1.15
commons-logging:commons-logging:1.2
dnsjava:dnsjava:3.3.1
dnsjava:dnsjava:3.5.1
guru.nidi.com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
guru.nidi.com.kitfox:svgSalamander:1.1.3
guru.nidi:graphviz-java-all-j2v8:0.17.0
guru.nidi:graphviz-java:0.17.0
guru.nidi:graphviz-java-all-j2v8:0.18.1
guru.nidi:graphviz-java:0.18.1
io.confluent:common-config:5.3.2
io.confluent:common-utils:5.3.2
io.confluent:kafka-avro-serializer:5.3.2
io.confluent:kafka-schema-registry-client:5.3.2
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.104
io.grpc:grpc-alts:1.43.2
io.grpc:grpc-api:1.43.2
io.grpc:grpc-auth:1.43.2
io.grpc:grpc-context:1.43.2
io.grpc:grpc-core:1.43.2
io.grpc:grpc-grpclb:1.43.2
io.grpc:grpc-netty-shaded:1.43.2
io.grpc:grpc-netty:1.43.2
io.grpc:grpc-protobuf-lite:1.43.2
io.grpc:grpc-protobuf:1.43.2
io.grpc:grpc-services:1.43.2
io.grpc:grpc-stub:1.43.2
io.grpc:grpc-xds:1.43.2
io.netty:netty-buffer:4.1.63.Final
io.netty:netty-codec-http2:4.1.63.Final
io.netty:netty-codec-http:4.1.63.Final
io.netty:netty-codec-socks:4.1.63.Final
io.netty:netty-codec:4.1.63.Final
io.netty:netty-common:4.1.63.Final
io.netty:netty-handler-proxy:4.1.63.Final
io.netty:netty-handler:4.1.63.Final
io.netty:netty-resolver:4.1.63.Final
io.netty:netty-tcnative-boringssl-static:2.0.33.Final
io.netty:netty-transport:4.1.63.Final
io.opencensus:opencensus-api:0.30.0
io.opencensus:opencensus-contrib-grpc-util:0.30.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-alts:1.47.0
io.grpc:grpc-api:1.47.0
io.grpc:grpc-auth:1.47.0
io.grpc:grpc-census:1.45.1
io.grpc:grpc-context:1.47.0
io.grpc:grpc-core:1.47.0
io.grpc:grpc-googleapis:1.47.0
io.grpc:grpc-grpclb:1.47.0
io.grpc:grpc-netty-shaded:1.47.0
io.grpc:grpc-netty:1.45.1
io.grpc:grpc-protobuf-lite:1.47.0
io.grpc:grpc-protobuf:1.47.0
io.grpc:grpc-services:1.47.0
io.grpc:grpc-stub:1.47.0
io.grpc:grpc-xds:1.47.0
io.netty:netty-buffer:4.1.72.Final
io.netty:netty-codec-http2:4.1.72.Final
io.netty:netty-codec-http:4.1.72.Final
io.netty:netty-codec-socks:4.1.72.Final
io.netty:netty-codec:4.1.72.Final
io.netty:netty-common:4.1.72.Final
io.netty:netty-handler-proxy:4.1.72.Final
io.netty:netty-handler:4.1.72.Final
io.netty:netty-resolver:4.1.72.Final
io.netty:netty-tcnative-boringssl-static:2.0.46.Final
io.netty:netty-tcnative-classes:2.0.46.Final
io.netty:netty-transport:4.1.72.Final
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-exemplar-util:0.31.0
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0
io.opencensus:opencensus-contrib-grpc-util:0.31.0
io.opencensus:opencensus-contrib-http-util:0.31.1
io.opencensus:opencensus-contrib-resource-util:0.31.0
io.opencensus:opencensus-exporter-metrics-util:0.31.0
io.opencensus:opencensus-exporter-stats-stackdriver:0.31.0
io.opencensus:opencensus-impl-core:0.31.0
io.opencensus:opencensus-impl:0.31.0
io.opencensus:opencensus-proto:0.2.0
io.perfmark:perfmark-api:0.23.0
it.unimi.dsi:fastutil:6.5.16
io.perfmark:perfmark-api:0.25.0
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.jdo:jdo2-api:2.3-20090302111651
javax.mail:mail:1.5.0-b01
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.10
net.arnx:nashorn-promise:0.1.1
net.bytebuddy:byte-buddy:1.10.17
net.java.dev.jna:jna:5.5.0
net.bytebuddy:byte-buddy:1.12.9
net.java.dev.jna:jna:5.8.0
org.apache.arrow:arrow-format:5.0.0
org.apache.arrow:arrow-memory-core:5.0.0
org.apache.arrow:arrow-vector:5.0.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-fn-execution:2.37.0
org.apache.beam:beam-model-job-management:2.37.0
org.apache.beam:beam-model-pipeline:2.37.0
org.apache.beam:beam-runners-core-construction-java:2.37.0
org.apache.beam:beam-runners-core-java:2.37.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
org.apache.beam:beam-runners-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-core:2.37.0
org.apache.beam:beam-sdks-java-expansion-service:2.37.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
org.apache.beam:beam-sdks-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
org.apache.beam:beam-sdks-java-io-kafka:2.37.0
org.apache.beam:beam-model-fn-execution:2.40.0
org.apache.beam:beam-model-job-management:2.40.0
org.apache.beam:beam-model-pipeline:2.40.0
org.apache.beam:beam-runners-core-construction-java:2.40.0
org.apache.beam:beam-runners-core-java:2.40.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.40.0
org.apache.beam:beam-runners-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-core:2.40.0
org.apache.beam:beam-sdks-java-expansion-service:2.40.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.40.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.40.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.40.0
org.apache.beam:beam-sdks-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0
org.apache.beam:beam-sdks-java-io-kafka:2.40.0
org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
org.apache.beam:beam-vendor-grpc-1_43_2:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-compress:1.21
org.apache.commons:commons-csv:1.9.0
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-lang3:3.12.0
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.15
org.bouncycastle:bcpg-jdk15on:1.67
org.bouncycastle:bcpkix-jdk15on:1.67
org.bouncycastle:bcprov-jdk15on:1.67
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.21.0
org.checkerframework:checker-qual:3.22.2
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.20
org.codehaus.mojo:animal-sniffer-annotations:1.21
org.conscrypt:conscrypt-openjdk-uber:2.5.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.flywaydb:flyway-core:8.5.13
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.gwtproject:gwt-user:2.10.0
org.hamcrest:hamcrest:2.1
org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
org.hibernate:hibernate-core:5.4.23.Final
org.hibernate:hibernate-hikaricp:5.4.23.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.4.1.Final
org.hibernate:hibernate-core:5.6.10.Final
org.hibernate:hibernate-hikaricp:5.6.10.Final
org.jboss.logging:jboss-logging:3.4.3.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.1.3.Final
org.jboss:jandex:2.4.2.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20200518
org.jsoup:jsoup:1.13.1
org.jsoup:jsoup:1.15.2
org.jvnet.staxex:stax-ex:1.8
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:9.1
org.ow2.asm:asm-commons:9.0
org.ow2.asm:asm-tree:9.1
org.ow2.asm:asm-util:9.1
org.ow2.asm:asm:9.1
org.postgresql:postgresql:42.2.18
org.ow2.asm:asm-analysis:9.3
org.ow2.asm:asm-commons:9.2
org.ow2.asm:asm-tree:9.3
org.ow2.asm:asm-util:9.3
org.ow2.asm:asm:9.3
org.postgresql:postgresql:42.4.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.slf4j:jcl-over-slf4j:1.7.30
org.slf4j:jul-to-slf4j:1.7.30
org.slf4j:slf4j-api:1.7.30
org.slf4j:slf4j-jdk14:1.7.28
org.springframework:spring-core:4.3.18.RELEASE
org.springframework:spring-expression:4.3.18.RELEASE
org.testcontainers:database-commons:1.15.2
org.testcontainers:jdbc:1.15.2
org.testcontainers:postgresql:1.15.2
org.testcontainers:testcontainers:1.15.2
org.threeten:threetenbp:1.5.2
org.slf4j:slf4j-api:2.0.0-alpha7
org.slf4j:slf4j-jdk14:2.0.0-alpha7
org.springframework:spring-core:5.3.18
org.springframework:spring-expression:5.3.18
org.springframework:spring-jcl:5.3.18
org.testcontainers:database-commons:1.17.3
org.testcontainers:jdbc:1.17.3
org.testcontainers:postgresql:1.17.3
org.testcontainers:testcontainers:1.17.3
org.threeten:threetenbp:1.6.0
org.tukaani:xz:1.5
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3
org.webjars.npm:viz.js-graphviz-java:2.1.3
org.xerial.snappy:snappy-java:1.1.8.4
org.yaml:snakeyaml:1.28
org.yaml:snakeyaml:1.30
us.fatehi:schemacrawler-api:16.10.1
us.fatehi:schemacrawler-diagram:16.10.1
us.fatehi:schemacrawler-tools:16.10.1

View File

@@ -12,10 +12,10 @@ com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:5.0.1
com.google.inject:guice:5.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.1
com.google.protobuf:protobuf-java:3.13.0
com.google.protobuf:protobuf-java:4.0.0-rc-2
com.google.template:soy:2021-02-01
com.ibm.icu:icu4j:57.1
javax.annotation:jsr250-api:1.0

View File

@@ -3,39 +3,41 @@
# This file is expected to be part of source control.
com.github.ben-manes.caffeine:caffeine:2.7.0
com.github.kevinstern:software-and-algorithms:1.0
com.google.auto.value:auto-value:1.7.4
com.google.auto.value:auto-value:1.9
com.google.auto:auto-common:0.10
com.google.code.findbugs:jFormatString:3.0.0
com.google.code.findbugs:jsr305:3.0.2
com.google.dagger:dagger-compiler:2.33
com.google.dagger:dagger-producers:2.33
com.google.dagger:dagger-spi:2.33
com.google.dagger:dagger:2.33
com.google.dagger:dagger-compiler:2.42
com.google.dagger:dagger-producers:2.42
com.google.dagger:dagger-spi:2.42
com.google.dagger:dagger:2.42
com.google.devtools.ksp:symbol-processing-api:1.5.30-1.0.0
com.google.errorprone:error_prone_annotation:2.3.4
com.google.errorprone:error_prone_annotations:2.5.1
com.google.errorprone:error_prone_annotations:2.7.1
com.google.errorprone:error_prone_check_api:2.3.4
com.google.errorprone:error_prone_core:2.3.4
com.google.errorprone:error_prone_type_annotations:2.3.4
com.google.errorprone:javac-shaded:9-dev-r4023-3
com.google.googlejavaformat:google-java-format:1.5
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:30.1.1-jre
com.google.guava:guava:31.0.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.protobuf:protobuf-java:3.4.0
com.googlecode.java-diff-utils:diffutils:1.3.0
com.squareup:javapoet:1.13.0
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.persistence:javax.persistence-api:2.2
net.ltgt.gradle.incap:incap:0.2
org.checkerframework:checker-compat-qual:2.5.3
org.checkerframework:checker-qual:3.8.0
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.12.0
org.checkerframework:dataflow:3.0.0
org.checkerframework:javacutil:3.0.0
org.jetbrains.kotlin:kotlin-stdlib-common:1.4.20
org.jetbrains.kotlin:kotlin-stdlib:1.4.20
org.jetbrains.kotlinx:kotlinx-metadata-jvm:0.1.0
org.jetbrains.kotlin:kotlin-stdlib-common:1.6.10
org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.5.32
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.5.32
org.jetbrains.kotlin:kotlin-stdlib:1.6.10
org.jetbrains.kotlinx:kotlinx-metadata-jvm:0.4.2
org.jetbrains:annotations:13.0
org.pcollections:pcollections:2.1.2
org.plumelib:plume-util:1.0.6

View File

@@ -4,139 +4,142 @@
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.101tec:zkclient:0.10
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.13.0
com.fasterxml.jackson.core:jackson-core:2.13.0
com.fasterxml.jackson.core:jackson-databind:2.13.0
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
com.fasterxml.jackson:jackson-bom:2.13.0
com.fasterxml.jackson.core:jackson-annotations:2.13.3
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson.core:jackson-databind:2.13.3
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.fasterxml:classmate:1.5.1
com.github.docker-java:docker-java-api:3.2.7
com.github.docker-java:docker-java-transport-zerodep:3.2.7
com.github.docker-java:docker-java-transport:3.2.7
com.github.jnr:jffi:1.3.1
com.github.ben-manes.caffeine:caffeine:2.9.3
com.github.docker-java:docker-java-api:3.2.13
com.github.docker-java:docker-java-transport-zerodep:3.2.13
com.github.docker-java:docker-java-transport:3.2.13
com.github.jnr:jffi:1.3.9
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.10.1
com.github.jnr:jnr-enxio:0.32.3
com.github.jnr:jnr-ffi:2.2.1
com.github.jnr:jnr-posix:3.1.4
com.github.jnr:jnr-unixsocket:0.38.5
com.github.jnr:jnr-constants:0.10.3
com.github.jnr:jnr-enxio:0.32.13
com.github.jnr:jnr-ffi:2.2.11
com.github.jnr:jnr-posix:3.1.15
com.github.jnr:jnr-unixsocket:0.38.17
com.github.jnr:jnr-x86asm:1.0.2
com.google.android:annotations:4.1.1.4
com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-appengine:1.35.2
com.google.api-client:google-api-client-jackson2:1.32.2
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.32.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api-client:google-api-client-java6:1.35.2
com.google.api-client:google-api-client-servlet:1.35.2
com.google.api-client:google-api-client:1.35.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:grpc-google-common-protos:2.7.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
com.google.api.grpc:proto-google-common-protos:2.7.1
com.google.api.grpc:proto-google-iam-v1:1.2.0
com.google.api:api-common:2.1.2
com.google.api:gax-grpc:2.8.1
com.google.api:gax-httpjson:0.93.1
com.google.api:gax:2.8.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:grpc-google-common-protos:2.8.3
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.6.2
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.10
com.google.api.grpc:proto-google-cloud-firestore-v1:3.1.0
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.3.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.3.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:proto-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:2.3.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.93.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.93.0
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-grpc:2.18.2
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
com.google.apis:google-api-services-cloudkms:v1-rev20220617-1.32.1
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
com.google.apis:google-api-services-drive:v2-rev393-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev20210624-1.32.1
com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-monitoring:v3-rev20220626-1.32.1
com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220513-1.32.1
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.86
com.google.appengine:appengine-api-stubs:1.9.86
com.google.appengine:appengine-remote-api:1.9.86
com.google.appengine:appengine-api-1.0-sdk:2.0.5
com.google.appengine:appengine-api-stubs:2.0.5
com.google.appengine:appengine-remote-api:2.0.5
com.google.appengine:appengine-testing:1.9.86
com.google.auth:google-auth-library-credentials:1.3.0
com.google.auth:google-auth-library-oauth2-http:1.3.0
com.google.auto.service:auto-service-annotations:1.0-rc7
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.service:auto-service-annotations:1.0.1
com.google.auto.value:auto-value-annotations:1.9
com.google.auto.value:auto-value:1.7.4
com.google.cloud.bigdataoss:gcsio:2.2.4
com.google.cloud.bigdataoss:util:2.2.4
com.google.cloud.bigtable:bigtable-client-core:1.25.1
com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
com.google.cloud:google-cloud-bigquerystorage:2.8.1
com.google.cloud:google-cloud-bigtable:1.27.1
com.google.cloud:google-cloud-core-grpc:2.3.5
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:2.3.5
com.google.cloud:google-cloud-firestore:3.0.10
com.google.cloud:google-cloud-nio:0.123.4
com.google.cloud:google-cloud-pubsub:1.115.1
com.google.cloud:google-cloud-pubsublite:1.4.8
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:6.17.4
com.google.cloud:google-cloud-storage:1.118.0
com.google.cloud:google-cloud-tasks:1.33.2
com.google.auto.value:auto-value:1.9
com.google.cloud.bigdataoss:gcsio:2.2.6
com.google.cloud.bigdataoss:util:2.2.6
com.google.cloud.bigtable:bigtable-client-core:1.26.3
com.google.cloud.bigtable:bigtable-metrics-api:1.26.3
com.google.cloud.datastore:datastore-v1-proto-client:2.2.10
com.google.cloud.sql:jdbc-socket-factory-core:1.6.1
com.google.cloud:google-cloud-bigquerystorage:2.12.2
com.google.cloud:google-cloud-bigtable:2.6.2
com.google.cloud:google-cloud-core-grpc:2.6.0
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-firestore:3.1.0
com.google.cloud:google-cloud-monitoring:1.82.0
com.google.cloud:google-cloud-nio:0.124.7
com.google.cloud:google-cloud-pubsub:1.116.4
com.google.cloud:google-cloud-pubsublite:1.5.4
com.google.cloud:google-cloud-secretmanager:2.3.0
com.google.cloud:google-cloud-spanner:6.23.3
com.google.cloud:google-cloud-storage:2.9.0
com.google.cloud:google-cloud-tasks:2.3.0
com.google.cloud:grpc-gcp:1.1.0
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.1.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.9
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.10.0
com.google.dagger:dagger:2.42
com.google.errorprone:error_prone_annotations:2.14.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flatbuffers:flatbuffers-java:1.12.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.flogger:google-extensions:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava-testlib:30.1.1-jre
com.google.guava:guava:31.0.1-jre
com.google.guava:guava-testlib:31.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.9.0
com.google.http-client:google-http-client-apache-v2:1.40.1
com.google.http-client:google-http-client-appengine:1.39.2
com.google.http-client:google-http-client-gson:1.41.0
com.google.http-client:google-http-client-jackson2:1.41.0
com.google.http-client:google-http-client-protobuf:1.40.1
com.google.http-client:google-http-client:1.41.0
com.google.gwt:gwt-user:2.10.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client-protobuf:1.41.7
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -144,21 +147,21 @@ com.google.jsinterop:jsinterop-annotations:2.0.0
com.google.monitoring-client:contrib:1.0.7
com.google.monitoring-client:metrics:1.0.7
com.google.monitoring-client:stackdriver:1.0.7
com.google.oauth-client:google-oauth-client-appengine:1.31.4
com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.32.1
com.google.protobuf:protobuf-java-util:3.19.2
com.google.protobuf:protobuf-java:3.19.2
com.google.re2j:re2j:1.6
com.google.oauth-client:google-oauth-client-appengine:1.34.1
com.google.oauth-client:google-oauth-client-java6:1.34.1
com.google.oauth-client:google-oauth-client-jetty:1.34.1
com.google.oauth-client:google-oauth-client-servlet:1.34.1
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.re2j:re2j:1.7
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.googlecode.charts4j:charts4j:1.3
com.google.truth.extensions:truth-java8-extension:1.1.3
com.google.truth:truth:1.1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
com.ibm.icu:icu4j:71.1
com.jcraft:jsch:0.1.55
com.lmax:disruptor:3.4.2
com.squareup.okhttp3:okhttp:3.11.0
com.squareup.okio:okio:1.14.0
com.sun.istack:istack-commons-runtime:3.0.7
@@ -168,151 +171,158 @@ com.thoughtworks.qdox:qdox:1.12.1
com.zaxxer:HikariCP:3.4.5
commons-codec:commons-codec:1.15
commons-logging:commons-logging:1.2
dnsjava:dnsjava:3.3.1
dnsjava:dnsjava:3.5.1
io.confluent:common-config:5.3.2
io.confluent:common-utils:5.3.2
io.confluent:kafka-avro-serializer:5.3.2
io.confluent:kafka-schema-registry-client:5.3.2
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.104
io.grpc:grpc-alts:1.43.2
io.grpc:grpc-api:1.43.2
io.grpc:grpc-auth:1.43.2
io.grpc:grpc-context:1.43.2
io.grpc:grpc-core:1.43.2
io.grpc:grpc-grpclb:1.43.2
io.grpc:grpc-netty-shaded:1.43.2
io.grpc:grpc-netty:1.43.2
io.grpc:grpc-protobuf-lite:1.43.2
io.grpc:grpc-protobuf:1.43.2
io.grpc:grpc-services:1.43.2
io.grpc:grpc-stub:1.43.2
io.grpc:grpc-xds:1.43.2
io.netty:netty-buffer:4.1.63.Final
io.netty:netty-codec-http2:4.1.63.Final
io.netty:netty-codec-http:4.1.63.Final
io.netty:netty-codec-socks:4.1.63.Final
io.netty:netty-codec:4.1.63.Final
io.netty:netty-common:4.1.63.Final
io.netty:netty-handler-proxy:4.1.63.Final
io.netty:netty-handler:4.1.63.Final
io.netty:netty-resolver:4.1.63.Final
io.netty:netty-tcnative-boringssl-static:2.0.33.Final
io.netty:netty-transport:4.1.63.Final
io.opencensus:opencensus-api:0.30.0
io.opencensus:opencensus-contrib-grpc-util:0.30.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.grpc:grpc-alts:1.47.0
io.grpc:grpc-api:1.47.0
io.grpc:grpc-auth:1.47.0
io.grpc:grpc-census:1.45.1
io.grpc:grpc-context:1.47.0
io.grpc:grpc-core:1.47.0
io.grpc:grpc-googleapis:1.47.0
io.grpc:grpc-grpclb:1.47.0
io.grpc:grpc-netty-shaded:1.47.0
io.grpc:grpc-netty:1.45.1
io.grpc:grpc-protobuf-lite:1.47.0
io.grpc:grpc-protobuf:1.47.0
io.grpc:grpc-services:1.47.0
io.grpc:grpc-stub:1.47.0
io.grpc:grpc-xds:1.47.0
io.netty:netty-buffer:4.1.72.Final
io.netty:netty-codec-http2:4.1.72.Final
io.netty:netty-codec-http:4.1.72.Final
io.netty:netty-codec-socks:4.1.72.Final
io.netty:netty-codec:4.1.72.Final
io.netty:netty-common:4.1.72.Final
io.netty:netty-handler-proxy:4.1.72.Final
io.netty:netty-handler:4.1.72.Final
io.netty:netty-resolver:4.1.72.Final
io.netty:netty-tcnative-boringssl-static:2.0.46.Final
io.netty:netty-tcnative-classes:2.0.46.Final
io.netty:netty-transport:4.1.72.Final
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-exemplar-util:0.31.0
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0
io.opencensus:opencensus-contrib-grpc-util:0.31.0
io.opencensus:opencensus-contrib-http-util:0.31.1
io.opencensus:opencensus-contrib-resource-util:0.31.0
io.opencensus:opencensus-exporter-metrics-util:0.31.0
io.opencensus:opencensus-exporter-stats-stackdriver:0.31.0
io.opencensus:opencensus-impl-core:0.31.0
io.opencensus:opencensus-impl:0.31.0
io.opencensus:opencensus-proto:0.2.0
io.perfmark:perfmark-api:0.23.0
it.unimi.dsi:fastutil:6.5.16
io.perfmark:perfmark-api:0.25.0
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.jdo:jdo2-api:2.3-20090302111651
javax.mail:mail:1.5.0-b01
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.10
junit:junit:4.13.2
net.bytebuddy:byte-buddy-agent:1.10.19
net.bytebuddy:byte-buddy:1.10.19
net.java.dev.jna:jna:5.5.0
net.bytebuddy:byte-buddy-agent:1.12.10
net.bytebuddy:byte-buddy:1.12.10
net.java.dev.jna:jna:5.8.0
org.apache.arrow:arrow-format:5.0.0
org.apache.arrow:arrow-memory-core:5.0.0
org.apache.arrow:arrow-vector:5.0.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-fn-execution:2.37.0
org.apache.beam:beam-model-job-management:2.37.0
org.apache.beam:beam-model-pipeline:2.37.0
org.apache.beam:beam-runners-core-construction-java:2.37.0
org.apache.beam:beam-runners-core-java:2.37.0
org.apache.beam:beam-runners-direct-java:2.37.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
org.apache.beam:beam-runners-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-core:2.37.0
org.apache.beam:beam-sdks-java-expansion-service:2.37.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
org.apache.beam:beam-sdks-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
org.apache.beam:beam-sdks-java-io-kafka:2.37.0
org.apache.beam:beam-model-fn-execution:2.40.0
org.apache.beam:beam-model-job-management:2.40.0
org.apache.beam:beam-model-pipeline:2.40.0
org.apache.beam:beam-runners-core-construction-java:2.40.0
org.apache.beam:beam-runners-core-java:2.40.0
org.apache.beam:beam-runners-direct-java:2.40.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.40.0
org.apache.beam:beam-runners-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-core:2.40.0
org.apache.beam:beam-sdks-java-expansion-service:2.40.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.40.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.40.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.40.0
org.apache.beam:beam-sdks-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0
org.apache.beam:beam-sdks-java-io-kafka:2.40.0
org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
org.apache.beam:beam-vendor-grpc-1_43_2:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-compress:1.21
org.apache.commons:commons-csv:1.9.0
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.ftpserver:ftplet-api:1.0.6
org.apache.ftpserver:ftpserver-core:1.0.6
org.apache.commons:commons-lang3:3.11
org.apache.commons:commons-text:1.9
org.apache.ftpserver:ftplet-api:1.2.0
org.apache.ftpserver:ftpserver-core:1.2.0
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.15
org.apache.mina:mina-core:2.0.4
org.apache.mina:mina-core:2.1.6
org.apache.sshd:sshd-core:2.0.0
org.apache.sshd:sshd-scp:2.0.0
org.apache.sshd:sshd-sftp:2.0.0
org.apache.tomcat:tomcat-annotations-api:8.0.5
org.apiguardian:apiguardian-api:1.1.0
org.apache.tomcat:tomcat-annotations-api:10.1.0-M16
org.bouncycastle:bcpg-jdk15on:1.67
org.bouncycastle:bcpkix-jdk15on:1.67
org.bouncycastle:bcprov-jdk15on:1.67
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.21.0
org.checkerframework:checker-qual:3.22.2
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.20
org.codehaus.mojo:animal-sniffer-annotations:1.21
org.conscrypt:conscrypt-openjdk-uber:2.5.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.flywaydb:flyway-core:8.5.13
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.gwtproject:gwt-user:2.10.0
org.hamcrest:hamcrest-core:2.2
org.hamcrest:hamcrest-library:2.2
org.hamcrest:hamcrest:2.2
org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
org.hibernate:hibernate-core:5.4.23.Final
org.hibernate:hibernate-hikaricp:5.4.23.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.4.1.Final
org.hibernate:hibernate-core:5.6.10.Final
org.hibernate:hibernate-hikaricp:5.6.10.Final
org.jboss.logging:jboss-logging:3.4.3.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.1.3.Final
org.jboss:jandex:2.4.2.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20200518
org.jsoup:jsoup:1.13.1
org.junit-pioneer:junit-pioneer:0.7.0
org.junit.jupiter:junit-jupiter-api:5.7.0
org.junit.jupiter:junit-jupiter-engine:5.7.0
org.junit.jupiter:junit-jupiter-migrationsupport:5.7.0
org.junit.jupiter:junit-jupiter-params:5.7.0
org.junit.platform:junit-platform-commons:1.7.0
org.junit.platform:junit-platform-engine:1.7.0
org.junit.platform:junit-platform-launcher:1.7.0
org.junit.platform:junit-platform-runner:1.7.0
org.junit.platform:junit-platform-suite-api:1.7.0
org.junit:junit-bom:5.7.0
org.jsoup:jsoup:1.15.2
org.junit-pioneer:junit-pioneer:1.7.1
org.junit.jupiter:junit-jupiter-api:5.9.0-RC1
org.junit.jupiter:junit-jupiter-engine:5.9.0-RC1
org.junit.jupiter:junit-jupiter-migrationsupport:5.9.0-RC1
org.junit.jupiter:junit-jupiter-params:5.9.0-RC1
org.junit.platform:junit-platform-commons:1.9.0-RC1
org.junit.platform:junit-platform-engine:1.9.0-RC1
org.junit.platform:junit-platform-launcher:1.9.0-RC1
org.junit.platform:junit-platform-runner:1.9.0-RC1
org.junit.platform:junit-platform-suite-api:1.9.0-RC1
org.junit.platform:junit-platform-suite-commons:1.9.0-RC1
org.junit:junit-bom:5.9.0-RC1
org.jvnet.staxex:stax-ex:1.8
org.mockito:mockito-core:3.7.7
org.mockito:mockito-junit-jupiter:3.7.7
org.mockito:mockito-core:4.6.1
org.mockito:mockito-junit-jupiter:4.6.1
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:3.1
org.objenesis:objenesis:3.2
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:9.1
org.ow2.asm:asm-commons:9.0
org.ow2.asm:asm-tree:9.1
org.ow2.asm:asm-util:9.1
org.ow2.asm:asm:9.1
org.postgresql:postgresql:42.2.18
org.ow2.asm:asm-analysis:9.3
org.ow2.asm:asm-commons:9.2
org.ow2.asm:asm-tree:9.3
org.ow2.asm:asm-util:9.3
org.ow2.asm:asm:9.3
org.postgresql:postgresql:42.4.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.seleniumhq.selenium:selenium-api:3.141.59
org.seleniumhq.selenium:selenium-chrome-driver:3.141.59
org.seleniumhq.selenium:selenium-edge-driver:3.141.59
@@ -323,20 +333,21 @@ org.seleniumhq.selenium:selenium-opera-driver:3.141.59
org.seleniumhq.selenium:selenium-remote-driver:3.141.59
org.seleniumhq.selenium:selenium-safari-driver:3.141.59
org.seleniumhq.selenium:selenium-support:3.141.59
org.slf4j:slf4j-api:1.7.30
org.springframework:spring-core:4.3.18.RELEASE
org.springframework:spring-expression:4.3.18.RELEASE
org.testcontainers:database-commons:1.15.2
org.testcontainers:jdbc:1.15.2
org.testcontainers:junit-jupiter:1.15.2
org.testcontainers:postgresql:1.15.2
org.testcontainers:selenium:1.15.2
org.testcontainers:testcontainers:1.15.2
org.threeten:threetenbp:1.5.2
org.slf4j:slf4j-api:1.7.36
org.springframework:spring-core:5.3.18
org.springframework:spring-expression:5.3.18
org.springframework:spring-jcl:5.3.18
org.testcontainers:database-commons:1.17.3
org.testcontainers:jdbc:1.17.3
org.testcontainers:junit-jupiter:1.17.3
org.testcontainers:postgresql:1.17.3
org.testcontainers:selenium:1.17.3
org.testcontainers:testcontainers:1.17.3
org.threeten:threetenbp:1.6.0
org.tukaani:xz:1.5
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.8.4
org.yaml:snakeyaml:1.28
org.yaml:snakeyaml:1.30
us.fatehi:schemacrawler-api:16.10.1
us.fatehi:schemacrawler-diagram:16.10.1
us.fatehi:schemacrawler-tools:16.10.1

View File

@@ -4,138 +4,141 @@
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.101tec:zkclient:0.10
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.13.0
com.fasterxml.jackson.core:jackson-core:2.13.0
com.fasterxml.jackson.core:jackson-databind:2.13.0
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
com.fasterxml.jackson:jackson-bom:2.13.0
com.fasterxml.jackson.core:jackson-annotations:2.13.3
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson.core:jackson-databind:2.13.3
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.fasterxml:classmate:1.5.1
com.github.docker-java:docker-java-api:3.2.7
com.github.docker-java:docker-java-transport-zerodep:3.2.7
com.github.docker-java:docker-java-transport:3.2.7
com.github.jnr:jffi:1.3.1
com.github.ben-manes.caffeine:caffeine:2.9.3
com.github.docker-java:docker-java-api:3.2.13
com.github.docker-java:docker-java-transport-zerodep:3.2.13
com.github.docker-java:docker-java-transport:3.2.13
com.github.jnr:jffi:1.3.9
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.10.1
com.github.jnr:jnr-enxio:0.32.3
com.github.jnr:jnr-ffi:2.2.1
com.github.jnr:jnr-posix:3.1.4
com.github.jnr:jnr-unixsocket:0.38.5
com.github.jnr:jnr-constants:0.10.3
com.github.jnr:jnr-enxio:0.32.13
com.github.jnr:jnr-ffi:2.2.11
com.github.jnr:jnr-posix:3.1.15
com.github.jnr:jnr-unixsocket:0.38.17
com.github.jnr:jnr-x86asm:1.0.2
com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-appengine:1.35.2
com.google.api-client:google-api-client-jackson2:1.32.2
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.32.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api-client:google-api-client-java6:1.35.2
com.google.api-client:google-api-client-servlet:1.35.2
com.google.api-client:google-api-client:1.35.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:grpc-google-common-protos:2.7.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
com.google.api.grpc:proto-google-common-protos:2.7.1
com.google.api.grpc:proto-google-iam-v1:1.2.0
com.google.api:api-common:2.1.2
com.google.api:gax-grpc:2.8.1
com.google.api:gax-httpjson:0.93.1
com.google.api:gax:2.8.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:grpc-google-common-protos:2.8.3
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.6.2
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.10
com.google.api.grpc:proto-google-cloud-firestore-v1:3.1.0
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.3.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.3.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:proto-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:2.3.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.93.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.93.0
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-grpc:2.18.2
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
com.google.apis:google-api-services-cloudkms:v1-rev20220617-1.32.1
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
com.google.apis:google-api-services-drive:v2-rev393-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev20210624-1.32.1
com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-monitoring:v3-rev20220626-1.32.1
com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220513-1.32.1
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.86
com.google.appengine:appengine-api-stubs:1.9.86
com.google.appengine:appengine-remote-api:1.9.86
com.google.appengine:appengine-api-1.0-sdk:2.0.5
com.google.appengine:appengine-api-stubs:2.0.5
com.google.appengine:appengine-remote-api:2.0.5
com.google.appengine:appengine-testing:1.9.86
com.google.auth:google-auth-library-credentials:1.3.0
com.google.auth:google-auth-library-oauth2-http:1.3.0
com.google.auto.service:auto-service-annotations:1.0-rc7
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.service:auto-service-annotations:1.0.1
com.google.auto.value:auto-value-annotations:1.9
com.google.auto.value:auto-value:1.7.4
com.google.cloud.bigdataoss:gcsio:2.2.4
com.google.cloud.bigdataoss:util:2.2.4
com.google.cloud.bigtable:bigtable-client-core:1.25.1
com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
com.google.cloud:google-cloud-bigquerystorage:2.8.1
com.google.cloud:google-cloud-bigtable:1.27.1
com.google.cloud:google-cloud-core-grpc:2.3.5
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:2.3.5
com.google.cloud:google-cloud-firestore:3.0.10
com.google.cloud:google-cloud-nio:0.123.4
com.google.cloud:google-cloud-pubsub:1.115.1
com.google.cloud:google-cloud-pubsublite:1.4.8
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:6.17.4
com.google.cloud:google-cloud-storage:1.118.0
com.google.cloud:google-cloud-tasks:1.33.2
com.google.auto.value:auto-value:1.9
com.google.cloud.bigdataoss:gcsio:2.2.6
com.google.cloud.bigdataoss:util:2.2.6
com.google.cloud.bigtable:bigtable-client-core:1.26.3
com.google.cloud.bigtable:bigtable-metrics-api:1.26.3
com.google.cloud.datastore:datastore-v1-proto-client:2.2.10
com.google.cloud.sql:jdbc-socket-factory-core:1.6.1
com.google.cloud:google-cloud-bigquerystorage:2.12.2
com.google.cloud:google-cloud-bigtable:2.6.2
com.google.cloud:google-cloud-core-grpc:2.6.0
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-firestore:3.1.0
com.google.cloud:google-cloud-monitoring:1.82.0
com.google.cloud:google-cloud-nio:0.124.7
com.google.cloud:google-cloud-pubsub:1.116.4
com.google.cloud:google-cloud-pubsublite:1.5.4
com.google.cloud:google-cloud-secretmanager:2.3.0
com.google.cloud:google-cloud-spanner:6.23.3
com.google.cloud:google-cloud-storage:2.9.0
com.google.cloud:google-cloud-tasks:2.3.0
com.google.cloud:grpc-gcp:1.1.0
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.1.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.9
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.10.0
com.google.dagger:dagger:2.42
com.google.errorprone:error_prone_annotations:2.14.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flatbuffers:flatbuffers-java:1.12.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.flogger:google-extensions:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava-testlib:30.1.1-jre
com.google.guava:guava:31.0.1-jre
com.google.guava:guava-testlib:31.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.9.0
com.google.http-client:google-http-client-apache-v2:1.40.1
com.google.http-client:google-http-client-appengine:1.39.2
com.google.http-client:google-http-client-gson:1.41.0
com.google.http-client:google-http-client-jackson2:1.41.0
com.google.http-client:google-http-client-protobuf:1.40.1
com.google.http-client:google-http-client:1.41.0
com.google.gwt:gwt-user:2.10.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client-protobuf:1.41.7
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -143,21 +146,21 @@ com.google.jsinterop:jsinterop-annotations:2.0.0
com.google.monitoring-client:contrib:1.0.7
com.google.monitoring-client:metrics:1.0.7
com.google.monitoring-client:stackdriver:1.0.7
com.google.oauth-client:google-oauth-client-appengine:1.31.4
com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.32.1
com.google.protobuf:protobuf-java-util:3.19.2
com.google.protobuf:protobuf-java:3.19.2
com.google.re2j:re2j:1.6
com.google.oauth-client:google-oauth-client-appengine:1.34.1
com.google.oauth-client:google-oauth-client-java6:1.34.1
com.google.oauth-client:google-oauth-client-jetty:1.34.1
com.google.oauth-client:google-oauth-client-servlet:1.34.1
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.re2j:re2j:1.7
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.googlecode.charts4j:charts4j:1.3
com.google.truth.extensions:truth-java8-extension:1.1.3
com.google.truth:truth:1.1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
com.ibm.icu:icu4j:71.1
com.jcraft:jsch:0.1.55
com.lmax:disruptor:3.4.2
com.squareup.okhttp3:okhttp:3.11.0
com.squareup.okio:okio:1.14.0
com.sun.istack:istack-commons-runtime:3.0.7
@@ -167,144 +170,152 @@ com.thoughtworks.qdox:qdox:1.12.1
com.zaxxer:HikariCP:3.4.5
commons-codec:commons-codec:1.15
commons-logging:commons-logging:1.2
dnsjava:dnsjava:3.3.1
dnsjava:dnsjava:3.5.1
io.confluent:common-config:5.3.2
io.confluent:common-utils:5.3.2
io.confluent:kafka-avro-serializer:5.3.2
io.confluent:kafka-schema-registry-client:5.3.2
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.104
io.grpc:grpc-alts:1.43.2
io.grpc:grpc-api:1.43.2
io.grpc:grpc-auth:1.43.2
io.grpc:grpc-context:1.43.2
io.grpc:grpc-core:1.43.2
io.grpc:grpc-grpclb:1.43.2
io.grpc:grpc-netty-shaded:1.43.2
io.grpc:grpc-netty:1.43.2
io.grpc:grpc-protobuf-lite:1.43.2
io.grpc:grpc-protobuf:1.43.2
io.grpc:grpc-stub:1.43.2
io.netty:netty-buffer:4.1.63.Final
io.netty:netty-codec-http2:4.1.63.Final
io.netty:netty-codec-http:4.1.63.Final
io.netty:netty-codec:4.1.63.Final
io.netty:netty-common:4.1.63.Final
io.netty:netty-handler:4.1.63.Final
io.netty:netty-resolver:4.1.63.Final
io.netty:netty-tcnative-boringssl-static:2.0.33.Final
io.netty:netty-transport:4.1.63.Final
io.opencensus:opencensus-api:0.30.0
io.opencensus:opencensus-contrib-grpc-util:0.30.0
io.opencensus:opencensus-contrib-http-util:0.28.0
it.unimi.dsi:fastutil:6.5.16
io.grpc:grpc-alts:1.45.1
io.grpc:grpc-api:1.47.0
io.grpc:grpc-auth:1.45.1
io.grpc:grpc-census:1.45.1
io.grpc:grpc-context:1.47.0
io.grpc:grpc-core:1.45.1
io.grpc:grpc-grpclb:1.45.1
io.grpc:grpc-netty-shaded:1.45.1
io.grpc:grpc-netty:1.45.1
io.grpc:grpc-protobuf-lite:1.47.0
io.grpc:grpc-protobuf:1.47.0
io.grpc:grpc-services:1.45.1
io.grpc:grpc-stub:1.47.0
io.grpc:grpc-xds:1.45.1
io.netty:netty-buffer:4.1.72.Final
io.netty:netty-codec-http2:4.1.72.Final
io.netty:netty-codec-http:4.1.72.Final
io.netty:netty-codec:4.1.72.Final
io.netty:netty-common:4.1.72.Final
io.netty:netty-handler:4.1.72.Final
io.netty:netty-resolver:4.1.72.Final
io.netty:netty-tcnative-boringssl-static:2.0.46.Final
io.netty:netty-tcnative-classes:2.0.46.Final
io.netty:netty-transport:4.1.72.Final
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-exemplar-util:0.31.0
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0
io.opencensus:opencensus-contrib-grpc-util:0.31.0
io.opencensus:opencensus-contrib-http-util:0.31.1
io.opencensus:opencensus-contrib-resource-util:0.31.0
io.opencensus:opencensus-exporter-metrics-util:0.31.0
io.opencensus:opencensus-exporter-stats-stackdriver:0.31.0
io.opencensus:opencensus-impl-core:0.31.0
io.opencensus:opencensus-impl:0.31.0
io.opencensus:opencensus-proto:0.2.0
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.jdo:jdo2-api:2.3-20090302111651
javax.mail:mail:1.5.0-b01
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.10
junit:junit:4.13.2
net.bytebuddy:byte-buddy-agent:1.10.19
net.bytebuddy:byte-buddy:1.10.19
net.java.dev.jna:jna:5.5.0
net.bytebuddy:byte-buddy-agent:1.12.10
net.bytebuddy:byte-buddy:1.12.10
net.java.dev.jna:jna:5.8.0
org.apache.arrow:arrow-format:5.0.0
org.apache.arrow:arrow-memory-core:5.0.0
org.apache.arrow:arrow-vector:5.0.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-fn-execution:2.37.0
org.apache.beam:beam-model-job-management:2.37.0
org.apache.beam:beam-model-pipeline:2.37.0
org.apache.beam:beam-runners-core-construction-java:2.37.0
org.apache.beam:beam-runners-core-java:2.37.0
org.apache.beam:beam-runners-direct-java:2.37.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
org.apache.beam:beam-runners-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-core:2.37.0
org.apache.beam:beam-sdks-java-expansion-service:2.37.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
org.apache.beam:beam-sdks-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
org.apache.beam:beam-sdks-java-io-kafka:2.37.0
org.apache.beam:beam-model-fn-execution:2.40.0
org.apache.beam:beam-model-job-management:2.40.0
org.apache.beam:beam-model-pipeline:2.40.0
org.apache.beam:beam-runners-core-construction-java:2.40.0
org.apache.beam:beam-runners-core-java:2.40.0
org.apache.beam:beam-runners-direct-java:2.40.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.40.0
org.apache.beam:beam-runners-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-core:2.40.0
org.apache.beam:beam-sdks-java-expansion-service:2.40.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.40.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.40.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.40.0
org.apache.beam:beam-sdks-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0
org.apache.beam:beam-sdks-java-io-kafka:2.40.0
org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
org.apache.beam:beam-vendor-grpc-1_43_2:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-compress:1.21
org.apache.commons:commons-csv:1.9.0
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.ftpserver:ftplet-api:1.0.6
org.apache.ftpserver:ftpserver-core:1.0.6
org.apache.commons:commons-lang3:3.11
org.apache.commons:commons-text:1.9
org.apache.ftpserver:ftplet-api:1.2.0
org.apache.ftpserver:ftpserver-core:1.2.0
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.15
org.apache.mina:mina-core:2.0.4
org.apache.mina:mina-core:2.1.6
org.apache.sshd:sshd-core:2.0.0
org.apache.sshd:sshd-scp:2.0.0
org.apache.sshd:sshd-sftp:2.0.0
org.apache.tomcat:tomcat-annotations-api:8.0.5
org.apiguardian:apiguardian-api:1.1.0
org.apache.tomcat:tomcat-annotations-api:10.1.0-M16
org.apiguardian:apiguardian-api:1.1.2
org.bouncycastle:bcpg-jdk15on:1.67
org.bouncycastle:bcpkix-jdk15on:1.67
org.bouncycastle:bcprov-jdk15on:1.67
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.21.0
org.checkerframework:checker-qual:3.22.2
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.conscrypt:conscrypt-openjdk-uber:2.5.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.flywaydb:flyway-core:8.5.13
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.gwtproject:gwt-user:2.10.0
org.hamcrest:hamcrest-core:2.2
org.hamcrest:hamcrest-library:2.2
org.hamcrest:hamcrest:2.2
org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
org.hibernate:hibernate-core:5.4.23.Final
org.hibernate:hibernate-hikaricp:5.4.23.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.4.1.Final
org.hibernate:hibernate-core:5.6.10.Final
org.hibernate:hibernate-hikaricp:5.6.10.Final
org.jboss.logging:jboss-logging:3.4.3.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.1.3.Final
org.jboss:jandex:2.4.2.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20200518
org.jsoup:jsoup:1.13.1
org.junit-pioneer:junit-pioneer:0.7.0
org.junit.jupiter:junit-jupiter-api:5.7.0
org.junit.jupiter:junit-jupiter-engine:5.7.0
org.junit.jupiter:junit-jupiter-migrationsupport:5.7.0
org.junit.jupiter:junit-jupiter-params:5.7.0
org.junit.platform:junit-platform-commons:1.7.0
org.junit.platform:junit-platform-engine:1.7.0
org.junit.platform:junit-platform-launcher:1.7.0
org.junit.platform:junit-platform-runner:1.7.0
org.junit.platform:junit-platform-suite-api:1.7.0
org.junit:junit-bom:5.7.0
org.jsoup:jsoup:1.15.2
org.junit-pioneer:junit-pioneer:1.7.1
org.junit.jupiter:junit-jupiter-api:5.9.0-RC1
org.junit.jupiter:junit-jupiter-engine:5.9.0-RC1
org.junit.jupiter:junit-jupiter-migrationsupport:5.9.0-RC1
org.junit.jupiter:junit-jupiter-params:5.9.0-RC1
org.junit.platform:junit-platform-commons:1.9.0-RC1
org.junit.platform:junit-platform-engine:1.9.0-RC1
org.junit.platform:junit-platform-launcher:1.9.0-RC1
org.junit.platform:junit-platform-runner:1.9.0-RC1
org.junit.platform:junit-platform-suite-api:1.9.0-RC1
org.junit:junit-bom:5.9.0-RC1
org.jvnet.staxex:stax-ex:1.8
org.mockito:mockito-core:3.7.7
org.mockito:mockito-junit-jupiter:3.7.7
org.mockito:mockito-core:4.6.1
org.mockito:mockito-junit-jupiter:4.6.1
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:3.1
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:9.1
org.ow2.asm:asm-commons:9.0
org.ow2.asm:asm-tree:9.1
org.ow2.asm:asm-util:9.1
org.ow2.asm:asm:9.1
org.postgresql:postgresql:42.2.18
org.ow2.asm:asm-analysis:9.3
org.ow2.asm:asm-commons:9.2
org.ow2.asm:asm-tree:9.3
org.ow2.asm:asm-util:9.3
org.ow2.asm:asm:9.3
org.postgresql:postgresql:42.4.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.seleniumhq.selenium:selenium-api:3.141.59
org.seleniumhq.selenium:selenium-chrome-driver:3.141.59
org.seleniumhq.selenium:selenium-edge-driver:3.141.59
@@ -315,20 +326,21 @@ org.seleniumhq.selenium:selenium-opera-driver:3.141.59
org.seleniumhq.selenium:selenium-remote-driver:3.141.59
org.seleniumhq.selenium:selenium-safari-driver:3.141.59
org.seleniumhq.selenium:selenium-support:3.141.59
org.slf4j:slf4j-api:1.7.30
org.springframework:spring-core:4.3.18.RELEASE
org.springframework:spring-expression:4.3.18.RELEASE
org.testcontainers:database-commons:1.15.2
org.testcontainers:jdbc:1.15.2
org.testcontainers:junit-jupiter:1.15.2
org.testcontainers:postgresql:1.15.2
org.testcontainers:selenium:1.15.2
org.testcontainers:testcontainers:1.15.2
org.threeten:threetenbp:1.5.2
org.slf4j:slf4j-api:1.7.36
org.springframework:spring-core:5.3.18
org.springframework:spring-expression:5.3.18
org.springframework:spring-jcl:5.3.18
org.testcontainers:database-commons:1.17.3
org.testcontainers:jdbc:1.17.3
org.testcontainers:junit-jupiter:1.17.3
org.testcontainers:postgresql:1.17.3
org.testcontainers:selenium:1.17.3
org.testcontainers:testcontainers:1.17.3
org.threeten:threetenbp:1.6.0
org.tukaani:xz:1.5
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.8.4
org.yaml:snakeyaml:1.28
org.yaml:snakeyaml:1.30
us.fatehi:schemacrawler-api:16.10.1
us.fatehi:schemacrawler-diagram:16.10.1
us.fatehi:schemacrawler-tools:16.10.1

View File

@@ -4,144 +4,143 @@
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.101tec:zkclient:0.10
com.beust:jcommander:1.60
com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.13.0
com.fasterxml.jackson.core:jackson-core:2.13.0
com.fasterxml.jackson.core:jackson-databind:2.13.0
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
com.fasterxml.jackson:jackson-bom:2.13.0
com.fasterxml.jackson.core:jackson-annotations:2.13.3
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson.core:jackson-databind:2.13.3
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.fasterxml:classmate:1.5.1
com.github.docker-java:docker-java-api:3.2.7
com.github.docker-java:docker-java-transport-zerodep:3.2.7
com.github.docker-java:docker-java-transport:3.2.7
com.github.jnr:jffi:1.3.1
com.github.ben-manes.caffeine:caffeine:2.9.3
com.github.docker-java:docker-java-api:3.2.13
com.github.docker-java:docker-java-transport-zerodep:3.2.13
com.github.docker-java:docker-java-transport:3.2.13
com.github.jnr:jffi:1.3.9
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.10.1
com.github.jnr:jnr-enxio:0.32.3
com.github.jnr:jnr-ffi:2.2.1
com.github.jnr:jnr-posix:3.1.4
com.github.jnr:jnr-unixsocket:0.38.5
com.github.jnr:jnr-constants:0.10.3
com.github.jnr:jnr-enxio:0.32.13
com.github.jnr:jnr-ffi:2.2.11
com.github.jnr:jnr-posix:3.1.15
com.github.jnr:jnr-unixsocket:0.38.17
com.github.jnr:jnr-x86asm:1.0.2
com.google.android:annotations:4.1.1.4
com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-appengine:1.35.2
com.google.api-client:google-api-client-jackson2:1.32.2
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.32.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api-client:google-api-client-java6:1.35.2
com.google.api-client:google-api-client-servlet:1.35.2
com.google.api-client:google-api-client:1.35.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:grpc-google-common-protos:2.7.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
com.google.api.grpc:proto-google-common-protos:2.7.1
com.google.api.grpc:proto-google-iam-v1:1.2.0
com.google.api:api-common:2.1.2
com.google.api:gax-grpc:2.8.1
com.google.api:gax-httpjson:0.93.1
com.google.api:gax:2.8.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:grpc-google-common-protos:2.8.3
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.6.2
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.10
com.google.api.grpc:proto-google-cloud-firestore-v1:3.1.0
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.3.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.3.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:proto-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:2.3.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.93.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.93.0
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-grpc:2.18.2
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
com.google.apis:google-api-services-cloudkms:v1-rev20220617-1.32.1
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
com.google.apis:google-api-services-drive:v2-rev393-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev20210624-1.32.1
com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-monitoring:v3-rev20220626-1.32.1
com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220513-1.32.1
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.86
com.google.appengine:appengine-api-stubs:1.9.86
com.google.appengine:appengine-remote-api:1.9.86
com.google.appengine:appengine-api-1.0-sdk:2.0.5
com.google.appengine:appengine-api-stubs:2.0.5
com.google.appengine:appengine-remote-api:2.0.5
com.google.appengine:appengine-testing:1.9.86
com.google.auth:google-auth-library-credentials:1.3.0
com.google.auth:google-auth-library-oauth2-http:1.3.0
com.google.auto.service:auto-service-annotations:1.0-rc7
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.service:auto-service-annotations:1.0.1
com.google.auto.value:auto-value-annotations:1.9
com.google.auto.value:auto-value:1.7.4
com.google.cloud.bigdataoss:gcsio:2.2.4
com.google.cloud.bigdataoss:util:2.2.4
com.google.cloud.bigtable:bigtable-client-core:1.25.1
com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
com.google.cloud.sql:postgres-socket-factory:1.2.1
com.google.cloud:google-cloud-bigquerystorage:2.8.1
com.google.cloud:google-cloud-bigtable:1.27.1
com.google.cloud:google-cloud-core-grpc:2.3.5
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:2.3.5
com.google.cloud:google-cloud-firestore:3.0.10
com.google.cloud:google-cloud-nio:0.123.4
com.google.cloud:google-cloud-pubsub:1.115.1
com.google.cloud:google-cloud-pubsublite:1.4.8
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:6.17.4
com.google.cloud:google-cloud-storage:1.118.0
com.google.cloud:google-cloud-tasks:1.33.2
com.google.auto.value:auto-value:1.9
com.google.cloud.bigdataoss:gcsio:2.2.6
com.google.cloud.bigdataoss:util:2.2.6
com.google.cloud.bigtable:bigtable-client-core:1.26.3
com.google.cloud.bigtable:bigtable-metrics-api:1.26.3
com.google.cloud.datastore:datastore-v1-proto-client:2.2.10
com.google.cloud.sql:jdbc-socket-factory-core:1.6.1
com.google.cloud.sql:postgres-socket-factory:1.6.1
com.google.cloud:google-cloud-bigquerystorage:2.12.2
com.google.cloud:google-cloud-bigtable:2.6.2
com.google.cloud:google-cloud-core-grpc:2.6.0
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-firestore:3.1.0
com.google.cloud:google-cloud-monitoring:1.82.0
com.google.cloud:google-cloud-nio:0.124.7
com.google.cloud:google-cloud-pubsub:1.116.4
com.google.cloud:google-cloud-pubsublite:1.5.4
com.google.cloud:google-cloud-secretmanager:2.3.0
com.google.cloud:google-cloud-spanner:6.23.3
com.google.cloud:google-cloud-storage:2.9.0
com.google.cloud:google-cloud-tasks:2.3.0
com.google.cloud:grpc-gcp:1.1.0
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.1.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.9
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.10.0
com.google.dagger:dagger:2.42
com.google.errorprone:error_prone_annotations:2.14.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flatbuffers:flatbuffers-java:1.12.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.flogger:google-extensions:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava-testlib:30.1.1-jre
com.google.guava:guava:31.0.1-jre
com.google.guava:guava-testlib:31.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.9.0
com.google.http-client:google-http-client-apache-v2:1.40.1
com.google.http-client:google-http-client-appengine:1.39.2
com.google.http-client:google-http-client-gson:1.41.0
com.google.http-client:google-http-client-jackson2:1.41.0
com.google.http-client:google-http-client-protobuf:1.40.1
com.google.http-client:google-http-client:1.41.0
com.google.gwt:gwt-user:2.10.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client-protobuf:1.41.7
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -149,21 +148,21 @@ com.google.jsinterop:jsinterop-annotations:2.0.0
com.google.monitoring-client:contrib:1.0.7
com.google.monitoring-client:metrics:1.0.7
com.google.monitoring-client:stackdriver:1.0.7
com.google.oauth-client:google-oauth-client-appengine:1.31.4
com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.32.1
com.google.protobuf:protobuf-java-util:3.19.2
com.google.protobuf:protobuf-java:3.19.2
com.google.re2j:re2j:1.6
com.google.oauth-client:google-oauth-client-appengine:1.34.1
com.google.oauth-client:google-oauth-client-java6:1.34.1
com.google.oauth-client:google-oauth-client-jetty:1.34.1
com.google.oauth-client:google-oauth-client-servlet:1.34.1
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.re2j:re2j:1.7
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.googlecode.charts4j:charts4j:1.3
com.google.truth.extensions:truth-java8-extension:1.1.3
com.google.truth:truth:1.1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
com.ibm.icu:icu4j:71.1
com.jcraft:jsch:0.1.55
com.lmax:disruptor:3.4.2
com.squareup.okhttp3:okhttp:3.11.0
com.squareup.okio:okio:1.14.0
com.sun.istack:istack-commons-runtime:3.0.7
@@ -173,156 +172,167 @@ com.thoughtworks.qdox:qdox:1.12.1
com.zaxxer:HikariCP:3.4.5
commons-codec:commons-codec:1.15
commons-logging:commons-logging:1.2
dnsjava:dnsjava:3.3.1
dnsjava:dnsjava:3.5.1
guru.nidi.com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
guru.nidi.com.kitfox:svgSalamander:1.1.3
guru.nidi:graphviz-java-all-j2v8:0.17.0
guru.nidi:graphviz-java:0.17.0
guru.nidi:graphviz-java-all-j2v8:0.18.1
guru.nidi:graphviz-java:0.18.1
io.confluent:common-config:5.3.2
io.confluent:common-utils:5.3.2
io.confluent:kafka-avro-serializer:5.3.2
io.confluent:kafka-schema-registry-client:5.3.2
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.104
io.github.java-diff-utils:java-diff-utils:4.9
io.grpc:grpc-alts:1.43.2
io.grpc:grpc-api:1.43.2
io.grpc:grpc-auth:1.43.2
io.grpc:grpc-context:1.43.2
io.grpc:grpc-core:1.43.2
io.grpc:grpc-grpclb:1.43.2
io.grpc:grpc-netty-shaded:1.43.2
io.grpc:grpc-netty:1.43.2
io.grpc:grpc-protobuf-lite:1.43.2
io.grpc:grpc-protobuf:1.43.2
io.grpc:grpc-services:1.43.2
io.grpc:grpc-stub:1.43.2
io.grpc:grpc-xds:1.43.2
io.netty:netty-buffer:4.1.63.Final
io.netty:netty-codec-http2:4.1.63.Final
io.netty:netty-codec-http:4.1.63.Final
io.netty:netty-codec-socks:4.1.63.Final
io.netty:netty-codec:4.1.63.Final
io.netty:netty-common:4.1.63.Final
io.netty:netty-handler-proxy:4.1.63.Final
io.netty:netty-handler:4.1.63.Final
io.netty:netty-resolver:4.1.63.Final
io.netty:netty-tcnative-boringssl-static:2.0.33.Final
io.netty:netty-transport:4.1.63.Final
io.opencensus:opencensus-api:0.30.0
io.opencensus:opencensus-contrib-grpc-util:0.30.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.github.java-diff-utils:java-diff-utils:4.11
io.grpc:grpc-alts:1.47.0
io.grpc:grpc-api:1.47.0
io.grpc:grpc-auth:1.47.0
io.grpc:grpc-census:1.45.1
io.grpc:grpc-context:1.47.0
io.grpc:grpc-core:1.47.0
io.grpc:grpc-googleapis:1.47.0
io.grpc:grpc-grpclb:1.47.0
io.grpc:grpc-netty-shaded:1.47.0
io.grpc:grpc-netty:1.45.1
io.grpc:grpc-protobuf-lite:1.47.0
io.grpc:grpc-protobuf:1.47.0
io.grpc:grpc-services:1.47.0
io.grpc:grpc-stub:1.47.0
io.grpc:grpc-xds:1.47.0
io.netty:netty-buffer:4.1.72.Final
io.netty:netty-codec-http2:4.1.72.Final
io.netty:netty-codec-http:4.1.72.Final
io.netty:netty-codec-socks:4.1.72.Final
io.netty:netty-codec:4.1.72.Final
io.netty:netty-common:4.1.72.Final
io.netty:netty-handler-proxy:4.1.72.Final
io.netty:netty-handler:4.1.72.Final
io.netty:netty-resolver:4.1.72.Final
io.netty:netty-tcnative-boringssl-static:2.0.46.Final
io.netty:netty-tcnative-classes:2.0.46.Final
io.netty:netty-transport:4.1.72.Final
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-exemplar-util:0.31.0
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0
io.opencensus:opencensus-contrib-grpc-util:0.31.0
io.opencensus:opencensus-contrib-http-util:0.31.1
io.opencensus:opencensus-contrib-resource-util:0.31.0
io.opencensus:opencensus-exporter-metrics-util:0.31.0
io.opencensus:opencensus-exporter-stats-stackdriver:0.31.0
io.opencensus:opencensus-impl-core:0.31.0
io.opencensus:opencensus-impl:0.31.0
io.opencensus:opencensus-proto:0.2.0
io.perfmark:perfmark-api:0.23.0
it.unimi.dsi:fastutil:6.5.16
io.perfmark:perfmark-api:0.25.0
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.jdo:jdo2-api:2.3-20090302111651
javax.mail:mail:1.5.0-b01
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.10
junit:junit:4.13.2
net.arnx:nashorn-promise:0.1.1
net.bytebuddy:byte-buddy-agent:1.10.19
net.bytebuddy:byte-buddy:1.10.19
net.java.dev.jna:jna:5.5.0
net.bytebuddy:byte-buddy-agent:1.12.10
net.bytebuddy:byte-buddy:1.12.10
net.java.dev.jna:jna:5.8.0
org.apache.arrow:arrow-format:5.0.0
org.apache.arrow:arrow-memory-core:5.0.0
org.apache.arrow:arrow-vector:5.0.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-fn-execution:2.37.0
org.apache.beam:beam-model-job-management:2.37.0
org.apache.beam:beam-model-pipeline:2.37.0
org.apache.beam:beam-runners-core-construction-java:2.37.0
org.apache.beam:beam-runners-core-java:2.37.0
org.apache.beam:beam-runners-direct-java:2.37.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
org.apache.beam:beam-runners-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-core:2.37.0
org.apache.beam:beam-sdks-java-expansion-service:2.37.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
org.apache.beam:beam-sdks-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
org.apache.beam:beam-sdks-java-io-kafka:2.37.0
org.apache.beam:beam-model-fn-execution:2.40.0
org.apache.beam:beam-model-job-management:2.40.0
org.apache.beam:beam-model-pipeline:2.40.0
org.apache.beam:beam-runners-core-construction-java:2.40.0
org.apache.beam:beam-runners-core-java:2.40.0
org.apache.beam:beam-runners-direct-java:2.40.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.40.0
org.apache.beam:beam-runners-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-core:2.40.0
org.apache.beam:beam-sdks-java-expansion-service:2.40.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.40.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.40.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.40.0
org.apache.beam:beam-sdks-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0
org.apache.beam:beam-sdks-java-io-kafka:2.40.0
org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
org.apache.beam:beam-vendor-grpc-1_43_2:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-compress:1.21
org.apache.commons:commons-csv:1.9.0
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.ftpserver:ftplet-api:1.0.6
org.apache.ftpserver:ftpserver-core:1.0.6
org.apache.commons:commons-lang3:3.11
org.apache.commons:commons-text:1.9
org.apache.ftpserver:ftplet-api:1.2.0
org.apache.ftpserver:ftpserver-core:1.2.0
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.15
org.apache.mina:mina-core:2.0.4
org.apache.mina:mina-core:2.1.6
org.apache.sshd:sshd-core:2.0.0
org.apache.sshd:sshd-scp:2.0.0
org.apache.sshd:sshd-sftp:2.0.0
org.apache.tomcat:tomcat-annotations-api:8.0.5
org.apiguardian:apiguardian-api:1.1.0
org.apache.tomcat:tomcat-annotations-api:10.1.0-M16
org.bouncycastle:bcpg-jdk15on:1.67
org.bouncycastle:bcpkix-jdk15on:1.67
org.bouncycastle:bcprov-jdk15on:1.67
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.21.0
org.checkerframework:checker-qual:3.22.2
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.20
org.codehaus.mojo:animal-sniffer-annotations:1.21
org.conscrypt:conscrypt-openjdk-uber:2.5.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.flywaydb:flyway-core:8.5.13
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.gwtproject:gwt-user:2.10.0
org.hamcrest:hamcrest-core:2.2
org.hamcrest:hamcrest-library:2.2
org.hamcrest:hamcrest:2.2
org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
org.hibernate:hibernate-core:5.4.23.Final
org.hibernate:hibernate-hikaricp:5.4.23.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.4.1.Final
org.hibernate:hibernate-core:5.6.10.Final
org.hibernate:hibernate-hikaricp:5.6.10.Final
org.jboss.logging:jboss-logging:3.4.3.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.1.3.Final
org.jboss:jandex:2.4.2.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20200518
org.jsoup:jsoup:1.13.1
org.junit-pioneer:junit-pioneer:0.7.0
org.junit.jupiter:junit-jupiter-api:5.7.0
org.junit.jupiter:junit-jupiter-engine:5.7.0
org.junit.jupiter:junit-jupiter-migrationsupport:5.7.0
org.junit.jupiter:junit-jupiter-params:5.7.0
org.junit.platform:junit-platform-commons:1.7.0
org.junit.platform:junit-platform-engine:1.7.0
org.junit.platform:junit-platform-launcher:1.7.0
org.junit.platform:junit-platform-runner:1.7.0
org.junit.platform:junit-platform-suite-api:1.7.0
org.junit:junit-bom:5.7.0
org.jsoup:jsoup:1.15.2
org.junit-pioneer:junit-pioneer:1.7.1
org.junit.jupiter:junit-jupiter-api:5.9.0-RC1
org.junit.jupiter:junit-jupiter-engine:5.9.0-RC1
org.junit.jupiter:junit-jupiter-migrationsupport:5.9.0-RC1
org.junit.jupiter:junit-jupiter-params:5.9.0-RC1
org.junit.platform:junit-platform-commons:1.9.0-RC1
org.junit.platform:junit-platform-engine:1.9.0-RC1
org.junit.platform:junit-platform-launcher:1.9.0-RC1
org.junit.platform:junit-platform-runner:1.9.0-RC1
org.junit.platform:junit-platform-suite-api:1.9.0-RC1
org.junit.platform:junit-platform-suite-commons:1.9.0-RC1
org.junit:junit-bom:5.9.0-RC1
org.jvnet.staxex:stax-ex:1.8
org.mockito:mockito-core:3.7.7
org.mockito:mockito-junit-jupiter:3.7.7
org.mockito:mockito-core:4.6.1
org.mockito:mockito-junit-jupiter:4.6.1
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:3.1
org.objenesis:objenesis:3.2
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:9.1
org.ow2.asm:asm-commons:9.0
org.ow2.asm:asm-tree:9.1
org.ow2.asm:asm-util:9.1
org.ow2.asm:asm:9.1
org.postgresql:postgresql:42.2.18
org.ow2.asm:asm-analysis:9.3
org.ow2.asm:asm-commons:9.2
org.ow2.asm:asm-tree:9.3
org.ow2.asm:asm-util:9.3
org.ow2.asm:asm:9.3
org.postgresql:postgresql:42.4.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.seleniumhq.selenium:selenium-api:3.141.59
org.seleniumhq.selenium:selenium-chrome-driver:3.141.59
org.seleniumhq.selenium:selenium-edge-driver:3.141.59
@@ -335,21 +345,22 @@ org.seleniumhq.selenium:selenium-safari-driver:3.141.59
org.seleniumhq.selenium:selenium-support:3.141.59
org.slf4j:jcl-over-slf4j:1.7.30
org.slf4j:jul-to-slf4j:1.7.30
org.slf4j:slf4j-api:1.7.30
org.springframework:spring-core:4.3.18.RELEASE
org.springframework:spring-expression:4.3.18.RELEASE
org.testcontainers:database-commons:1.15.2
org.testcontainers:jdbc:1.15.2
org.testcontainers:junit-jupiter:1.15.2
org.testcontainers:postgresql:1.15.2
org.testcontainers:selenium:1.15.2
org.testcontainers:testcontainers:1.15.2
org.threeten:threetenbp:1.5.2
org.slf4j:slf4j-api:1.7.36
org.springframework:spring-core:5.3.18
org.springframework:spring-expression:5.3.18
org.springframework:spring-jcl:5.3.18
org.testcontainers:database-commons:1.17.3
org.testcontainers:jdbc:1.17.3
org.testcontainers:junit-jupiter:1.17.3
org.testcontainers:postgresql:1.17.3
org.testcontainers:selenium:1.17.3
org.testcontainers:testcontainers:1.17.3
org.threeten:threetenbp:1.6.0
org.tukaani:xz:1.5
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3
org.webjars.npm:viz.js-graphviz-java:2.1.3
org.xerial.snappy:snappy-java:1.1.8.4
org.yaml:snakeyaml:1.28
org.yaml:snakeyaml:1.30
us.fatehi:schemacrawler-api:16.10.1
us.fatehi:schemacrawler-diagram:16.10.1
us.fatehi:schemacrawler-tools:16.10.1

View File

@@ -4,144 +4,143 @@
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.101tec:zkclient:0.10
com.beust:jcommander:1.60
com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
com.fasterxml.jackson.core:jackson-annotations:2.13.0
com.fasterxml.jackson.core:jackson-core:2.13.0
com.fasterxml.jackson.core:jackson-databind:2.13.0
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
com.fasterxml.jackson:jackson-bom:2.13.0
com.fasterxml.jackson.core:jackson-annotations:2.13.3
com.fasterxml.jackson.core:jackson-core:2.13.3
com.fasterxml.jackson.core:jackson-databind:2.13.3
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.13.3
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.13.3
com.fasterxml.jackson:jackson-bom:2.13.3
com.fasterxml:classmate:1.5.1
com.github.docker-java:docker-java-api:3.2.7
com.github.docker-java:docker-java-transport-zerodep:3.2.7
com.github.docker-java:docker-java-transport:3.2.7
com.github.jnr:jffi:1.3.1
com.github.ben-manes.caffeine:caffeine:2.9.3
com.github.docker-java:docker-java-api:3.2.13
com.github.docker-java:docker-java-transport-zerodep:3.2.13
com.github.docker-java:docker-java-transport:3.2.13
com.github.jnr:jffi:1.3.9
com.github.jnr:jnr-a64asm:1.0.0
com.github.jnr:jnr-constants:0.10.1
com.github.jnr:jnr-enxio:0.32.3
com.github.jnr:jnr-ffi:2.2.1
com.github.jnr:jnr-posix:3.1.4
com.github.jnr:jnr-unixsocket:0.38.5
com.github.jnr:jnr-constants:0.10.3
com.github.jnr:jnr-enxio:0.32.13
com.github.jnr:jnr-ffi:2.2.11
com.github.jnr:jnr-posix:3.1.15
com.github.jnr:jnr-unixsocket:0.38.17
com.github.jnr:jnr-x86asm:1.0.2
com.google.android:annotations:4.1.1.4
com.google.api-client:google-api-client-appengine:1.31.3
com.google.api-client:google-api-client-appengine:1.35.2
com.google.api-client:google-api-client-jackson2:1.32.2
com.google.api-client:google-api-client-java6:1.31.3
com.google.api-client:google-api-client-servlet:1.31.3
com.google.api-client:google-api-client:1.32.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api-client:google-api-client-java6:1.35.2
com.google.api-client:google-api-client-servlet:1.35.2
com.google.api-client:google-api-client:1.35.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:grpc-google-common-protos:2.7.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
com.google.api.grpc:proto-google-common-protos:2.7.1
com.google.api.grpc:proto-google-iam-v1:1.2.0
com.google.api:api-common:2.1.2
com.google.api:gax-grpc:2.8.1
com.google.api:gax-httpjson:0.93.1
com.google.api:gax:2.8.1
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:grpc-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:grpc-google-common-protos:2.8.3
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.12.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.136.2
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.136.2
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.6.2
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.6.2
com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.10
com.google.api.grpc:proto-google-cloud-firestore-v1:3.1.0
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.98.4
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.5.4
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.3.0
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.3.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.23.3
com.google.api.grpc:proto-google-cloud-spanner-v1:6.23.3
com.google.api.grpc:proto-google-cloud-storage-v2:2.2.2-alpha
com.google.api.grpc:proto-google-cloud-tasks-v2:2.3.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.93.0
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.93.0
com.google.api.grpc:proto-google-common-protos:2.9.0
com.google.api.grpc:proto-google-iam-v1:1.4.1
com.google.api:api-common:2.2.1
com.google.api:gax-grpc:2.18.2
com.google.api:gax-httpjson:0.103.2
com.google.api:gax:2.18.2
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
com.google.apis:google-api-services-appengine:v1-rev20220612-1.32.1
com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
com.google.apis:google-api-services-cloudkms:v1-rev20220617-1.32.1
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
com.google.apis:google-api-services-drive:v2-rev393-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
com.google.apis:google-api-services-groupssettings:v1-rev20210624-1.32.1
com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
com.google.apis:google-api-services-monitoring:v3-rev20220626-1.32.1
com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
com.google.apis:google-api-services-sheets:v4-rev20220620-1.32.1
com.google.apis:google-api-services-sqladmin:v1beta4-rev20220513-1.32.1
com.google.apis:google-api-services-storage:v1-rev20220608-1.32.1
com.google.appengine.tools:appengine-gcs-client:0.8.1
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.86
com.google.appengine:appengine-api-stubs:1.9.86
com.google.appengine:appengine-remote-api:1.9.86
com.google.appengine:appengine-api-1.0-sdk:2.0.5
com.google.appengine:appengine-api-stubs:2.0.5
com.google.appengine:appengine-remote-api:2.0.5
com.google.appengine:appengine-testing:1.9.86
com.google.auth:google-auth-library-credentials:1.3.0
com.google.auth:google-auth-library-oauth2-http:1.3.0
com.google.auto.service:auto-service-annotations:1.0-rc7
com.google.auth:google-auth-library-credentials:1.7.0
com.google.auth:google-auth-library-oauth2-http:1.7.0
com.google.auto.service:auto-service-annotations:1.0.1
com.google.auto.value:auto-value-annotations:1.9
com.google.auto.value:auto-value:1.7.4
com.google.cloud.bigdataoss:gcsio:2.2.4
com.google.cloud.bigdataoss:util:2.2.4
com.google.cloud.bigtable:bigtable-client-core:1.25.1
com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
com.google.cloud.sql:postgres-socket-factory:1.2.1
com.google.cloud:google-cloud-bigquerystorage:2.8.1
com.google.cloud:google-cloud-bigtable:1.27.1
com.google.cloud:google-cloud-core-grpc:2.3.5
com.google.cloud:google-cloud-core-http:1.95.4
com.google.cloud:google-cloud-core:2.3.5
com.google.cloud:google-cloud-firestore:3.0.10
com.google.cloud:google-cloud-nio:0.123.4
com.google.cloud:google-cloud-pubsub:1.115.1
com.google.cloud:google-cloud-pubsublite:1.4.8
com.google.cloud:google-cloud-secretmanager:1.4.0
com.google.cloud:google-cloud-spanner:6.17.4
com.google.cloud:google-cloud-storage:1.118.0
com.google.cloud:google-cloud-tasks:1.33.2
com.google.auto.value:auto-value:1.9
com.google.cloud.bigdataoss:gcsio:2.2.6
com.google.cloud.bigdataoss:util:2.2.6
com.google.cloud.bigtable:bigtable-client-core:1.26.3
com.google.cloud.bigtable:bigtable-metrics-api:1.26.3
com.google.cloud.datastore:datastore-v1-proto-client:2.2.10
com.google.cloud.sql:jdbc-socket-factory-core:1.6.1
com.google.cloud.sql:postgres-socket-factory:1.6.1
com.google.cloud:google-cloud-bigquerystorage:2.12.2
com.google.cloud:google-cloud-bigtable:2.6.2
com.google.cloud:google-cloud-core-grpc:2.6.0
com.google.cloud:google-cloud-core-http:2.8.0
com.google.cloud:google-cloud-core:2.8.0
com.google.cloud:google-cloud-firestore:3.1.0
com.google.cloud:google-cloud-monitoring:1.82.0
com.google.cloud:google-cloud-nio:0.124.7
com.google.cloud:google-cloud-pubsub:1.116.4
com.google.cloud:google-cloud-pubsublite:1.5.4
com.google.cloud:google-cloud-secretmanager:2.3.0
com.google.cloud:google-cloud-spanner:6.23.3
com.google.cloud:google-cloud-storage:2.9.0
com.google.cloud:google-cloud-tasks:2.3.0
com.google.cloud:grpc-gcp:1.1.0
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.1.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.9
com.google.code.gson:gson:2.9.0
com.google.common.html.types:types:1.0.6
com.google.dagger:dagger:2.33
com.google.errorprone:error_prone_annotations:2.10.0
com.google.dagger:dagger:2.42
com.google.errorprone:error_prone_annotations:2.14.0
com.google.escapevelocity:escapevelocity:0.9.1
com.google.flatbuffers:flatbuffers-java:1.12.0
com.google.flogger:flogger-system-backend:0.7.4
com.google.flogger:flogger:0.7.4
com.google.flogger:google-extensions:0.7.4
com.google.guava:failureaccess:1.0.1
com.google.guava:guava-testlib:30.1.1-jre
com.google.guava:guava:31.0.1-jre
com.google.guava:guava-testlib:31.1-jre
com.google.guava:guava:31.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.9.0
com.google.http-client:google-http-client-apache-v2:1.40.1
com.google.http-client:google-http-client-appengine:1.39.2
com.google.http-client:google-http-client-gson:1.41.0
com.google.http-client:google-http-client-jackson2:1.41.0
com.google.http-client:google-http-client-protobuf:1.40.1
com.google.http-client:google-http-client:1.41.0
com.google.gwt:gwt-user:2.10.0
com.google.http-client:google-http-client-apache-v2:1.42.0
com.google.http-client:google-http-client-appengine:1.42.0
com.google.http-client:google-http-client-gson:1.42.0
com.google.http-client:google-http-client-jackson2:1.42.0
com.google.http-client:google-http-client-protobuf:1.41.7
com.google.http-client:google-http-client:1.42.0
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
@@ -149,21 +148,21 @@ com.google.jsinterop:jsinterop-annotations:2.0.0
com.google.monitoring-client:contrib:1.0.7
com.google.monitoring-client:metrics:1.0.7
com.google.monitoring-client:stackdriver:1.0.7
com.google.oauth-client:google-oauth-client-appengine:1.31.4
com.google.oauth-client:google-oauth-client-java6:1.31.4
com.google.oauth-client:google-oauth-client-jetty:1.31.4
com.google.oauth-client:google-oauth-client-servlet:1.31.4
com.google.oauth-client:google-oauth-client:1.32.1
com.google.protobuf:protobuf-java-util:3.19.2
com.google.protobuf:protobuf-java:3.19.2
com.google.re2j:re2j:1.6
com.google.oauth-client:google-oauth-client-appengine:1.34.1
com.google.oauth-client:google-oauth-client-java6:1.34.1
com.google.oauth-client:google-oauth-client-jetty:1.34.1
com.google.oauth-client:google-oauth-client-servlet:1.34.1
com.google.oauth-client:google-oauth-client:1.34.1
com.google.protobuf:protobuf-java-util:3.21.1
com.google.protobuf:protobuf-java:3.21.1
com.google.re2j:re2j:1.7
com.google.template:soy:2021-02-01
com.google.truth.extensions:truth-java8-extension:1.1.2
com.google.truth:truth:1.1.2
com.googlecode.charts4j:charts4j:1.3
com.google.truth.extensions:truth-java8-extension:1.1.3
com.google.truth:truth:1.1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:68.2
com.ibm.icu:icu4j:71.1
com.jcraft:jsch:0.1.55
com.lmax:disruptor:3.4.2
com.squareup.okhttp3:okhttp:3.11.0
com.squareup.okio:okio:1.14.0
com.sun.istack:istack-commons-runtime:3.0.7
@@ -173,156 +172,167 @@ com.thoughtworks.qdox:qdox:1.12.1
com.zaxxer:HikariCP:3.4.5
commons-codec:commons-codec:1.15
commons-logging:commons-logging:1.2
dnsjava:dnsjava:3.3.1
dnsjava:dnsjava:3.5.1
guru.nidi.com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
guru.nidi.com.kitfox:svgSalamander:1.1.3
guru.nidi:graphviz-java-all-j2v8:0.17.0
guru.nidi:graphviz-java:0.17.0
guru.nidi:graphviz-java-all-j2v8:0.18.1
guru.nidi:graphviz-java:0.18.1
io.confluent:common-config:5.3.2
io.confluent:common-utils:5.3.2
io.confluent:kafka-avro-serializer:5.3.2
io.confluent:kafka-schema-registry-client:5.3.2
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.104
io.github.java-diff-utils:java-diff-utils:4.9
io.grpc:grpc-alts:1.43.2
io.grpc:grpc-api:1.43.2
io.grpc:grpc-auth:1.43.2
io.grpc:grpc-context:1.43.2
io.grpc:grpc-core:1.43.2
io.grpc:grpc-grpclb:1.43.2
io.grpc:grpc-netty-shaded:1.43.2
io.grpc:grpc-netty:1.43.2
io.grpc:grpc-protobuf-lite:1.43.2
io.grpc:grpc-protobuf:1.43.2
io.grpc:grpc-services:1.43.2
io.grpc:grpc-stub:1.43.2
io.grpc:grpc-xds:1.43.2
io.netty:netty-buffer:4.1.63.Final
io.netty:netty-codec-http2:4.1.63.Final
io.netty:netty-codec-http:4.1.63.Final
io.netty:netty-codec-socks:4.1.63.Final
io.netty:netty-codec:4.1.63.Final
io.netty:netty-common:4.1.63.Final
io.netty:netty-handler-proxy:4.1.63.Final
io.netty:netty-handler:4.1.63.Final
io.netty:netty-resolver:4.1.63.Final
io.netty:netty-tcnative-boringssl-static:2.0.33.Final
io.netty:netty-transport:4.1.63.Final
io.opencensus:opencensus-api:0.30.0
io.opencensus:opencensus-contrib-grpc-util:0.30.0
io.opencensus:opencensus-contrib-http-util:0.28.0
io.github.java-diff-utils:java-diff-utils:4.11
io.grpc:grpc-alts:1.47.0
io.grpc:grpc-api:1.47.0
io.grpc:grpc-auth:1.47.0
io.grpc:grpc-census:1.45.1
io.grpc:grpc-context:1.47.0
io.grpc:grpc-core:1.47.0
io.grpc:grpc-googleapis:1.47.0
io.grpc:grpc-grpclb:1.47.0
io.grpc:grpc-netty-shaded:1.47.0
io.grpc:grpc-netty:1.45.1
io.grpc:grpc-protobuf-lite:1.47.0
io.grpc:grpc-protobuf:1.47.0
io.grpc:grpc-services:1.47.0
io.grpc:grpc-stub:1.47.0
io.grpc:grpc-xds:1.47.0
io.netty:netty-buffer:4.1.72.Final
io.netty:netty-codec-http2:4.1.72.Final
io.netty:netty-codec-http:4.1.72.Final
io.netty:netty-codec-socks:4.1.72.Final
io.netty:netty-codec:4.1.72.Final
io.netty:netty-common:4.1.72.Final
io.netty:netty-handler-proxy:4.1.72.Final
io.netty:netty-handler:4.1.72.Final
io.netty:netty-resolver:4.1.72.Final
io.netty:netty-tcnative-boringssl-static:2.0.46.Final
io.netty:netty-tcnative-classes:2.0.46.Final
io.netty:netty-transport:4.1.72.Final
io.opencensus:opencensus-api:0.31.1
io.opencensus:opencensus-contrib-exemplar-util:0.31.0
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0
io.opencensus:opencensus-contrib-grpc-util:0.31.0
io.opencensus:opencensus-contrib-http-util:0.31.1
io.opencensus:opencensus-contrib-resource-util:0.31.0
io.opencensus:opencensus-exporter-metrics-util:0.31.0
io.opencensus:opencensus-exporter-stats-stackdriver:0.31.0
io.opencensus:opencensus-impl-core:0.31.0
io.opencensus:opencensus-impl:0.31.0
io.opencensus:opencensus-proto:0.2.0
io.perfmark:perfmark-api:0.23.0
it.unimi.dsi:fastutil:6.5.16
io.perfmark:perfmark-api:0.25.0
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.jdo:jdo2-api:2.3-20090302111651
javax.mail:mail:1.5.0-b01
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.10
junit:junit:4.13.2
net.arnx:nashorn-promise:0.1.1
net.bytebuddy:byte-buddy-agent:1.10.19
net.bytebuddy:byte-buddy:1.10.19
net.java.dev.jna:jna:5.5.0
net.bytebuddy:byte-buddy-agent:1.12.10
net.bytebuddy:byte-buddy:1.12.10
net.java.dev.jna:jna:5.8.0
org.apache.arrow:arrow-format:5.0.0
org.apache.arrow:arrow-memory-core:5.0.0
org.apache.arrow:arrow-vector:5.0.0
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-fn-execution:2.37.0
org.apache.beam:beam-model-job-management:2.37.0
org.apache.beam:beam-model-pipeline:2.37.0
org.apache.beam:beam-runners-core-construction-java:2.37.0
org.apache.beam:beam-runners-core-java:2.37.0
org.apache.beam:beam-runners-direct-java:2.37.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
org.apache.beam:beam-runners-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-core:2.37.0
org.apache.beam:beam-sdks-java-expansion-service:2.37.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
org.apache.beam:beam-sdks-java-fn-execution:2.37.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
org.apache.beam:beam-sdks-java-io-kafka:2.37.0
org.apache.beam:beam-model-fn-execution:2.40.0
org.apache.beam:beam-model-job-management:2.40.0
org.apache.beam:beam-model-pipeline:2.40.0
org.apache.beam:beam-runners-core-construction-java:2.40.0
org.apache.beam:beam-runners-core-java:2.40.0
org.apache.beam:beam-runners-direct-java:2.40.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.40.0
org.apache.beam:beam-runners-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-core:2.40.0
org.apache.beam:beam-sdks-java-expansion-service:2.40.0
org.apache.beam:beam-sdks-java-extensions-arrow:2.40.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.40.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.40.0
org.apache.beam:beam-sdks-java-fn-execution:2.40.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0
org.apache.beam:beam-sdks-java-io-kafka:2.40.0
org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
org.apache.beam:beam-vendor-grpc-1_43_2:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.20
org.apache.commons:commons-compress:1.21
org.apache.commons:commons-csv:1.9.0
org.apache.commons:commons-exec:1.3
org.apache.commons:commons-lang3:3.8.1
org.apache.commons:commons-text:1.6
org.apache.ftpserver:ftplet-api:1.0.6
org.apache.ftpserver:ftpserver-core:1.0.6
org.apache.commons:commons-lang3:3.11
org.apache.commons:commons-text:1.9
org.apache.ftpserver:ftplet-api:1.2.0
org.apache.ftpserver:ftpserver-core:1.2.0
org.apache.httpcomponents:httpclient:4.5.13
org.apache.httpcomponents:httpcore:4.4.15
org.apache.mina:mina-core:2.0.4
org.apache.mina:mina-core:2.1.6
org.apache.sshd:sshd-core:2.0.0
org.apache.sshd:sshd-scp:2.0.0
org.apache.sshd:sshd-sftp:2.0.0
org.apache.tomcat:tomcat-annotations-api:8.0.5
org.apiguardian:apiguardian-api:1.1.0
org.apache.tomcat:tomcat-annotations-api:10.1.0-M16
org.bouncycastle:bcpg-jdk15on:1.67
org.bouncycastle:bcpkix-jdk15on:1.67
org.bouncycastle:bcprov-jdk15on:1.67
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:3.21.0
org.checkerframework:checker-qual:3.22.2
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.20
org.codehaus.mojo:animal-sniffer-annotations:1.21
org.conscrypt:conscrypt-openjdk-uber:2.5.1
org.dom4j:dom4j:2.1.3
org.easymock:easymock:3.0
org.flywaydb:flyway-core:5.2.4
org.flywaydb:flyway-core:8.5.13
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.gwtproject:gwt-user:2.10.0
org.hamcrest:hamcrest-core:2.2
org.hamcrest:hamcrest-library:2.2
org.hamcrest:hamcrest:2.2
org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
org.hibernate:hibernate-core:5.4.23.Final
org.hibernate:hibernate-hikaricp:5.4.23.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.4.1.Final
org.hibernate:hibernate-core:5.6.10.Final
org.hibernate:hibernate-hikaricp:5.6.10.Final
org.jboss.logging:jboss-logging:3.4.3.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.1.3.Final
org.jboss:jandex:2.4.2.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20200518
org.jsoup:jsoup:1.13.1
org.junit-pioneer:junit-pioneer:0.7.0
org.junit.jupiter:junit-jupiter-api:5.7.0
org.junit.jupiter:junit-jupiter-engine:5.7.0
org.junit.jupiter:junit-jupiter-migrationsupport:5.7.0
org.junit.jupiter:junit-jupiter-params:5.7.0
org.junit.platform:junit-platform-commons:1.7.0
org.junit.platform:junit-platform-engine:1.7.0
org.junit.platform:junit-platform-launcher:1.7.0
org.junit.platform:junit-platform-runner:1.7.0
org.junit.platform:junit-platform-suite-api:1.7.0
org.junit:junit-bom:5.7.0
org.jsoup:jsoup:1.15.2
org.junit-pioneer:junit-pioneer:1.7.1
org.junit.jupiter:junit-jupiter-api:5.9.0-RC1
org.junit.jupiter:junit-jupiter-engine:5.9.0-RC1
org.junit.jupiter:junit-jupiter-migrationsupport:5.9.0-RC1
org.junit.jupiter:junit-jupiter-params:5.9.0-RC1
org.junit.platform:junit-platform-commons:1.9.0-RC1
org.junit.platform:junit-platform-engine:1.9.0-RC1
org.junit.platform:junit-platform-launcher:1.9.0-RC1
org.junit.platform:junit-platform-runner:1.9.0-RC1
org.junit.platform:junit-platform-suite-api:1.9.0-RC1
org.junit.platform:junit-platform-suite-commons:1.9.0-RC1
org.junit:junit-bom:5.9.0-RC1
org.jvnet.staxex:stax-ex:1.8
org.mockito:mockito-core:3.7.7
org.mockito:mockito-junit-jupiter:3.7.7
org.mockito:mockito-core:4.6.1
org.mockito:mockito-junit-jupiter:4.6.1
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:3.1
org.objenesis:objenesis:3.2
org.opentest4j:opentest4j:1.2.0
org.ow2.asm:asm-analysis:9.1
org.ow2.asm:asm-commons:9.0
org.ow2.asm:asm-tree:9.1
org.ow2.asm:asm-util:9.1
org.ow2.asm:asm:9.1
org.postgresql:postgresql:42.2.18
org.ow2.asm:asm-analysis:9.3
org.ow2.asm:asm-commons:9.2
org.ow2.asm:asm-tree:9.3
org.ow2.asm:asm-util:9.3
org.ow2.asm:asm:9.3
org.postgresql:postgresql:42.4.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.seleniumhq.selenium:selenium-api:3.141.59
org.seleniumhq.selenium:selenium-chrome-driver:3.141.59
org.seleniumhq.selenium:selenium-edge-driver:3.141.59
@@ -335,22 +345,23 @@ org.seleniumhq.selenium:selenium-safari-driver:3.141.59
org.seleniumhq.selenium:selenium-support:3.141.59
org.slf4j:jcl-over-slf4j:1.7.30
org.slf4j:jul-to-slf4j:1.7.30
org.slf4j:slf4j-api:1.7.30
org.slf4j:slf4j-jdk14:1.7.28
org.springframework:spring-core:4.3.18.RELEASE
org.springframework:spring-expression:4.3.18.RELEASE
org.testcontainers:database-commons:1.15.2
org.testcontainers:jdbc:1.15.2
org.testcontainers:junit-jupiter:1.15.2
org.testcontainers:postgresql:1.15.2
org.testcontainers:selenium:1.15.2
org.testcontainers:testcontainers:1.15.2
org.threeten:threetenbp:1.5.2
org.slf4j:slf4j-api:2.0.0-alpha7
org.slf4j:slf4j-jdk14:2.0.0-alpha7
org.springframework:spring-core:5.3.18
org.springframework:spring-expression:5.3.18
org.springframework:spring-jcl:5.3.18
org.testcontainers:database-commons:1.17.3
org.testcontainers:jdbc:1.17.3
org.testcontainers:junit-jupiter:1.17.3
org.testcontainers:postgresql:1.17.3
org.testcontainers:selenium:1.17.3
org.testcontainers:testcontainers:1.17.3
org.threeten:threetenbp:1.6.0
org.tukaani:xz:1.5
org.w3c.css:sac:1.3
org.webjars.npm:viz.js-for-graphviz-java:2.1.3
org.webjars.npm:viz.js-graphviz-java:2.1.3
org.xerial.snappy:snappy-java:1.1.8.4
org.yaml:snakeyaml:1.28
org.yaml:snakeyaml:1.30
us.fatehi:schemacrawler-api:16.10.1
us.fatehi:schemacrawler-diagram:16.10.1
us.fatehi:schemacrawler-tools:16.10.1

View File

@@ -1,118 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import static com.google.appengine.api.ThreadManager.currentRequestThreadFactory;
import static com.google.common.util.concurrent.MoreExecutors.listeningDecorator;
import static google.registry.backup.ExportCommitLogDiffAction.LOWER_CHECKPOINT_TIME_PARAM;
import static google.registry.backup.ExportCommitLogDiffAction.UPPER_CHECKPOINT_TIME_PARAM;
import static google.registry.backup.RestoreCommitLogsAction.BUCKET_OVERRIDE_PARAM;
import static google.registry.backup.RestoreCommitLogsAction.FROM_TIME_PARAM;
import static google.registry.backup.RestoreCommitLogsAction.TO_TIME_PARAM;
import static google.registry.backup.SyncDatastoreToSqlSnapshotAction.SQL_SNAPSHOT_ID_PARAM;
import static google.registry.request.RequestParameters.extractOptionalParameter;
import static google.registry.request.RequestParameters.extractRequiredDatetimeParameter;
import static google.registry.request.RequestParameters.extractRequiredParameter;
import static java.util.concurrent.Executors.newFixedThreadPool;
import com.google.common.primitives.Ints;
import com.google.common.util.concurrent.ListeningExecutorService;
import dagger.Module;
import dagger.Provides;
import google.registry.cron.CommitLogFanoutAction;
import google.registry.request.HttpException.BadRequestException;
import google.registry.request.Parameter;
import java.lang.annotation.Documented;
import java.util.Optional;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import javax.inject.Qualifier;
import javax.servlet.http.HttpServletRequest;
import org.joda.time.DateTime;
/**
* Dagger module for backup package.
*
* @see "google.registry.module.backend.BackendComponent"
*/
@Module
public final class BackupModule {
/** Dagger qualifier for backups. */
@Qualifier
@Documented
public @interface Backups {}
/** Number of threads in the threaded executor. */
private static final int NUM_THREADS = 10;
@Provides
@Parameter("bucket")
static int provideBucket(HttpServletRequest req) {
String param = extractRequiredParameter(req, CommitLogFanoutAction.BUCKET_PARAM);
Integer bucket = Ints.tryParse(param);
if (bucket == null) {
throw new BadRequestException("Bad bucket id");
}
return bucket;
}
@Provides
@Parameter(LOWER_CHECKPOINT_TIME_PARAM)
static DateTime provideLowerCheckpointKey(HttpServletRequest req) {
return extractRequiredDatetimeParameter(req, LOWER_CHECKPOINT_TIME_PARAM);
}
@Provides
@Parameter(UPPER_CHECKPOINT_TIME_PARAM)
static DateTime provideUpperCheckpointKey(HttpServletRequest req) {
return extractRequiredDatetimeParameter(req, UPPER_CHECKPOINT_TIME_PARAM);
}
@Provides
@Parameter(BUCKET_OVERRIDE_PARAM)
static Optional<String> provideBucketOverride(HttpServletRequest req) {
return extractOptionalParameter(req, BUCKET_OVERRIDE_PARAM);
}
@Provides
@Parameter(FROM_TIME_PARAM)
static DateTime provideFromTime(HttpServletRequest req) {
return extractRequiredDatetimeParameter(req, FROM_TIME_PARAM);
}
@Provides
@Parameter(TO_TIME_PARAM)
static DateTime provideToTime(HttpServletRequest req) {
return extractRequiredDatetimeParameter(req, TO_TIME_PARAM);
}
@Provides
@Parameter(SQL_SNAPSHOT_ID_PARAM)
static String provideSqlSnapshotId(HttpServletRequest req) {
return extractRequiredParameter(req, SQL_SNAPSHOT_ID_PARAM);
}
@Provides
@Backups
static ListeningExecutorService provideListeningExecutorService() {
return listeningDecorator(newFixedThreadPool(NUM_THREADS, currentRequestThreadFactory()));
}
@Provides
static ScheduledExecutorService provideScheduledExecutorService() {
return Executors.newSingleThreadScheduledExecutor();
}
}

View File

@@ -1,83 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import com.google.appengine.api.datastore.EntityTranslator;
import com.google.common.collect.AbstractIterator;
import com.google.common.collect.ImmutableList;
import com.google.storage.onestore.v3.OnestoreEntity.EntityProto;
import google.registry.model.ImmutableObject;
import google.registry.model.annotations.DeleteAfterMigration;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Iterator;
/** Utilities for working with backups. */
@DeleteAfterMigration
public class BackupUtils {
/** Keys for user metadata fields on commit log files in GCS. */
public static final class GcsMetadataKeys {
private GcsMetadataKeys() {}
public static final String NUM_TRANSACTIONS = "num_transactions";
public static final String LOWER_BOUND_CHECKPOINT = "lower_bound_checkpoint";
public static final String UPPER_BOUND_CHECKPOINT = "upper_bound_checkpoint";
}
/**
* Converts the given {@link ImmutableObject} to a raw Datastore entity and write it to an {@link
* OutputStream} in delimited protocol buffer format.
*/
static void serializeEntity(ImmutableObject entity, OutputStream stream) throws IOException {
EntityTranslator.convertToPb(auditedOfy().saveIgnoringReadOnlyWithoutBackup().toEntity(entity))
.writeDelimitedTo(stream);
}
/**
* Return an iterator of {@link ImmutableObject} instances deserialized from the given stream.
*
* <p>This parses out delimited protocol buffers for raw Datastore entities and then Ofy-loads
* those as {@link ImmutableObject}.
*
* <p>The iterator reads from the stream on demand, and as such will fail if the stream is closed.
*/
public static Iterator<ImmutableObject> createDeserializingIterator(
final InputStream input, boolean withAppIdOverride) {
return new AbstractIterator<ImmutableObject>() {
@Override
protected ImmutableObject computeNext() {
EntityProto proto = new EntityProto();
if (proto.parseDelimitedFrom(input)) { // False means end of stream; other errors throw.
if (withAppIdOverride) {
proto = EntityImports.fixEntity(proto);
}
return auditedOfy().load().fromEntity(EntityTranslator.createFromPb(proto));
}
return endOfData();
}
};
}
public static ImmutableList<ImmutableObject> deserializeEntities(byte[] bytes) {
return ImmutableList.copyOf(
createDeserializingIterator(new ByteArrayInputStream(bytes), false));
}
}

View File

@@ -1,127 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import static google.registry.backup.ExportCommitLogDiffAction.LOWER_CHECKPOINT_TIME_PARAM;
import static google.registry.backup.ExportCommitLogDiffAction.UPPER_CHECKPOINT_TIME_PARAM;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.persistence.transaction.TransactionManagerFactory.ofyTm;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import com.google.common.collect.ImmutableMultimap;
import com.google.common.flogger.FluentLogger;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.ofy.CommitLogCheckpoint;
import google.registry.model.ofy.CommitLogCheckpointRoot;
import google.registry.request.Action;
import google.registry.request.Action.Service;
import google.registry.request.auth.Auth;
import google.registry.util.CloudTasksUtils;
import java.util.Optional;
import javax.inject.Inject;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/**
* Action that saves commit log checkpoints to Datastore and kicks off a diff export task.
*
* <p>We separate computing and saving the checkpoint from exporting it because the export to GCS is
* retryable but should not require the computation of a new checkpoint. Saving the checkpoint and
* enqueuing the export task are done transactionally, so any checkpoint that is saved will be
* exported to GCS very soon.
*
* <p>This action's supported method is GET rather than POST because it gets invoked via cron.
*/
@Action(
service = Action.Service.BACKEND,
path = "/_dr/cron/commitLogCheckpoint",
method = Action.Method.GET,
automaticallyPrintOk = true,
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
@DeleteAfterMigration
public final class CommitLogCheckpointAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final String QUEUE_NAME = "export-commits";
/**
* The amount of time enqueueing should be delayed.
*
* <p>The {@link ExportCommitLogDiffAction} is enqueued in {@link CommitLogCheckpointAction},
* which is inside a Datastore transaction that persists the checkpoint to be exported. After the
* switch to CloudTasks API, the task may be invoked before the Datastore transaction commits.
* When this happens, the checkpoint is not found which leads to {@link
* com.google.common.base.VerifyException}.
*
* <p>In order to invoke the task after the transaction commits, a reasonable delay should be
* added to each task. The latency of the request is mostly in the range of 4-6 seconds; Choosing
* a value 30% greater than the upper bound should solve the issue invoking a task before the
* transaction commits.
*/
static final Duration ENQUEUE_DELAY_SECONDS = Duration.standardSeconds(8);
@Inject CommitLogCheckpointStrategy strategy;
@Inject CloudTasksUtils cloudTasksUtils;
@Inject CommitLogCheckpointAction() {}
@Override
public void run() {
createCheckPointAndStartAsyncExport();
}
/**
* Creates a {@link CommitLogCheckpoint} and initiates an asynchronous export task.
*
* @return the {@code CommitLogCheckpoint} to be exported
*/
public Optional<CommitLogCheckpoint> createCheckPointAndStartAsyncExport() {
final CommitLogCheckpoint checkpoint = strategy.computeCheckpoint();
logger.atInfo().log(
"Generated candidate checkpoint for time: %s", checkpoint.getCheckpointTime());
boolean isCheckPointPersisted =
ofyTm()
.transact(
() -> {
DateTime lastWrittenTime =
CommitLogCheckpointRoot.loadRoot().getLastWrittenTime();
if (isBeforeOrAt(checkpoint.getCheckpointTime(), lastWrittenTime)) {
logger.atInfo().log(
"Newer checkpoint already written at time: %s", lastWrittenTime);
return false;
}
auditedOfy()
.saveIgnoringReadOnlyWithoutBackup()
.entities(
checkpoint,
CommitLogCheckpointRoot.create(checkpoint.getCheckpointTime()));
// Enqueue a diff task between previous and current checkpoints.
cloudTasksUtils.enqueue(
QUEUE_NAME,
cloudTasksUtils.createPostTaskWithDelay(
ExportCommitLogDiffAction.PATH,
Service.BACKEND.toString(),
ImmutableMultimap.of(
LOWER_CHECKPOINT_TIME_PARAM,
lastWrittenTime.toString(),
UPPER_CHECKPOINT_TIME_PARAM,
checkpoint.getCheckpointTime().toString()),
ENQUEUE_DELAY_SECONDS));
return true;
});
return isCheckPointPersisted ? Optional.of(checkpoint) : Optional.empty();
}
}

View File

@@ -1,164 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import static com.google.common.collect.Iterables.getOnlyElement;
import static com.google.common.collect.Maps.transformValues;
import static google.registry.model.ofy.CommitLogBucket.getBucketKey;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static google.registry.util.DateTimeUtils.earliestOf;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableMap;
import com.googlecode.objectify.Key;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.ofy.CommitLogBucket;
import google.registry.model.ofy.CommitLogCheckpoint;
import google.registry.model.ofy.CommitLogManifest;
import google.registry.model.ofy.Ofy;
import google.registry.util.Clock;
import java.util.List;
import java.util.Map.Entry;
import javax.inject.Inject;
import org.joda.time.DateTime;
/**
* Implementation of the procedure for determining point-in-time consistent commit log checkpoint.
*
* <p>This algorithm examines the recently written commit log data and uses a dual-read approach to
* determine a point-in-time consistent set of checkpoint times for the commit log buckets. By
* "consistent" we mean, generally speaking, that if the Datastore were restored by replaying all
* the commit logs up to the checkpoint times of the buckets, the result would be transactionally
* correct; there must be no "holes" where restored state depends on non-restored state.
*
* <p>The consistency guarantee really has two parts, only one of which is provided by this
* algorithm. The procedure below guarantees only that if the resulting checkpoint includes any
* given commit log, it will also include all the commit logs that were both 1) actually written
* before that commit log "in real life", and 2) have an earlier timestamp than that commit log.
* (These criteria do not necessarily imply each other, due to the lack of a global shared clock.)
* The rest of the guarantee comes from our Ofy customizations, which ensure that any transaction
* that depends on state from a previous transaction does indeed have a later timestamp.
*
* <h2>Procedure description</h2>
*
* <pre>{@code
* ComputeCheckpoint() -> returns a set consisting of a timestamp c(b_i) for every bucket b_i
*
* 1) read off the latest commit timestamp t(b_i) for every bucket b_i
* 2) iterate over the buckets b_i a second time, and
* a) do a consistent query for the next commit timestamp t'(b_i) where t'(b_i) > t(b_i)
* b) if present, add this timestamp t'(b_i) to a set S
* 3) compute a threshold time T* representing a time before all commits in S, as follows:
* a) if S is empty, let T* = +∞ (or the "end of time")
* b) else, let T* = T - Δ, for T = min(S) and some small Δ > 0
* 4) return the set given by: min(t(b_i), T*) for all b_i
* }</pre>
*
* <h2>Correctness proof of algorithm</h2>
*
* <p>{@literal As described above, the algorithm is correct as long as it can ensure the following:
* given a commit log X written at time t(X) to bucket b_x, and another commit log Y that was
* written "in real life" before X and for which t(Y) < t(X), then if X is included in the
* checkpoint, so is Y; that is, t(X) <= c(b_x) implies t(Y) <= c(b_y). }
*
* <p>{@literal To prove this, first note that we always have c(b_i) <= t(b_i) for every b_i, i.e.
* every commit log included in the checkpoint must have been seen in the first pass. Hence if X was
* included, then X must have been written by the time we started the second pass. But since Y was
* written "in real life" prior to X, we must have seen Y by the second pass too. }
*
* <p>{@literal Now assume towards a contradiction that X is indeed included but Y is not, i.e. that
* we have t(X) <= c(b_x) but t(Y) > c(b_y). If Y was seen in the first pass, i.e. t(Y) <= t(b_y),
* then by our assumption c(b_y) < t(Y) <= t(b_y), and therefore c(b_y) != t(b_y). By the definition
* of c(b_y) it must then equal T*, so we have T* < t(Y). However, this is a contradiction since
* t(Y) < t(X) and t(X) <= c(b_x) <= T*. If instead Y was seen in the second pass but not the first,
* t'(b_y) exists and we must have t'(b_y) <= t(Y), but then since T* < T <= t'(b_y) by definition,
* we again reach the contradiction T* < t(Y). }
*/
@DeleteAfterMigration
class CommitLogCheckpointStrategy {
@Inject Ofy ofy;
@Inject Clock clock;
@Inject CommitLogCheckpointStrategy() {}
/** Compute and return a new CommitLogCheckpoint for the current point in time. */
public CommitLogCheckpoint computeCheckpoint() {
DateTime checkpointTime = clock.nowUtc();
ImmutableMap<Integer, DateTime> firstPassTimes = readBucketTimestamps();
DateTime threshold = readNewCommitLogsAndFindThreshold(firstPassTimes);
return CommitLogCheckpoint.create(
checkpointTime,
computeBucketCheckpointTimes(firstPassTimes, threshold));
}
/**
* Returns a map from all bucket IDs to their current last written time values, fetched without
* a transaction so with no guarantee of consistency across buckets.
*/
@VisibleForTesting
ImmutableMap<Integer, DateTime> readBucketTimestamps() {
// Use a fresh session cache so that we get the latest data from Datastore.
return ofy.doWithFreshSessionCache(
() ->
CommitLogBucket.loadAllBuckets()
.stream()
.collect(
ImmutableMap.toImmutableMap(
CommitLogBucket::getBucketNum, CommitLogBucket::getLastWrittenTime)));
}
/**
* Returns a threshold value defined as the latest timestamp that is before all new commit logs,
* where "new" means having a commit time after the per-bucket timestamp in the given map.
* When no such commit logs exist, the threshold value is set to END_OF_TIME.
*/
@VisibleForTesting
DateTime readNewCommitLogsAndFindThreshold(ImmutableMap<Integer, DateTime> bucketTimes) {
DateTime timeBeforeAllNewCommits = END_OF_TIME;
for (Entry<Integer, DateTime> entry : bucketTimes.entrySet()) {
Key<CommitLogBucket> bucketKey = getBucketKey(entry.getKey());
DateTime bucketTime = entry.getValue();
// Add 1 to handle START_OF_TIME since 0 isn't a valid id - filter then uses >= instead of >.
Key<CommitLogManifest> keyForFilter =
Key.create(CommitLogManifest.create(bucketKey, bucketTime.plusMillis(1), null));
List<Key<CommitLogManifest>> manifestKeys =
ofy.load()
.type(CommitLogManifest.class)
.ancestor(bucketKey)
.filterKey(">=", keyForFilter)
.limit(1)
.keys()
.list();
if (!manifestKeys.isEmpty()) {
timeBeforeAllNewCommits = earliestOf(
timeBeforeAllNewCommits,
CommitLogManifest.extractCommitTime(getOnlyElement(manifestKeys)).minusMillis(1));
}
}
return timeBeforeAllNewCommits;
}
/**
* Returns the bucket checkpoint times produced by clamping the given set of bucket timestamps to
* at most the given threshold value.
*/
@VisibleForTesting
ImmutableMap<Integer, DateTime> computeBucketCheckpointTimes(
ImmutableMap<Integer, DateTime> firstPassTimes,
final DateTime threshold) {
return ImmutableMap.copyOf(
transformValues(firstPassTimes, firstPassTime -> earliestOf(firstPassTime, threshold)));
}
}

View File

@@ -1,140 +0,0 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.backup.BackupUtils.createDeserializingIterator;
import com.google.common.collect.ImmutableList;
import google.registry.model.ImmutableObject;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.ofy.CommitLogCheckpoint;
import google.registry.model.ofy.CommitLogManifest;
import google.registry.model.ofy.CommitLogMutation;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.channels.Channels;
import java.nio.channels.ReadableByteChannel;
import java.util.Iterator;
/**
* Helpers for reading CommitLog records from a file.
*
* <p>This class is adapted from {@link RestoreCommitLogsAction}, and will be used in the initial
* population of the Cloud SQL database.
*/
@DeleteAfterMigration
public final class CommitLogImports {
private CommitLogImports() {}
/**
* Returns entities in an {@code inputStream} (from a single CommitLog file) as an {@link
* ImmutableList} of {@link ImmutableList}s of {@link VersionedEntity} records where the inner
* lists each consist of one transaction. Upon completion the {@code inputStream} is closed.
*
* <p>The returned list may be empty, since CommitLogs are written at fixed intervals regardless
* if actual changes exist. Each sublist, however, will not be empty.
*
* <p>A CommitLog file starts with a {@link CommitLogCheckpoint}, followed by (repeated)
* subsequences of [{@link CommitLogManifest}, [{@link CommitLogMutation}] ...]. Each subsequence
* represents the changes in one transaction. The {@code CommitLogManifest} contains deleted
* entity keys, whereas each {@code CommitLogMutation} contains one whole entity.
*/
static ImmutableList<ImmutableList<VersionedEntity>> loadEntitiesByTransaction(
InputStream inputStream) {
try (InputStream input = new BufferedInputStream(inputStream)) {
Iterator<ImmutableObject> commitLogs = createDeserializingIterator(input, false);
checkState(commitLogs.hasNext());
checkState(commitLogs.next() instanceof CommitLogCheckpoint);
ImmutableList.Builder<ImmutableList<VersionedEntity>> resultBuilder =
new ImmutableList.Builder<>();
ImmutableList.Builder<VersionedEntity> currentTransactionBuilder =
new ImmutableList.Builder<>();
while (commitLogs.hasNext()) {
ImmutableObject currentObject = commitLogs.next();
if (currentObject instanceof CommitLogManifest) {
// CommitLogManifest means we are starting a new transaction
addIfNonempty(resultBuilder, currentTransactionBuilder);
currentTransactionBuilder = new ImmutableList.Builder<>();
VersionedEntity.fromManifest((CommitLogManifest) currentObject)
.forEach(currentTransactionBuilder::add);
} else if (currentObject instanceof CommitLogMutation) {
currentTransactionBuilder.add(
VersionedEntity.fromMutation((CommitLogMutation) currentObject));
} else {
throw new IllegalStateException(
String.format("Unknown entity type %s in commit logs", currentObject.getClass()));
}
}
// Add the last transaction in (if it's not empty)
addIfNonempty(resultBuilder, currentTransactionBuilder);
return resultBuilder.build();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
/**
* Returns entities in an {@code inputStream} (from a single CommitLog file) as an {@link
* ImmutableList} of {@link VersionedEntity} records. Upon completion the {@code inputStream} is
* closed.
*
* <p>The returned list may be empty, since CommitLogs are written at fixed intervals regardless
* if actual changes exist.
*
* <p>A CommitLog file starts with a {@link CommitLogCheckpoint}, followed by (repeated)
* subsequences of [{@link CommitLogManifest}, [{@link CommitLogMutation}] ...]. Each subsequence
* represents the changes in one transaction. The {@code CommitLogManifest} contains deleted
* entity keys, whereas each {@code CommitLogMutation} contains one whole entity.
*/
static ImmutableList<VersionedEntity> loadEntities(InputStream inputStream) {
return loadEntitiesByTransaction(inputStream).stream()
.flatMap(ImmutableList::stream)
.collect(toImmutableList());
}
/** Covenience method that adapts {@link #loadEntities(InputStream)} to a {@link File}. */
public static ImmutableList<VersionedEntity> loadEntities(File commitLogFile) {
try {
return loadEntities(new FileInputStream(commitLogFile));
} catch (IOException e) {
throw new RuntimeException(e);
}
}
/**
* Covenience method that adapts {@link #loadEntities(InputStream)} to a {@link
* ReadableByteChannel}.
*/
public static ImmutableList<VersionedEntity> loadEntities(ReadableByteChannel channel) {
return loadEntities(Channels.newInputStream(channel));
}
private static void addIfNonempty(
ImmutableList.Builder<ImmutableList<VersionedEntity>> resultBuilder,
ImmutableList.Builder<VersionedEntity> currentTransactionBuilder) {
ImmutableList<VersionedEntity> currentTransaction = currentTransactionBuilder.build();
if (!currentTransaction.isEmpty()) {
resultBuilder.add(currentTransaction);
}
}
}

View File

@@ -1,346 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.mapreduce.MapreduceRunner.PARAM_DRY_RUN;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.persistence.transaction.TransactionManagerFactory.ofyTm;
import static java.lang.Boolean.FALSE;
import static java.lang.Boolean.TRUE;
import com.google.appengine.tools.mapreduce.Mapper;
import com.google.appengine.tools.mapreduce.Reducer;
import com.google.appengine.tools.mapreduce.ReducerInput;
import com.google.auto.value.AutoValue;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMultiset;
import com.google.common.flogger.FluentLogger;
import com.googlecode.objectify.Key;
import google.registry.config.RegistryConfig.Config;
import google.registry.mapreduce.MapreduceRunner;
import google.registry.mapreduce.inputs.CommitLogManifestInput;
import google.registry.mapreduce.inputs.EppResourceInputs;
import google.registry.model.EppResource;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.ofy.CommitLogManifest;
import google.registry.model.ofy.CommitLogMutation;
import google.registry.model.translators.CommitLogRevisionsTranslatorFactory;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import javax.inject.Inject;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/**
* Task that garbage collects old {@link CommitLogManifest} entities.
*
* <p>Once commit logs have been written to GCS, we don't really need them in Datastore anymore,
* except to reconstruct point-in-time snapshots of the database. To make that possible, {@link
* EppResource}s have a {@link EppResource#getRevisions} method that returns the commit logs for
* older points in time. But that functionality is not useful after a certain amount of time, e.g.
* thirty days, so unneeded revisions are deleted (see {@link CommitLogRevisionsTranslatorFactory}).
* This leaves commit logs in the system that are unneeded (have no revisions pointing to them). So
* this task runs periodically to delete the "orphan" commit logs.
*
* <p>This action runs a mapreduce that goes over all existing {@link EppResource} and all {@link
* CommitLogManifest} older than commitLogDatastreRetention, and erases the commit logs aren't in an
* EppResource.
*/
@Action(
service = Action.Service.BACKEND,
path = "/_dr/task/deleteOldCommitLogs",
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
// No longer needed in SQL. Subject to future removal.
@Deprecated
@DeleteAfterMigration
public final class DeleteOldCommitLogsAction implements Runnable {
private static final int NUM_MAP_SHARDS = 20;
private static final int NUM_REDUCE_SHARDS = 10;
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
@Inject MapreduceRunner mrRunner;
@Inject Response response;
@Inject Clock clock;
@Inject
@Config("commitLogDatastoreRetention")
Duration maxAge;
@Inject
@Parameter(PARAM_DRY_RUN)
boolean isDryRun;
@Inject
DeleteOldCommitLogsAction() {}
@Override
public void run() {
DateTime deletionThreshold = clock.nowUtc().minus(maxAge);
logger.atInfo().log(
"Processing asynchronous deletion of unreferenced CommitLogManifests older than %s.",
deletionThreshold);
mrRunner
.setJobName("Delete old commit logs")
.setModuleName("backend")
.setDefaultMapShards(NUM_MAP_SHARDS)
.setDefaultReduceShards(NUM_REDUCE_SHARDS)
.runMapreduce(
new DeleteOldCommitLogsMapper(deletionThreshold),
new DeleteOldCommitLogsReducer(deletionThreshold, isDryRun),
ImmutableList.of(
new CommitLogManifestInput(deletionThreshold),
EppResourceInputs.createKeyInput(EppResource.class)))
.sendLinkToMapreduceConsole(response);
}
/**
* A mapper that iterates over all {@link EppResource} and {CommitLogManifest} entities.
*
* <p>It emits the target key and {@code false} for all revisions of each EppResources (meaning
* "don't delete this"), and {@code true} for all CommitLogRevisions (meaning "delete this").
*
* <p>The reducer will then delete all CommitLogRevisions that only have {@code true}.
*/
private static class DeleteOldCommitLogsMapper
extends Mapper<Key<?>, Key<CommitLogManifest>, Boolean> {
private static final long serialVersionUID = 8008689353479902948L;
private static final String KIND_MANIFEST = Key.getKind(CommitLogManifest.class);
private final DateTime threshold;
DeleteOldCommitLogsMapper(DateTime threshold) {
this.threshold = threshold;
}
@Override
public void map(final Key<?> key) {
// key is either a Key<CommitLogManifest> or a Key<? extends EppResource>.
//
// If it's a CommitLogManifest we just emit it as is (no need to load it).
if (key.getKind().equals(KIND_MANIFEST)) {
getContext().incrementCounter("old commit log manifests found");
// safe because we checked getKind
@SuppressWarnings("unchecked")
Key<CommitLogManifest> manifestKey = (Key<CommitLogManifest>) key;
emit(manifestKey, true);
return;
}
// If it isn't a Key<CommitLogManifest> then it should be an EppResource, which we need to
// load to emit the revisions.
//
Object object = auditedOfy().load().key(key).now();
checkNotNull(object, "Received a key to a missing object. key: %s", key);
checkState(
object instanceof EppResource,
"Received a key to an object that isn't EppResource nor CommitLogManifest."
+ " Key: %s object type: %s",
key,
object.getClass().getName());
getContext().incrementCounter("EPP resources found");
EppResource eppResource = (EppResource) object;
if (eppResource.getCreationTime().isAfter(threshold)) {
getContext().incrementCounter("EPP resources newer than threshold");
}
for (Key<CommitLogManifest> manifestKey : eppResource.getRevisions().values()) {
emit(manifestKey, false);
}
getContext()
.incrementCounter("EPP resource revisions found", eppResource.getRevisions().size());
checkAndLogRevisionCoverageError(eppResource);
}
/**
* Check if given eppResource has the required revisions.
*
* <p>Revisions are used to recreate the state of the resource at a given day in the past
* "commitLogDatastoreRenention". To do that, we need at least one revision that's older than
* this duration (is dated before "threshold"), or at least one revision within a day of the
* resource's creation if it was created after the threshold.
*
* <p>Here we check that the given eppResource has the revisions it needs.
*
* <p>It's just a sanity check - since we're relying on the revisions to be correct for the
* deletion to work. We want to alert any problems we find in the revisions.
*
* <p>This really checks {@link CommitLogRevisionsTranslatorFactory#transformBeforeSave}.
* There's nothing we can do at this point to prevent the damage - we only report on it.
*/
private void checkAndLogRevisionCoverageError(EppResource eppResource) {
// First - check if there even are revisions
if (eppResource.getRevisions().isEmpty()) {
getContext().incrementCounter("EPP resources missing all revisions (SEE LOGS)");
logger.atSevere().log("EPP resource missing all revisions: %s", Key.create(eppResource));
return;
}
// Next, check if there's a revision that's older than "CommitLogDatastoreRetention". There
// should have been at least one at the time this resource was saved.
//
// Alternatively, if the resource is newer than the threshold - there should be at least one
// revision within a day of the creation time.
DateTime oldestRevisionDate = eppResource.getRevisions().firstKey();
if (oldestRevisionDate.isBefore(threshold)
|| oldestRevisionDate.isBefore(eppResource.getCreationTime().plusDays(1))) {
// We're OK!
return;
}
// The oldest revision date is newer than the threshold! This shouldn't happen.
getContext().incrementCounter("EPP resources missing pre-threshold revision (SEE LOGS)");
logger.atSevere().log(
"EPP resource missing old enough revision: "
+ "%s (created on %s) has %d revisions between %s and %s, while threshold is %s.",
Key.create(eppResource),
eppResource.getCreationTime(),
eppResource.getRevisions().size(),
eppResource.getRevisions().firstKey(),
eppResource.getRevisions().lastKey(),
threshold);
// We want to see how bad it is though: if the difference is less than a day then this might
// still be OK (we only need logs for the end of the day). But if it's more than a day, then
// we are 100% sure we can't recreate all the history we need from the revisions.
Duration interval = new Duration(threshold, oldestRevisionDate);
if (interval.isLongerThan(Duration.standardDays(1))) {
getContext()
.incrementCounter("EPP resources missing pre-(threshold+1d) revision (SEE LOGS)");
}
}
}
/**
* Reducer that deletes unreferenced {@link CommitLogManifest} + child {@link CommitLogMutation}.
*
* <p>It receives the manifestKey to possibly delete, and a list of boolean 'verdicts' from
* various sources (the "old manifests" source and the "still referenced" source) on whether it's
* OK to delete this manifestKey. If even one source returns "false" (meaning "it's not OK to
* delete this manifest") then it won't be deleted.
*/
static class DeleteOldCommitLogsReducer extends Reducer<Key<CommitLogManifest>, Boolean, Void> {
private static final long serialVersionUID = -4918760187627937268L;
private final DateTime deletionThreshold;
private final boolean isDryRun;
@AutoValue
abstract static class DeletionResult {
enum Status {
ALREADY_DELETED,
AFTER_THRESHOLD,
SUCCESS
}
public abstract Status status();
public abstract int numDeleted();
static DeletionResult create(Status status, int numDeleted) {
return new AutoValue_DeleteOldCommitLogsAction_DeleteOldCommitLogsReducer_DeletionResult(
status, numDeleted);
}
}
DeleteOldCommitLogsReducer(DateTime deletionThreshold, boolean isDryRun) {
this.deletionThreshold = deletionThreshold;
this.isDryRun = isDryRun;
}
@Override
public void reduce(
final Key<CommitLogManifest> manifestKey, ReducerInput<Boolean> canDeleteVerdicts) {
ImmutableMultiset<Boolean> canDeleteMultiset = ImmutableMultiset.copyOf(canDeleteVerdicts);
if (canDeleteMultiset.count(TRUE) > 1) {
getContext().incrementCounter("commit log manifests incorrectly mapped multiple times");
}
if (canDeleteMultiset.count(FALSE) > 1) {
getContext().incrementCounter("commit log manifests referenced multiple times");
}
if (canDeleteMultiset.contains(FALSE)) {
getContext()
.incrementCounter(
canDeleteMultiset.contains(TRUE)
? "old commit log manifests still referenced"
: "new (or nonexistent) commit log manifests referenced");
getContext()
.incrementCounter("EPP resource revisions handled", canDeleteMultiset.count(FALSE));
return;
}
DeletionResult deletionResult =
ofyTm()
.transactNew(
() -> {
CommitLogManifest manifest = auditedOfy().load().key(manifestKey).now();
// It is possible that the same manifestKey was run twice, if a shard had to be
// restarted or some weird failure. If this happens, we want to exit
// immediately. Note that this can never happen in dryRun.
if (manifest == null) {
return DeletionResult.create(DeletionResult.Status.ALREADY_DELETED, 0);
}
// Doing a sanity check on the date. This is the only place we use the
// CommitLogManifest, so maybe removing this test will improve performance.
// However, unless it's proven that the performance boost is significant (and
// we've tested this enough to be sure it never happens)- the safety of "let's
// not delete stuff we need from prod" is more important.
if (manifest.getCommitTime().isAfter(deletionThreshold)) {
return DeletionResult.create(DeletionResult.Status.AFTER_THRESHOLD, 0);
}
Iterable<Key<CommitLogMutation>> commitLogMutationKeys =
auditedOfy()
.load()
.type(CommitLogMutation.class)
.ancestor(manifestKey)
.keys()
.iterable();
ImmutableList<Key<?>> keysToDelete =
ImmutableList.<Key<?>>builder()
.addAll(commitLogMutationKeys)
.add(manifestKey)
.build();
// Normally in a dry run we would log the entities that would be deleted, but
// those can number in the millions so we skip the logging.
if (!isDryRun) {
auditedOfy().deleteWithoutBackup().keys(keysToDelete);
}
return DeletionResult.create(
DeletionResult.Status.SUCCESS, keysToDelete.size());
});
switch (deletionResult.status()) {
case SUCCESS:
getContext().incrementCounter("old commit log manifests deleted");
getContext().incrementCounter("total entities deleted", deletionResult.numDeleted());
break;
case ALREADY_DELETED:
getContext().incrementCounter("attempts to delete an already deleted manifest");
break;
case AFTER_THRESHOLD:
logger.atSevere().log(
"Won't delete CommitLogManifest %s that is too recent.", manifestKey);
getContext().incrementCounter("manifests incorrectly assigned for deletion (SEE LOGS)");
break;
}
}
}
}

View File

@@ -1,117 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import com.google.apphosting.api.ApiProxy;
import com.google.storage.onestore.v3.OnestoreEntity;
import com.google.storage.onestore.v3.OnestoreEntity.EntityProto;
import com.google.storage.onestore.v3.OnestoreEntity.Path;
import com.google.storage.onestore.v3.OnestoreEntity.Property.Meaning;
import com.google.storage.onestore.v3.OnestoreEntity.PropertyValue.ReferenceValue;
import google.registry.model.annotations.DeleteAfterMigration;
import java.nio.charset.StandardCharsets;
import java.util.Objects;
/** Utilities for handling imported Datastore entities. */
@DeleteAfterMigration
public class EntityImports {
/**
* Transitively sets the {@code appId} of all keys in a foreign entity to that of the current
* system.
*/
public static EntityProto fixEntity(EntityProto entityProto) {
String currentAappId = ApiProxy.getCurrentEnvironment().getAppId();
if (Objects.equals(currentAappId, entityProto.getKey().getApp())) {
return entityProto;
}
return fixEntity(entityProto, currentAappId);
}
private static EntityProto fixEntity(EntityProto entityProto, String appId) {
if (entityProto.hasKey()) {
fixKey(entityProto, appId);
}
for (OnestoreEntity.Property property : entityProto.mutablePropertys()) {
fixProperty(property, appId);
}
for (OnestoreEntity.Property property : entityProto.mutableRawPropertys()) {
fixProperty(property, appId);
}
// CommitLogMutation embeds an entity as bytes, which needs additional fixes.
if (isCommitLogMutation(entityProto)) {
fixMutationEntityProtoBytes(entityProto, appId);
}
return entityProto;
}
private static boolean isCommitLogMutation(EntityProto entityProto) {
if (!entityProto.hasKey()) {
return false;
}
Path path = entityProto.getKey().getPath();
if (path.elementSize() == 0) {
return false;
}
return Objects.equals(
path.getElement(path.elementSize() - 1).getType(StandardCharsets.UTF_8),
"CommitLogMutation");
}
private static void fixMutationEntityProtoBytes(EntityProto entityProto, String appId) {
for (OnestoreEntity.Property property : entityProto.mutableRawPropertys()) {
if (Objects.equals(property.getName(), "entityProtoBytes")) {
OnestoreEntity.PropertyValue value = property.getValue();
EntityProto fixedProto =
fixEntity(bytesToEntityProto(value.getStringValueAsBytes()), appId);
value.setStringValueAsBytes(fixedProto.toByteArray());
return;
}
}
}
private static void fixKey(EntityProto entityProto, String appId) {
entityProto.getMutableKey().setApp(appId);
}
private static void fixKey(ReferenceValue referenceValue, String appId) {
referenceValue.setApp(appId);
}
private static void fixProperty(OnestoreEntity.Property property, String appId) {
OnestoreEntity.PropertyValue value = property.getMutableValue();
if (value.hasReferenceValue()) {
fixKey(value.getMutableReferenceValue(), appId);
return;
}
if (property.getMeaningEnum().equals(Meaning.ENTITY_PROTO)) {
EntityProto embeddedProto = bytesToEntityProto(value.getStringValueAsBytes());
fixEntity(embeddedProto, appId);
value.setStringValueAsBytes(embeddedProto.toByteArray());
}
}
private static EntityProto bytesToEntityProto(byte[] bytes) {
EntityProto entityProto = new EntityProto();
boolean isParsed = entityProto.parseFrom(bytes);
if (!isParsed) {
throw new IllegalStateException("Failed to parse raw bytes as EntityProto.");
}
return entityProto;
}
}

View File

@@ -1,223 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import static com.google.common.base.MoreObjects.firstNonNull;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Verify.verifyNotNull;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.Iterables.concat;
import static com.google.common.collect.Lists.partition;
import static google.registry.backup.BackupUtils.GcsMetadataKeys.LOWER_BOUND_CHECKPOINT;
import static google.registry.backup.BackupUtils.GcsMetadataKeys.NUM_TRANSACTIONS;
import static google.registry.backup.BackupUtils.GcsMetadataKeys.UPPER_BOUND_CHECKPOINT;
import static google.registry.backup.BackupUtils.serializeEntity;
import static google.registry.model.ofy.CommitLogBucket.getBucketKey;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static google.registry.util.DateTimeUtils.isAtOrAfter;
import static java.util.Comparator.comparingLong;
import com.google.cloud.storage.BlobId;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import com.googlecode.objectify.Key;
import google.registry.config.RegistryConfig.Config;
import google.registry.gcs.GcsUtils;
import google.registry.model.ImmutableObject;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.ofy.CommitLogBucket;
import google.registry.model.ofy.CommitLogCheckpoint;
import google.registry.model.ofy.CommitLogManifest;
import google.registry.model.ofy.CommitLogMutation;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
import java.io.IOException;
import java.io.OutputStream;
import java.util.Collection;
import java.util.List;
import java.util.Map;
import javax.annotation.Nullable;
import javax.inject.Inject;
import org.joda.time.DateTime;
/** Action that exports the diff between two commit log checkpoints to GCS. */
@Action(
service = Action.Service.BACKEND,
path = ExportCommitLogDiffAction.PATH,
method = Action.Method.POST,
automaticallyPrintOk = true,
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
@DeleteAfterMigration
public final class ExportCommitLogDiffAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
static final String PATH = "/_dr/task/exportCommitLogDiff";
static final String UPPER_CHECKPOINT_TIME_PARAM = "upperCheckpointTime";
static final String LOWER_CHECKPOINT_TIME_PARAM = "lowerCheckpointTime";
public static final String DIFF_FILE_PREFIX = "commit_diff_until_";
@Inject GcsUtils gcsUtils;
@Inject @Config("commitLogGcsBucket") String gcsBucket;
@Inject @Config("commitLogDiffExportBatchSize") int batchSize;
@Inject @Parameter(LOWER_CHECKPOINT_TIME_PARAM) DateTime lowerCheckpointTime;
@Inject @Parameter(UPPER_CHECKPOINT_TIME_PARAM) DateTime upperCheckpointTime;
@Inject ExportCommitLogDiffAction() {}
@Override
public void run() {
logger.atInfo().log(
"Exporting commit log diffs between %s and %s.", lowerCheckpointTime, upperCheckpointTime);
checkArgument(isAtOrAfter(lowerCheckpointTime, START_OF_TIME));
checkArgument(lowerCheckpointTime.isBefore(upperCheckpointTime));
// Load the boundary checkpoints - lower is exclusive and may not exist (on the first export,
// when lowerCheckpointTime is START_OF_TIME), whereas the upper is inclusive and must exist.
CommitLogCheckpoint lowerCheckpoint =
lowerCheckpointTime.isAfter(START_OF_TIME)
? verifyNotNull(
auditedOfy().load().key(CommitLogCheckpoint.createKey(lowerCheckpointTime)).now())
: null;
CommitLogCheckpoint upperCheckpoint =
verifyNotNull(
auditedOfy().load().key(CommitLogCheckpoint.createKey(upperCheckpointTime)).now());
// Load the keys of all the manifests to include in this diff.
List<Key<CommitLogManifest>> sortedKeys = loadAllDiffKeys(lowerCheckpoint, upperCheckpoint);
logger.atInfo().log("Found %d manifests to export.", sortedKeys.size());
// Open an output channel to GCS, wrapped in a stream for convenience.
try (OutputStream gcsStream =
gcsUtils.openOutputStream(
BlobId.of(gcsBucket, DIFF_FILE_PREFIX + upperCheckpointTime),
ImmutableMap.of(
LOWER_BOUND_CHECKPOINT, lowerCheckpointTime.toString(),
UPPER_BOUND_CHECKPOINT, upperCheckpointTime.toString(),
NUM_TRANSACTIONS, Integer.toString(sortedKeys.size())))) {
// Export the upper checkpoint itself.
serializeEntity(upperCheckpoint, gcsStream);
// If there are no manifests to export, stop early, now that we've written out the file with
// the checkpoint itself (which is needed for restores, even if it's empty).
if (sortedKeys.isEmpty()) {
return;
}
// Export to GCS in chunks, one per fixed batch of commit logs. While processing one batch,
// asynchronously load the entities for the next one.
List<List<Key<CommitLogManifest>>> keyChunks = partition(sortedKeys, batchSize);
// Objectify's map return type is asynchronous. Calling .values() will block until it loads.
Map<?, CommitLogManifest> nextChunkToExport = auditedOfy().load().keys(keyChunks.get(0));
for (int i = 0; i < keyChunks.size(); i++) {
// Force the async load to finish.
Collection<CommitLogManifest> chunkValues = nextChunkToExport.values();
logger.atInfo().log("Loaded %d manifests.", chunkValues.size());
// Since there is no hard bound on how much data this might be, take care not to let the
// Objectify session cache fill up and potentially run out of memory. This is the only safe
// point to do this since at this point there is no async load in progress.
auditedOfy().clearSessionCache();
// Kick off the next async load, which can happen in parallel to the current GCS export.
if (i + 1 < keyChunks.size()) {
nextChunkToExport = auditedOfy().load().keys(keyChunks.get(i + 1));
}
exportChunk(gcsStream, chunkValues);
logger.atInfo().log("Exported %d manifests.", chunkValues.size());
}
} catch (IOException e) {
throw new RuntimeException(e);
}
logger.atInfo().log("Exported %d total manifests.", sortedKeys.size());
}
/**
* Loads all the diff keys, sorted in a transaction-consistent chronological order.
*
* @param lowerCheckpoint exclusive lower bound on keys in this diff, or null if no lower bound
* @param upperCheckpoint inclusive upper bound on keys in this diff
*/
private ImmutableList<Key<CommitLogManifest>> loadAllDiffKeys(
@Nullable final CommitLogCheckpoint lowerCheckpoint,
final CommitLogCheckpoint upperCheckpoint) {
// Fetch the keys (no data) between these checkpoints, and sort by timestamp. This ordering is
// transaction-consistent by virtue of our checkpoint strategy and our customized Ofy; see
// CommitLogCheckpointStrategy for the proof. We break ties by sorting on bucket ID to ensure
// a deterministic order.
return upperCheckpoint
.getBucketTimestamps()
.keySet()
.stream()
.flatMap(
bucketNum ->
Streams.stream(loadDiffKeysFromBucket(lowerCheckpoint, upperCheckpoint, bucketNum)))
.sorted(
comparingLong(Key<CommitLogManifest>::getId)
.thenComparingLong(a -> a.getParent().getId()))
.collect(toImmutableList());
}
/**
* Loads the diff keys for one bucket.
*
* @param lowerCheckpoint exclusive lower bound on keys in this diff, or null if no lower bound
* @param upperCheckpoint inclusive upper bound on keys in this diff
* @param bucketNum the bucket to load diff keys from
*/
private Iterable<Key<CommitLogManifest>> loadDiffKeysFromBucket(
@Nullable CommitLogCheckpoint lowerCheckpoint,
CommitLogCheckpoint upperCheckpoint,
int bucketNum) {
// If no lower checkpoint exists, or if it exists but had no timestamp for this bucket number
// (because the bucket count was increased between these checkpoints), then use START_OF_TIME
// as the effective exclusive lower bound.
DateTime lowerCheckpointBucketTime =
firstNonNull(
(lowerCheckpoint == null) ? null : lowerCheckpoint.getBucketTimestamps().get(bucketNum),
START_OF_TIME);
// Since START_OF_TIME=0 is not a valid id in a key, add 1 to both bounds. Then instead of
// loading lowerBound < x <= upperBound, we can load lowerBound <= x < upperBound.
DateTime lowerBound = lowerCheckpointBucketTime.plusMillis(1);
DateTime upperBound = upperCheckpoint.getBucketTimestamps().get(bucketNum).plusMillis(1);
// If the lower and upper bounds are equal, there can't be any results, so skip the query.
if (lowerBound.equals(upperBound)) {
return ImmutableSet.of();
}
Key<CommitLogBucket> bucketKey = getBucketKey(bucketNum);
return auditedOfy()
.load()
.type(CommitLogManifest.class)
.ancestor(bucketKey)
.filterKey(">=", CommitLogManifest.createKey(bucketKey, lowerBound))
.filterKey("<", CommitLogManifest.createKey(bucketKey, upperBound))
.keys();
}
/** Writes a chunks-worth of manifests and associated mutations to GCS. */
private void exportChunk(OutputStream gcsStream, Collection<CommitLogManifest> chunk)
throws IOException {
// Kickoff async loads for all the manifests in the chunk.
ImmutableList.Builder<Iterable<? extends ImmutableObject>> entities =
new ImmutableList.Builder<>();
for (CommitLogManifest manifest : chunk) {
entities.add(ImmutableList.of(manifest));
entities.add(auditedOfy().load().type(CommitLogMutation.class).ancestor(manifest));
}
for (ImmutableObject entity : concat(entities.build())) {
serializeEntity(entity, gcsStream);
}
}
}

View File

@@ -1,246 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.backup.BackupUtils.GcsMetadataKeys.LOWER_BOUND_CHECKPOINT;
import static google.registry.backup.ExportCommitLogDiffAction.DIFF_FILE_PREFIX;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import static google.registry.util.DateTimeUtils.latestOf;
import com.google.cloud.storage.BlobId;
import com.google.cloud.storage.BlobInfo;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableList;
import com.google.common.flogger.FluentLogger;
import com.google.common.util.concurrent.Futures;
import com.google.common.util.concurrent.ListenableFuture;
import com.google.common.util.concurrent.ListeningExecutorService;
import com.google.common.util.concurrent.UncheckedExecutionException;
import google.registry.backup.BackupModule.Backups;
import google.registry.gcs.GcsUtils;
import google.registry.model.annotations.DeleteAfterMigration;
import java.io.IOException;
import java.time.Duration;
import java.util.Map;
import java.util.TreeMap;
import java.util.concurrent.ScheduledExecutorService;
import javax.annotation.Nullable;
import javax.inject.Inject;
import javax.inject.Provider;
import org.joda.time.DateTime;
/** Utility class to list commit logs diff files stored on GCS. */
@DeleteAfterMigration
class GcsDiffFileLister {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
/** Timeout for retrieving per-file information from GCS. */
private static final Duration FILE_INFO_TIMEOUT_DURATION = Duration.ofMinutes(1);
@Inject GcsUtils gcsUtils;
@Inject @Backups Provider<ListeningExecutorService> executorProvider;
@Inject ScheduledExecutorService scheduledExecutorService;
@Inject
GcsDiffFileLister() {}
/**
* Traverses the sequence of diff files backwards from checkpointTime and inserts the file
* metadata into "sequence". Returns true if a complete sequence was discovered, false if one or
* more files are missing.
*
* @throws UncheckedExecutionException wrapping a {@link java.util.concurrent.TimeoutException} if
* the GCS call fails to finish within one minute, or wrapping any other exception if
* something else goes wrong.
*/
private boolean constructDiffSequence(
String gcsBucket,
Map<DateTime, ListenableFuture<BlobInfo>> upperBoundTimesToBlobInfo,
DateTime fromTime,
DateTime lastTime,
TreeMap<DateTime, BlobInfo> sequence) {
DateTime checkpointTime = lastTime;
while (isBeforeOrAt(fromTime, checkpointTime)) {
BlobInfo blobInfo;
if (upperBoundTimesToBlobInfo.containsKey(checkpointTime)) {
blobInfo =
Futures.getUnchecked(
Futures.withTimeout(
upperBoundTimesToBlobInfo.get(checkpointTime),
FILE_INFO_TIMEOUT_DURATION,
scheduledExecutorService));
} else {
String filename = DIFF_FILE_PREFIX + checkpointTime;
logger.atInfo().log("Patching GCS list; discovered file: %s", filename);
blobInfo = getBlobInfo(gcsBucket, filename);
// If we hit a gap, quit.
if (blobInfo == null) {
logger.atWarning().log(
"Gap discovered in sequence terminating at %s, missing file: %s",
sequence.lastKey(), filename);
logger.atInfo().log("Found sequence from %s to %s.", checkpointTime, lastTime);
return false;
}
}
sequence.put(checkpointTime, blobInfo);
checkpointTime = getLowerBoundTime(blobInfo);
}
logger.atInfo().log("Found sequence from %s to %s.", checkpointTime, lastTime);
return true;
}
ImmutableList<BlobInfo> listDiffFiles(
String gcsBucket, DateTime fromTime, @Nullable DateTime toTime) {
logger.atInfo().log("Requested restore from time: %s", fromTime);
if (toTime != null) {
logger.atInfo().log(" Until time: %s", toTime);
}
// List all of the diff files on GCS and build a map from each file's upper checkpoint time
// (extracted from the filename) to its asynchronously-loaded metadata, keeping only files with
// an upper checkpoint time > fromTime.
TreeMap<DateTime, ListenableFuture<BlobInfo>> upperBoundTimesToBlobInfo = new TreeMap<>();
String commitLogDiffPrefix = getCommitLogDiffPrefix(fromTime, toTime);
ImmutableList<String> filenames;
try {
filenames =
gcsUtils.listFolderObjects(gcsBucket, commitLogDiffPrefix).stream()
.map(s -> commitLogDiffPrefix + s)
.collect(toImmutableList());
} catch (IOException e) {
throw new RuntimeException(e);
}
DateTime lastUpperBoundTime = START_OF_TIME;
TreeMap<DateTime, BlobInfo> sequence = new TreeMap<>();
ListeningExecutorService executor = executorProvider.get();
try {
for (String filename : filenames) {
String strippedFilename = filename.replaceFirst(DIFF_FILE_PREFIX, "");
DateTime upperBoundTime = DateTime.parse(strippedFilename);
if (isInRange(upperBoundTime, fromTime, toTime)) {
upperBoundTimesToBlobInfo.put(
upperBoundTime, executor.submit(() -> getBlobInfo(gcsBucket, filename)));
lastUpperBoundTime = latestOf(upperBoundTime, lastUpperBoundTime);
}
}
if (upperBoundTimesToBlobInfo.isEmpty()) {
logger.atInfo().log("No files found.");
return ImmutableList.of();
}
// Reconstruct the sequence of files by traversing backwards from "lastUpperBoundTime" (i.e.
// the last file that we found) and finding its previous file until we either run out of files
// or get to one that precedes "fromTime".
//
// GCS file listing is eventually consistent, so it's possible that we are missing a file. The
// metadata of a file is sufficient to identify the preceding file, so if we start from the
// last file and work backwards we can verify that we have no holes in our chain (although we
// may be missing files at the end).
logger.atInfo().log("Restoring until: %s", lastUpperBoundTime);
boolean inconsistentFileSet =
!constructDiffSequence(
gcsBucket, upperBoundTimesToBlobInfo, fromTime, lastUpperBoundTime, sequence);
// Verify that all of the elements in the original set are represented in the sequence. If we
// find anything that's not represented, construct a sequence for it.
boolean checkForMoreExtraDiffs = true; // Always loop at least once.
while (checkForMoreExtraDiffs) {
checkForMoreExtraDiffs = false;
for (DateTime key : upperBoundTimesToBlobInfo.descendingKeySet()) {
if (!isInRange(key, fromTime, toTime)) {
break;
}
if (!sequence.containsKey(key)) {
// Recalculate the sequence for purely informational purposes.
logger.atWarning().log(
"Fork found in commit log history. The following sequence "
+ "is disconnected from the sequence of the final commit:");
constructDiffSequence(gcsBucket, upperBoundTimesToBlobInfo, fromTime, key, sequence);
checkForMoreExtraDiffs = true;
inconsistentFileSet = true;
break;
}
}
}
checkState(
!inconsistentFileSet,
"Unable to compute commit diff history, there are either gaps or forks in the history "
+ "file set. Check log for details.");
} finally {
executor.shutdown();
}
logger.atInfo().log(
"Actual restore from time: %s", getLowerBoundTime(sequence.firstEntry().getValue()));
logger.atInfo().log("Found %d files to restore.", sequence.size());
return ImmutableList.copyOf(sequence.values());
}
/**
* Returns true if 'time' is in range of 'start' and 'end'.
*
* <p>If 'end' is null, returns true if 'time' is after 'start'.
*/
private boolean isInRange(DateTime time, DateTime start, @Nullable DateTime end) {
return isBeforeOrAt(start, time) && (end == null || isBeforeOrAt(time, end));
}
private DateTime getLowerBoundTime(BlobInfo blobInfo) {
return DateTime.parse(blobInfo.getMetadata().get(LOWER_BOUND_CHECKPOINT));
}
private BlobInfo getBlobInfo(String gcsBucket, String filename) {
return gcsUtils.getBlobInfo(BlobId.of(gcsBucket, filename));
}
/**
* Returns a prefix guaranteed to cover all commit log diff files in the given range.
*
* <p>The listObjects call can be fairly slow if we search over many thousands or tens of
* thousands of files, so we restrict the search space. The commit logs have a file format of
* "commit_diff_until_2021-05-11T06:48:00.070Z" so we can often filter down as far as the hour.
*
* <p>Here, we get the longest prefix possible based on which fields (year, month, day, hour) the
* times in question have in common.
*/
@VisibleForTesting
static String getCommitLogDiffPrefix(DateTime from, @Nullable DateTime to) {
StringBuilder result = new StringBuilder(DIFF_FILE_PREFIX);
if (to == null || from.getYear() != to.getYear()) {
return result.toString();
}
result.append(from.getYear()).append('-');
if (from.getMonthOfYear() != to.getMonthOfYear()) {
return result.toString();
}
result.append(String.format("%02d-", from.getMonthOfYear()));
if (from.getDayOfMonth() != to.getDayOfMonth()) {
return result.toString();
}
result.append(String.format("%02dT", from.getDayOfMonth()));
if (from.getHourOfDay() != to.getHourOfDay()) {
return result.toString();
}
result.append(String.format("%02d:", from.getHourOfDay()));
return result.toString();
}
}

View File

@@ -1,321 +0,0 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.backup.ExportCommitLogDiffAction.DIFF_FILE_PREFIX;
import static google.registry.backup.RestoreCommitLogsAction.DRY_RUN_PARAM;
import static google.registry.model.ofy.EntityWritePriorities.getEntityPriority;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.util.DateTimeUtils.isAtOrAfter;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import static javax.servlet.http.HttpServletResponse.SC_NO_CONTENT;
import static javax.servlet.http.HttpServletResponse.SC_OK;
import static org.joda.time.Duration.standardHours;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.Key;
import com.google.cloud.storage.BlobInfo;
import com.google.common.collect.ImmutableList;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryConfig.Config;
import google.registry.gcs.GcsUtils;
import google.registry.model.UpdateAutoTimestamp;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.common.DatabaseMigrationStateSchedule;
import google.registry.model.common.DatabaseMigrationStateSchedule.MigrationState;
import google.registry.model.common.DatabaseMigrationStateSchedule.ReplayDirection;
import google.registry.model.replay.DatastoreEntity;
import google.registry.model.replay.DatastoreOnlyEntity;
import google.registry.model.replay.NonReplicatedEntity;
import google.registry.model.replay.ReplaySpecializer;
import google.registry.model.replay.SqlReplayCheckpoint;
import google.registry.model.server.Lock;
import google.registry.model.translators.VKeyTranslatorFactory;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Action.Method;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import google.registry.util.RequestStatusChecker;
import java.io.IOException;
import java.io.InputStream;
import java.util.Optional;
import javax.inject.Inject;
import javax.servlet.http.HttpServletResponse;
import org.joda.time.DateTime;
import org.joda.time.Duration;
import org.joda.time.Seconds;
/** Action that replays commit logs to Cloud SQL to keep it up to date. */
@Action(
service = Action.Service.BACKEND,
path = ReplayCommitLogsToSqlAction.PATH,
method = Method.POST,
automaticallyPrintOk = true,
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
@DeleteAfterMigration
public class ReplayCommitLogsToSqlAction implements Runnable {
static final String PATH = "/_dr/task/replayCommitLogsToSql";
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
public static final String REPLAY_TO_SQL_LOCK_NAME =
ReplayCommitLogsToSqlAction.class.getSimpleName();
public static final Duration REPLAY_TO_SQL_LOCK_LEASE_LENGTH = standardHours(1);
// Stop / pause where we are if we've been replaying for more than five minutes to avoid GAE
// request timeouts
private static final Duration REPLAY_TIMEOUT_DURATION = Duration.standardMinutes(5);
@Inject GcsUtils gcsUtils;
@Inject Response response;
@Inject RequestStatusChecker requestStatusChecker;
@Inject GcsDiffFileLister diffLister;
@Inject Clock clock;
@Inject
@Config("commitLogGcsBucket")
String gcsBucket;
/** If true, will exit after logging the commit log files that would otherwise be replayed. */
@Inject
@Parameter(DRY_RUN_PARAM)
boolean dryRun;
@Inject
ReplayCommitLogsToSqlAction() {}
@Override
public void run() {
DateTime startTime = clock.nowUtc();
MigrationState state = DatabaseMigrationStateSchedule.getValueAtTime(startTime);
if (!state.getReplayDirection().equals(ReplayDirection.DATASTORE_TO_SQL)) {
String message =
String.format(
"Skipping ReplayCommitLogsToSqlAction because we are in migration phase %s.", state);
logger.atInfo().log(message);
// App Engine will retry on any non-2xx status code, which we don't want in this case.
response.setStatus(SC_NO_CONTENT);
response.setPayload(message);
return;
}
Optional<Lock> lock =
Lock.acquireSql(
REPLAY_TO_SQL_LOCK_NAME,
null,
REPLAY_TO_SQL_LOCK_LEASE_LENGTH,
requestStatusChecker,
false);
if (!lock.isPresent()) {
String message = "Can't acquire SQL commit log replay lock, aborting.";
logger.atSevere().log(message);
// App Engine will retry on any non-2xx status code, which we don't want in this case.
// Let the next run after the next export happen naturally.
response.setStatus(SC_NO_CONTENT);
response.setPayload(message);
return;
}
try {
logger.atInfo().log("Beginning replay of commit logs.");
String resultMessage;
if (dryRun) {
resultMessage = executeDryRun();
} else {
resultMessage = replayFiles(startTime);
}
response.setStatus(SC_OK);
response.setPayload(resultMessage);
logger.atInfo().log(resultMessage);
} catch (Throwable t) {
String message = "Errored out replaying files.";
logger.atSevere().withCause(t).log(message);
response.setStatus(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);
response.setPayload(message);
} finally {
lock.ifPresent(Lock::releaseSql);
}
}
private String executeDryRun() {
// Start at the first millisecond we haven't seen yet
DateTime searchStartTime = jpaTm().transact(() -> SqlReplayCheckpoint.get().plusMillis(1));
// Search through the end of the hour
DateTime searchEndTime =
searchStartTime.withMinuteOfHour(59).withSecondOfMinute(59).withMillisOfSecond(999);
ImmutableList<String> fileBatch =
diffLister.listDiffFiles(gcsBucket, searchStartTime, searchEndTime).stream()
.map(BlobInfo::getName)
.collect(toImmutableList());
return String.format(
"Running in dry-run mode, the first set of commit log files processed would be from "
+ "searching from %s to %s and would contain %d file(s). They are (limit 10): \n%s",
searchStartTime,
searchEndTime,
fileBatch.size(),
fileBatch.stream().limit(10).collect(toImmutableList()));
}
private String replayFiles(DateTime startTime) {
DateTime replayTimeoutTime = startTime.plus(REPLAY_TIMEOUT_DURATION);
DateTime searchStartTime = jpaTm().transact(() -> SqlReplayCheckpoint.get().plusMillis(1));
int filesProcessed = 0;
int transactionsProcessed = 0;
// Starting from one millisecond after the last file we processed, search for and import files
// one hour at a time until we catch up to the current time or we hit the replay timeout (in
// which case the next run will pick up from where we leave off).
//
// We use hour-long batches because GCS supports filename prefix-based searches.
while (true) {
if (isAtOrAfter(clock.nowUtc(), replayTimeoutTime)) {
return createResponseString(
"Reached max execution time", startTime, filesProcessed, transactionsProcessed);
}
if (isBeforeOrAt(clock.nowUtc(), searchStartTime)) {
return createResponseString(
"Caught up to current time", startTime, filesProcessed, transactionsProcessed);
}
// Search through the end of the hour
DateTime searchEndTime =
searchStartTime.withMinuteOfHour(59).withSecondOfMinute(59).withMillisOfSecond(999);
ImmutableList<BlobInfo> fileBatch =
diffLister.listDiffFiles(gcsBucket, searchStartTime, searchEndTime);
if (fileBatch.isEmpty()) {
logger.atInfo().log(
"No remaining files found in hour %s, continuing search in the next hour.",
searchStartTime.toString("yyyy-MM-dd HH"));
}
for (BlobInfo file : fileBatch) {
transactionsProcessed += processFile(file);
filesProcessed++;
if (clock.nowUtc().isAfter(replayTimeoutTime)) {
return createResponseString(
"Reached max execution time", startTime, filesProcessed, transactionsProcessed);
}
}
searchStartTime = searchEndTime.plusMillis(1);
}
}
private String createResponseString(
String msg, DateTime startTime, int filesProcessed, int transactionsProcessed) {
double tps =
(double) transactionsProcessed
/ (double) Seconds.secondsBetween(startTime, clock.nowUtc()).getSeconds();
return String.format(
"%s after replaying %d file(s) containing %d total transaction(s) (%.2f tx/s).",
msg, filesProcessed, transactionsProcessed, tps);
}
/**
* Replays the commit logs in the given commit log file and returns the number of transactions
* committed.
*/
private int processFile(BlobInfo metadata) {
try (InputStream input = gcsUtils.openInputStream(metadata.getBlobId())) {
// Load and process the Datastore transactions one at a time
ImmutableList<ImmutableList<VersionedEntity>> allTransactions =
CommitLogImports.loadEntitiesByTransaction(input);
try (UpdateAutoTimestamp.DisableAutoUpdateResource disabler =
UpdateAutoTimestamp.disableAutoUpdate()) {
allTransactions.forEach(
transaction -> jpaTm().transact(() -> replayTransaction(transaction)));
}
// if we succeeded, set the last-seen time
DateTime checkpoint = DateTime.parse(metadata.getName().substring(DIFF_FILE_PREFIX.length()));
jpaTm().transact(() -> SqlReplayCheckpoint.set(checkpoint));
logger.atInfo().log(
"Replayed %d transactions from commit log file %s with size %d B.",
allTransactions.size(), metadata.getName(), metadata.getSize());
return allTransactions.size();
} catch (IOException e) {
throw new RuntimeException(
"Errored out while replaying commit log file " + metadata.getName(), e);
}
}
private void replayTransaction(ImmutableList<VersionedEntity> transaction) {
transaction.stream()
.sorted(ReplayCommitLogsToSqlAction::compareByWeight)
.forEach(
versionedEntity -> {
if (versionedEntity.getEntity().isPresent()) {
handleEntityPut(versionedEntity.getEntity().get());
} else {
handleEntityDelete(versionedEntity);
}
});
}
private void handleEntityPut(Entity entity) {
Object ofyPojo = auditedOfy().toPojo(entity);
try {
if (ofyPojo instanceof DatastoreEntity) {
DatastoreEntity datastoreEntity = (DatastoreEntity) ofyPojo;
datastoreEntity
.toSqlEntity()
.ifPresent(
sqlEntity -> {
sqlEntity.beforeSqlSaveOnReplay();
jpaTm().putIgnoringReadOnlyWithoutBackup(sqlEntity);
});
} else {
// this should never happen, but we shouldn't fail on it
logger.atSevere().log(
"%s does not implement DatastoreEntity, which is necessary for SQL replay.",
ofyPojo.getClass());
}
} catch (Throwable t) {
logger.atSevere().log("Error when replaying object %s.", ofyPojo);
throw t;
}
}
private void handleEntityDelete(VersionedEntity entityToDelete) {
Key key = entityToDelete.key();
VKey<?> entityVKey;
try {
entityVKey = VKeyTranslatorFactory.createVKey(key);
} catch (RuntimeException e) {
// This means that the key wasn't convertible to VKey through the standard methods or via
// a createVKey method. This means that the object isn't persisted in SQL so we ignore it.
logger.atInfo().log(
"Skipping SQL delete for kind %s since it is not convertible.", key.getKind());
return;
}
try {
Class<?> entityClass = entityVKey.getKind();
// Delete the key iff the class represents a JPA entity that is replicated
if (!NonReplicatedEntity.class.isAssignableFrom(entityClass)
&& !DatastoreOnlyEntity.class.isAssignableFrom(entityClass)
&& entityClass.getAnnotation(javax.persistence.Entity.class) != null) {
ReplaySpecializer.beforeSqlDelete(entityVKey);
jpaTm().deleteIgnoringReadOnlyWithoutBackup(entityVKey);
}
} catch (Throwable t) {
logger.atSevere().log("Error when deleting key %s.", entityVKey);
throw t;
}
}
private static int compareByWeight(VersionedEntity a, VersionedEntity b) {
return getEntityPriority(a.key().getKind(), !a.getEntity().isPresent())
- getEntityPriority(b.key().getKind(), !b.getEntity().isPresent());
}
}

View File

@@ -1,208 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.Iterators.peekingIterator;
import static google.registry.backup.BackupUtils.createDeserializingIterator;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.EntityTranslator;
import com.google.cloud.storage.BlobInfo;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Lists;
import com.google.common.collect.PeekingIterator;
import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.Result;
import com.googlecode.objectify.util.ResultNow;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryEnvironment;
import google.registry.gcs.GcsUtils;
import google.registry.model.ImmutableObject;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.ofy.CommitLogBucket;
import google.registry.model.ofy.CommitLogCheckpoint;
import google.registry.model.ofy.CommitLogCheckpointRoot;
import google.registry.model.ofy.CommitLogManifest;
import google.registry.model.ofy.CommitLogMutation;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
import google.registry.util.Retrier;
import java.io.IOException;
import java.io.InputStream;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import java.util.stream.Stream;
import javax.inject.Inject;
import org.joda.time.DateTime;
/** Restore Registry 2 commit logs from GCS to Datastore. */
@Action(
service = Action.Service.TOOLS,
path = RestoreCommitLogsAction.PATH,
method = Action.Method.POST,
automaticallyPrintOk = true,
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
@DeleteAfterMigration
public class RestoreCommitLogsAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
public static final String PATH = "/_dr/task/restoreCommitLogs";
static final String DRY_RUN_PARAM = "dryRun";
static final String FROM_TIME_PARAM = "fromTime";
static final String TO_TIME_PARAM = "toTime";
static final String BUCKET_OVERRIDE_PARAM = "gcsBucket";
private static final ImmutableSet<RegistryEnvironment> FORBIDDEN_ENVIRONMENTS =
ImmutableSet.of(RegistryEnvironment.PRODUCTION, RegistryEnvironment.SANDBOX);
@Inject GcsUtils gcsUtils;
@Inject @Parameter(DRY_RUN_PARAM) boolean dryRun;
@Inject @Parameter(FROM_TIME_PARAM) DateTime fromTime;
@Inject @Parameter(TO_TIME_PARAM) DateTime toTime;
@Inject
@Parameter(BUCKET_OVERRIDE_PARAM)
Optional<String> gcsBucketOverride;
@Inject DatastoreService datastoreService;
@Inject GcsDiffFileLister diffLister;
@Inject
@Config("commitLogGcsBucket")
String defaultGcsBucket;
@Inject Retrier retrier;
@Inject RestoreCommitLogsAction() {}
@Override
public void run() {
checkArgument(
!FORBIDDEN_ENVIRONMENTS.contains(RegistryEnvironment.get()),
"DO NOT RUN IN PRODUCTION OR SANDBOX.");
if (dryRun) {
logger.atInfo().log("Running in dry-run mode.");
}
String gcsBucket = gcsBucketOverride.orElse(defaultGcsBucket);
logger.atInfo().log("Restoring from %s.", gcsBucket);
List<BlobInfo> diffFiles = diffLister.listDiffFiles(gcsBucket, fromTime, toTime);
if (diffFiles.isEmpty()) {
logger.atInfo().log("Nothing to restore.");
return;
}
Map<Integer, DateTime> bucketTimestamps = new HashMap<>();
CommitLogCheckpoint lastCheckpoint = null;
for (BlobInfo metadata : diffFiles) {
logger.atInfo().log("Restoring: %s", metadata.getName());
try (InputStream input = gcsUtils.openInputStream(metadata.getBlobId())) {
PeekingIterator<ImmutableObject> commitLogs =
peekingIterator(createDeserializingIterator(input, true));
lastCheckpoint = (CommitLogCheckpoint) commitLogs.next();
saveOfy(ImmutableList.of(lastCheckpoint)); // Save the checkpoint itself.
while (commitLogs.hasNext()) {
CommitLogManifest manifest = restoreOneTransaction(commitLogs);
bucketTimestamps.put(manifest.getBucketId(), manifest.getCommitTime());
}
} catch (IOException e) {
throw new RuntimeException(e);
}
}
// Restore the CommitLogCheckpointRoot and CommitLogBuckets.
saveOfy(
Streams.concat(
bucketTimestamps
.entrySet()
.stream()
.map(
entry ->
new CommitLogBucket.Builder()
.setBucketNum(entry.getKey())
.setLastWrittenTime(entry.getValue())
.build()),
Stream.of(CommitLogCheckpointRoot.create(lastCheckpoint.getCheckpointTime())))
.collect(toImmutableList()));
logger.atInfo().log("Restore complete.");
}
/**
* Restore the contents of one transaction to Datastore.
*
* <p>The objects to delete are listed in the {@link CommitLogManifest}, which will be the first
* object in the iterable. The objects to save follow, each as a {@link CommitLogMutation}. We
* restore by deleting the deletes and recreating the saves from their proto form. We also save
* the commit logs themselves back to Datastore, so that the commit log system itself is
* transparently restored alongside the data.
*
* @return the manifest, for use in restoring the {@link CommitLogBucket}.
*/
private CommitLogManifest restoreOneTransaction(PeekingIterator<ImmutableObject> commitLogs) {
final CommitLogManifest manifest = (CommitLogManifest) commitLogs.next();
Result<?> deleteResult = deleteAsync(manifest.getDeletions());
List<Entity> entitiesToSave = Lists.newArrayList(auditedOfy().save().toEntity(manifest));
while (commitLogs.hasNext() && commitLogs.peek() instanceof CommitLogMutation) {
CommitLogMutation mutation = (CommitLogMutation) commitLogs.next();
entitiesToSave.add(auditedOfy().save().toEntity(mutation));
entitiesToSave.add(EntityTranslator.createFromPbBytes(mutation.getEntityProtoBytes()));
}
saveRaw(entitiesToSave);
try {
deleteResult.now();
} catch (Exception e) {
retrier.callWithRetry(
() -> deleteAsync(manifest.getDeletions()).now(), RuntimeException.class);
}
return manifest;
}
private void saveRaw(List<Entity> entitiesToSave) {
if (dryRun) {
logger.atInfo().log("Would have saved entities: %s", entitiesToSave);
return;
}
retrier.callWithRetry(() -> datastoreService.put(entitiesToSave), RuntimeException.class);
}
private void saveOfy(Iterable<? extends ImmutableObject> objectsToSave) {
if (dryRun) {
logger.atInfo().log("Would have saved entities: %s", objectsToSave);
return;
}
retrier.callWithRetry(
() -> auditedOfy().saveWithoutBackup().entities(objectsToSave).now(),
RuntimeException.class);
}
private Result<?> deleteAsync(Set<Key<?>> keysToDelete) {
if (dryRun) {
logger.atInfo().log("Would have deleted entities: %s", keysToDelete);
}
return dryRun || keysToDelete.isEmpty()
? new ResultNow<Void>(null)
: auditedOfy().deleteWithoutBackup().keys(keysToDelete);
}
}

View File

@@ -1,187 +0,0 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import static javax.servlet.http.HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
import static javax.servlet.http.HttpServletResponse.SC_OK;
import com.google.common.flogger.FluentLogger;
import google.registry.beam.comparedb.LatestDatastoreSnapshotFinder;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.ofy.CommitLogCheckpoint;
import google.registry.model.replay.ReplicateToDatastoreAction;
import google.registry.request.Action;
import google.registry.request.Action.Service;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Sleeper;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import java.util.Optional;
import javax.inject.Inject;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/**
* Synchronizes Datastore to a given SQL snapshot when SQL is the primary database.
*
* <p>The caller takes the responsibility for:
*
* <ul>
* <li>verifying the current migration stage
* <li>acquiring the {@link ReplicateToDatastoreAction#REPLICATE_TO_DATASTORE_LOCK_NAME
* replication lock}, and
* <li>while holding the lock, creating an SQL snapshot and invoking this action with the snapshot
* id
* </ul>
*
* The caller may release the replication lock upon receiving the response from this action. Please
* refer to {@link google.registry.tools.ValidateDatastoreCommand} for more information on usage.
*
* <p>This action plays SQL transactions up to the user-specified snapshot, creates a new CommitLog
* checkpoint, and exports all CommitLogs to GCS up to this checkpoint. The timestamp of this
* checkpoint can be used to recreate a Datastore snapshot that is equivalent to the given SQL
* snapshot. If this action succeeds, the checkpoint timestamp is included in the response (the
* format of which is defined by {@link #SUCCESS_RESPONSE_TEMPLATE}).
*/
@Action(
service = Service.BACKEND,
path = SyncDatastoreToSqlSnapshotAction.PATH,
method = Action.Method.POST,
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
@DeleteAfterMigration
public class SyncDatastoreToSqlSnapshotAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
public static final String PATH = "/_dr/task/syncDatastoreToSqlSnapshot";
public static final String SUCCESS_RESPONSE_TEMPLATE =
"Datastore is up-to-date with provided SQL snapshot (%s). CommitLog timestamp is (%s).";
static final String SQL_SNAPSHOT_ID_PARAM = "sqlSnapshotId";
private static final int COMMITLOGS_PRESENCE_CHECK_ATTEMPTS = 10;
private static final Duration COMMITLOGS_PRESENCE_CHECK_DELAY = Duration.standardSeconds(6);
private final Response response;
private final Sleeper sleeper;
@Config("commitLogGcsBucket")
private final String gcsBucket;
private final GcsDiffFileLister gcsDiffFileLister;
private final LatestDatastoreSnapshotFinder datastoreSnapshotFinder;
private final CommitLogCheckpointAction commitLogCheckpointAction;
private final String sqlSnapshotId;
@Inject
SyncDatastoreToSqlSnapshotAction(
Response response,
Sleeper sleeper,
@Config("commitLogGcsBucket") String gcsBucket,
GcsDiffFileLister gcsDiffFileLister,
LatestDatastoreSnapshotFinder datastoreSnapshotFinder,
CommitLogCheckpointAction commitLogCheckpointAction,
@Parameter(SQL_SNAPSHOT_ID_PARAM) String sqlSnapshotId) {
this.response = response;
this.sleeper = sleeper;
this.gcsBucket = gcsBucket;
this.gcsDiffFileLister = gcsDiffFileLister;
this.datastoreSnapshotFinder = datastoreSnapshotFinder;
this.commitLogCheckpointAction = commitLogCheckpointAction;
this.sqlSnapshotId = sqlSnapshotId;
}
@Override
public void run() {
logger.atInfo().log("Datastore validation invoked. SqlSnapshotId is %s.", sqlSnapshotId);
try {
CommitLogCheckpoint checkpoint = ensureDatabasesComparable(sqlSnapshotId);
response.setStatus(SC_OK);
response.setPayload(
String.format(SUCCESS_RESPONSE_TEMPLATE, sqlSnapshotId, checkpoint.getCheckpointTime()));
return;
} catch (Throwable e) {
logger.atSevere().withCause(e).log("Failed to sync Datastore to SQL.");
response.setStatus(SC_INTERNAL_SERVER_ERROR);
response.setPayload(getStackTrace(e));
}
}
private static String getStackTrace(Throwable e) {
try {
ByteArrayOutputStream bis = new ByteArrayOutputStream();
PrintStream printStream = new PrintStream(bis);
e.printStackTrace(printStream);
printStream.close();
return bis.toString();
} catch (RuntimeException re) {
return re.getMessage();
}
}
private CommitLogCheckpoint ensureDatabasesComparable(String sqlSnapshotId) {
// Replicate SQL transaction to Datastore, up to when this snapshot is taken.
int playbacks = ReplicateToDatastoreAction.replayAllTransactions(Optional.of(sqlSnapshotId));
logger.atInfo().log("Played %s SQL transactions.", playbacks);
Optional<CommitLogCheckpoint> checkpoint = exportCommitLogs();
if (!checkpoint.isPresent()) {
throw new RuntimeException("Cannot create CommitLog checkpoint");
}
logger.atInfo().log(
"CommitLog checkpoint created at %s.", checkpoint.get().getCheckpointTime());
verifyCommitLogsPersisted(checkpoint.get());
return checkpoint.get();
}
private Optional<CommitLogCheckpoint> exportCommitLogs() {
// Trigger an async CommitLog export to GCS. Will check file availability later.
// Although we can add support to synchronous execution, it can disrupt the export cadence
// when the system is busy
Optional<CommitLogCheckpoint> checkpoint =
commitLogCheckpointAction.createCheckPointAndStartAsyncExport();
// Failure to create checkpoint most likely caused by race with cron-triggered checkpointing.
// Retry once.
if (!checkpoint.isPresent()) {
commitLogCheckpointAction.createCheckPointAndStartAsyncExport();
}
return checkpoint;
}
private void verifyCommitLogsPersisted(CommitLogCheckpoint checkpoint) {
DateTime exportStartTime =
datastoreSnapshotFinder
.getSnapshotInfo(checkpoint.getCheckpointTime().toInstant())
.exportInterval()
.getStart();
logger.atInfo().log("Found Datastore export at %s", exportStartTime);
for (int attempts = 0; attempts < COMMITLOGS_PRESENCE_CHECK_ATTEMPTS; attempts++) {
try {
gcsDiffFileLister.listDiffFiles(gcsBucket, exportStartTime, checkpoint.getCheckpointTime());
return;
} catch (IllegalStateException e) {
// Gap in commitlog files. Fall through to sleep and retry.
logger.atInfo().log("Commitlog files not yet found on GCS.");
}
sleeper.sleepInterruptibly(COMMITLOGS_PRESENCE_CHECK_DELAY);
}
throw new RuntimeException("Cannot find all commitlog files.");
}
}

View File

@@ -1,200 +0,0 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.backup;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.EntityTranslator;
import com.google.appengine.api.datastore.Key;
import com.google.auto.value.AutoValue;
import com.google.auto.value.extension.memoized.Memoized;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.ofy.CommitLogManifest;
import google.registry.model.ofy.CommitLogMutation;
import java.io.Serializable;
import java.util.Arrays;
import java.util.Optional;
import java.util.stream.Stream;
import javax.annotation.Nullable;
/**
* A Datastore {@link Entity Entity's} serialized state with timestamp. The intended use case is a
* multi-stage pipeline where an Entity's Java form is not needed in most stages.
*
* <p>For a new or updated Entity, its serialized bytes are stored along with its Datastore {@link
* Key}. For a deleted entity, only its Datastore {@link Key} is stored, and the {@link
* #entityProtoBytes} field is left unset.
*
* <p>Storing raw bytes is motivated by two factors. First, since I/O is frequent and the Java
* objects are rarely needed in our target use case, storing raw bytes is the most efficient
* approach. More importantly, due to our data model and our customization of {@link
* google.registry.model.ofy.ObjectifyService ObjectifyService}, it is challenging to implement a
* serializer for Objectify entities that preserves the value of all properties. Without such
* serializers, Objectify entities cannot be used in a pipeline.
*
* <p>Objectify entities do not implement {@link Serializable}, serialization of such objects is as
* follows:
*
* <ul>
* <li>Convert an Objectify entity to a Datastore {@link Entity}: {@code
* auditedOfy().save().toEntity(..)}
* <li>Entity is serializable, but the more efficient approach is to convert an Entity to a
* ProtocolBuffer ({@link com.google.storage.onestore.v3.OnestoreEntity.EntityProto}) and then
* to raw bytes.
* </ul>
*
* <p>When the first conversion above is applied to an Objectify entity, a property value in the
* output may differ from the input in two situations:
*
* <ul>
* <li>If a property is of an assign-on-persist data type, e.g., {@link
* google.registry.model.UpdateAutoTimestamp}.
* <li>If it is related to CommitLog management, e.g., {@link google.registry.model.EppResource
* EppResource.revisions}.
* </ul>
*
* <p>Working around the side effects caused by our customization is difficult. Any solution would
* likely rely on Objectify's stack of context. However, many Objectify invocations in our code base
* are hardcoded to call the customized version of ObjectifyService, rendering Objectify's stack
* useless.
*
* <p>For now, this inability to use Objectify entities in pipelines is mostly a testing problem: we
* can not perform {@link org.apache.beam.sdk.testing.PAssert BEAM pipeline assertions} on Objectify
* entities. {@code InitSqlTestUtils.assertContainsExactlyElementsIn} is an example of a workaround.
*
* <p>Note that {@link Optional java.util.Optional} is not serializable, therefore cannot be used as
* property type in this class.
*/
@AutoValue
@DeleteAfterMigration
public abstract class VersionedEntity implements Serializable {
private static final long serialVersionUID = 1L;
public abstract long commitTimeMills();
/** The {@link Key} of the {@link Entity}. */
public abstract Key key();
/** Serialized form of the {@link Entity}. This property is {@code null} for a deleted Entity. */
@Nullable
abstract ImmutableBytes entityProtoBytes();
@Memoized
public Optional<Entity> getEntity() {
return Optional.ofNullable(entityProtoBytes())
.map(ImmutableBytes::getBytes)
.map(EntityTranslator::createFromPbBytes);
}
public boolean isDelete() {
return entityProtoBytes() == null;
}
/**
* Converts deleted entity keys in {@code manifest} into a {@link Stream} of {@link
* VersionedEntity VersionedEntities}. See {@link CommitLogImports#loadEntities} for more
* information.
*/
static Stream<VersionedEntity> fromManifest(CommitLogManifest manifest) {
long commitTimeMillis = manifest.getCommitTime().getMillis();
return manifest.getDeletions().stream()
.map(com.googlecode.objectify.Key::getRaw)
.map(key -> newBuilder().commitTimeMills(commitTimeMillis).key(key).build());
}
/* Converts a {@link CommitLogMutation} to a {@link VersionedEntity}. */
static VersionedEntity fromMutation(CommitLogMutation mutation) {
return from(
com.googlecode.objectify.Key.create(mutation).getParent().getId(),
mutation.getEntityProtoBytes());
}
public static VersionedEntity from(long commitTimeMillis, byte[] entityProtoBytes) {
return newBuilder()
.entityProtoBytes(entityProtoBytes)
.key(EntityTranslator.createFromPbBytes(entityProtoBytes).getKey())
.commitTimeMills(commitTimeMillis)
.build();
}
private static Builder newBuilder() {
return new AutoValue_VersionedEntity.Builder();
}
@AutoValue.Builder
public abstract static class Builder {
public abstract Builder commitTimeMills(long commitTimeMillis);
abstract Builder entityProtoBytes(ImmutableBytes bytes);
public abstract Builder key(Key key);
public abstract VersionedEntity build();
Builder entityProtoBytes(byte[] bytes) {
return entityProtoBytes(new ImmutableBytes(bytes));
}
}
/**
* Wraps a byte array and prevents it from being modified by its original owner.
*
* <p>While this class seems an overkill, it exists for two reasons:
*
* <ul>
* <li>It is easier to override the {@link #equals} method here (for value-equivalence check)
* than to override the AutoValue-generated {@code equals} method.
* <li>To appease the style checker, which forbids arrays as AutoValue property.
* </ul>
*/
static final class ImmutableBytes implements Serializable {
private static final long serialVersionUID = 1L;
private final byte[] bytes;
ImmutableBytes(byte[] bytes) {
this.bytes = Arrays.copyOf(bytes, bytes.length);
}
/**
* Returns the saved byte array. Invocation is restricted to trusted callers, who must not
* modify the array.
*/
byte[] getBytes() {
return bytes;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof ImmutableBytes)) {
return false;
}
ImmutableBytes that = (ImmutableBytes) o;
// Do not use Objects.equals, which checks reference identity instead of data in array.
return Arrays.equals(bytes, that.bytes);
}
@Override
public int hashCode() {
// Do not use Objects.hashCode, which hashes the reference, not the data in array.
return Arrays.hashCode(bytes);
}
}
}

View File

@@ -1,16 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
@javax.annotation.ParametersAreNonnullByDefault
package google.registry.backup;

View File

@@ -71,7 +71,7 @@ public final class AsyncTaskEnqueuer {
public AsyncTaskEnqueuer(
@Named(QUEUE_ASYNC_DELETE) Queue asyncDeletePullQueue,
@Named(QUEUE_ASYNC_HOST_RENAME) Queue asyncDnsRefreshPullQueue,
@Config("asyncDeleteFlowMapreduceDelay") Duration asyncDeleteDelay,
@Config("asyncDeleteDelay") Duration asyncDeleteDelay,
CloudTasksUtils cloudTasksUtils,
Retrier retrier) {
this.asyncDeletePullQueue = asyncDeletePullQueue;

View File

@@ -21,6 +21,7 @@ import static google.registry.batch.AsyncTaskEnqueuer.PARAM_RESOURCE_KEY;
import static google.registry.batch.AsyncTaskEnqueuer.QUEUE_ASYNC_ACTIONS;
import static google.registry.batch.AsyncTaskEnqueuer.QUEUE_ASYNC_DELETE;
import static google.registry.batch.AsyncTaskEnqueuer.QUEUE_ASYNC_HOST_RENAME;
import static google.registry.request.RequestParameters.extractBooleanParameter;
import static google.registry.request.RequestParameters.extractIntParameter;
import static google.registry.request.RequestParameters.extractLongParameter;
import static google.registry.request.RequestParameters.extractOptionalBooleanParameter;
@@ -45,6 +46,9 @@ import org.joda.time.DateTime;
@Module
public class BatchModule {
public static final String PARAM_DRY_RUN = "dryRun";
public static final String PARAM_FAST = "fast";
@Provides
@Parameter("jobName")
static Optional<String> provideJobName(HttpServletRequest req) {
@@ -112,6 +116,18 @@ public class BatchModule {
req, ExpandRecurringBillingEventsAction.PARAM_CURSOR_TIME);
}
@Provides
@Parameter(PARAM_FAST)
static boolean provideIsFast(HttpServletRequest req) {
return extractBooleanParameter(req, PARAM_FAST);
}
@Provides
@Parameter(PARAM_DRY_RUN)
static boolean provideIsDryRun(HttpServletRequest req) {
return extractBooleanParameter(req, PARAM_DRY_RUN);
}
@Provides
@Named(QUEUE_ASYNC_ACTIONS)
static Queue provideAsyncActionsPushQueue() {

View File

@@ -1,621 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch;
import static com.google.appengine.api.taskqueue.QueueConstants.maxLeaseCount;
import static com.google.appengine.api.taskqueue.QueueFactory.getQueue;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.math.IntMath.divide;
import static com.googlecode.objectify.Key.getKind;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_CLIENT_TRANSACTION_ID;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_IS_SUPERUSER;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_REQUESTED_TIME;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_REQUESTING_CLIENT_ID;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_RESOURCE_KEY;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_SERVER_TRANSACTION_ID;
import static google.registry.batch.AsyncTaskEnqueuer.QUEUE_ASYNC_DELETE;
import static google.registry.model.EppResourceUtils.isActive;
import static google.registry.model.EppResourceUtils.isDeleted;
import static google.registry.model.ResourceTransferUtils.denyPendingTransfer;
import static google.registry.model.ResourceTransferUtils.handlePendingTransferOnDelete;
import static google.registry.model.ResourceTransferUtils.updateForeignKeyIndexDeletionTime;
import static google.registry.model.eppcommon.StatusValue.PENDING_DELETE;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.model.reporting.HistoryEntry.Type.CONTACT_DELETE;
import static google.registry.model.reporting.HistoryEntry.Type.CONTACT_DELETE_FAILURE;
import static google.registry.model.reporting.HistoryEntry.Type.HOST_DELETE;
import static google.registry.model.reporting.HistoryEntry.Type.HOST_DELETE_FAILURE;
import static google.registry.model.transfer.TransferStatus.SERVER_CANCELLED;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static java.math.RoundingMode.CEILING;
import static java.util.concurrent.TimeUnit.DAYS;
import static java.util.concurrent.TimeUnit.SECONDS;
import static java.util.logging.Level.INFO;
import static java.util.logging.Level.SEVERE;
import static org.joda.time.Duration.standardHours;
import com.google.appengine.api.taskqueue.LeaseOptions;
import com.google.appengine.api.taskqueue.Queue;
import com.google.appengine.api.taskqueue.TaskHandle;
import com.google.appengine.api.taskqueue.TransientFailureException;
import com.google.appengine.tools.mapreduce.Mapper;
import com.google.appengine.tools.mapreduce.Reducer;
import com.google.appengine.tools.mapreduce.ReducerInput;
import com.google.auto.value.AutoValue;
import com.google.common.collect.HashMultiset;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.Iterators;
import com.google.common.collect.Multiset;
import com.google.common.flogger.FluentLogger;
import com.googlecode.objectify.Key;
import google.registry.batch.AsyncTaskMetrics.OperationResult;
import google.registry.batch.AsyncTaskMetrics.OperationType;
import google.registry.batch.DeleteContactsAndHostsAction.DeletionResult.Type;
import google.registry.dns.DnsQueue;
import google.registry.mapreduce.MapreduceRunner;
import google.registry.mapreduce.UnlockerOutput;
import google.registry.mapreduce.inputs.EppResourceInputs;
import google.registry.mapreduce.inputs.NullInput;
import google.registry.model.EppResource;
import google.registry.model.ImmutableObject;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.annotations.ExternalMessagingName;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.DomainBase;
import google.registry.model.eppcommon.StatusValue;
import google.registry.model.eppcommon.Trid;
import google.registry.model.eppoutput.EppResponse.ResponseData;
import google.registry.model.host.HostResource;
import google.registry.model.poll.PendingActionNotificationResponse.ContactPendingActionNotificationResponse;
import google.registry.model.poll.PendingActionNotificationResponse.HostPendingActionNotificationResponse;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.server.Lock;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import google.registry.util.NonFinalForTesting;
import google.registry.util.RequestStatusChecker;
import google.registry.util.Retrier;
import google.registry.util.SystemClock;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
import java.util.Optional;
import java.util.logging.Level;
import javax.annotation.Nullable;
import javax.inject.Inject;
import javax.inject.Named;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/**
* A mapreduce that processes batch asynchronous deletions of contact and host resources by mapping
* over all domains and checking for any references to the contacts/hosts in pending deletion.
*/
@Deprecated
@Action(
service = Action.Service.BACKEND,
path = "/_dr/task/deleteContactsAndHosts",
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
@DeleteAfterMigration
public class DeleteContactsAndHostsAction implements Runnable {
static final String KIND_CONTACT = getKind(ContactResource.class);
static final String KIND_HOST = getKind(HostResource.class);
private static final Duration LEASE_LENGTH = standardHours(4);
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final int MAX_REDUCE_SHARDS = 50;
private static final int DELETES_PER_SHARD = 5;
@Inject AsyncTaskMetrics asyncTaskMetrics;
@Inject Clock clock;
@Inject MapreduceRunner mrRunner;
@Inject @Named(QUEUE_ASYNC_DELETE) Queue queue;
@Inject RequestStatusChecker requestStatusChecker;
@Inject Response response;
@Inject Retrier retrier;
@Inject DeleteContactsAndHostsAction() {}
@Override
public void run() {
// Check if the lock can be acquired, and if not, a previous run of this mapreduce is still
// executing, so return early.
Optional<Lock> lock =
Lock.acquire(
DeleteContactsAndHostsAction.class.getSimpleName(),
null,
LEASE_LENGTH,
requestStatusChecker,
false);
if (!lock.isPresent()) {
logRespondAndUnlock(INFO, "Can't acquire lock; aborting.", lock);
return;
}
// Lease the async tasks to process.
LeaseOptions options =
LeaseOptions.Builder.withCountLimit(maxLeaseCount())
.leasePeriod(LEASE_LENGTH.getStandardSeconds(), SECONDS);
List<TaskHandle> tasks = queue.leaseTasks(options);
asyncTaskMetrics.recordContactHostDeletionBatchSize(tasks.size());
// Check if there are no tasks to process, and if so, return early.
if (tasks.isEmpty()) {
logRespondAndUnlock(INFO, "No contact/host deletion tasks in pull queue; finishing.", lock);
return;
}
Multiset<String> kindCounts = HashMultiset.create(2);
ImmutableList.Builder<DeletionRequest> builder = new ImmutableList.Builder<>();
ImmutableList.Builder<Key<? extends EppResource>> resourceKeys = new ImmutableList.Builder<>();
final List<DeletionRequest> requestsToDelete = new ArrayList<>();
for (TaskHandle task : tasks) {
try {
DeletionRequest deletionRequest = DeletionRequest.createFromTask(task, clock.nowUtc());
if (deletionRequest.isDeletionAllowed()) {
builder.add(deletionRequest);
resourceKeys.add(deletionRequest.key());
kindCounts.add(deletionRequest.key().getKind());
} else {
requestsToDelete.add(deletionRequest);
}
} catch (Exception e) {
logger.atSevere().withCause(e).log(
"Could not parse async deletion request, delaying task for a day: %s", task);
// Grab the lease for a whole day, so that it won't continue throwing errors every five
// minutes.
queue.modifyTaskLease(task, 1L, DAYS);
}
}
deleteStaleTasksWithRetry(requestsToDelete);
ImmutableList<DeletionRequest> deletionRequests = builder.build();
if (deletionRequests.isEmpty()) {
logRespondAndUnlock(
INFO, "No async deletions to process because all were already handled.", lock);
} else {
logger.atInfo().log(
"Processing asynchronous deletion of %d contacts and %d hosts: %s",
kindCounts.count(KIND_CONTACT), kindCounts.count(KIND_HOST), resourceKeys.build());
runMapreduce(deletionRequests, lock);
}
}
/**
* Deletes a list of tasks associated with deletion requests from the async delete queue using a
* retrier.
*/
private void deleteStaleTasksWithRetry(final List<DeletionRequest> deletionRequests) {
if (deletionRequests.isEmpty()) {
return;
}
final List<TaskHandle> tasks =
deletionRequests.stream().map(DeletionRequest::task).collect(toImmutableList());
retrier.callWithRetry(() -> queue.deleteTask(tasks), TransientFailureException.class);
deletionRequests.forEach(
deletionRequest ->
asyncTaskMetrics.recordAsyncFlowResult(
deletionRequest.getMetricOperationType(),
OperationResult.STALE,
deletionRequest.requestedTime()));
}
private void runMapreduce(ImmutableList<DeletionRequest> deletionRequests, Optional<Lock> lock) {
try {
int numReducers =
Math.min(MAX_REDUCE_SHARDS, divide(deletionRequests.size(), DELETES_PER_SHARD, CEILING));
mrRunner
.setJobName("Check for EPP resource references and then delete")
.setModuleName("backend")
.setDefaultReduceShards(numReducers)
.runMapreduce(
new DeleteContactsAndHostsMapper(deletionRequests),
new DeleteEppResourceReducer(),
ImmutableList.of(
// Add an extra shard that maps over a null domain. See the mapper code for why.
new NullInput<>(), EppResourceInputs.createEntityInput(DomainBase.class)),
new UnlockerOutput<Void>(lock.get()))
.sendLinkToMapreduceConsole(response);
} catch (Throwable t) {
logRespondAndUnlock(SEVERE, "Error starting mapreduce to delete contacts/hosts.", lock);
}
}
private void logRespondAndUnlock(Level level, String message, Optional<Lock> lock) {
logger.at(level).log(message);
response.setPayload(message);
lock.ifPresent(Lock::release);
}
/**
* A mapper that iterates over all {@link DomainBase} entities.
*
* <p>It emits the target key and {@code true} for domains referencing the target resource. For
* the special input of {@code null} it emits the target key and {@code false}.
*/
public static class DeleteContactsAndHostsMapper
extends Mapper<DomainBase, DeletionRequest, Boolean> {
private static final long serialVersionUID = -253652818502690537L;
private final ImmutableList<DeletionRequest> deletionRequests;
DeleteContactsAndHostsMapper(ImmutableList<DeletionRequest> resourcesToDelete) {
this.deletionRequests = resourcesToDelete;
}
@Override
public void map(DomainBase domain) {
for (DeletionRequest deletionRequest : deletionRequests) {
if (domain == null) {
// The reducer only runs if at least one value is emitted. We add a null input to the
// mapreduce and emit one 'false' for each deletion request so that the reducer always
// runs for each requested deletion (so that it can finish up tasks if nothing else).
emit(deletionRequest, false);
} else if (isActive(domain, deletionRequest.lastUpdateTime())
&& isLinked(domain, deletionRequest.key())) {
emit(deletionRequest, true);
getContext()
.incrementCounter(
String.format("active Domain-%s links found", deletionRequest.key().getKind()));
}
}
if (domain != null) {
getContext().incrementCounter("domains processed");
}
}
/** Determine whether the target resource is a linked resource on the domain. */
private boolean isLinked(DomainBase domain, Key<? extends EppResource> resourceKey) {
if (resourceKey.getKind().equals(KIND_CONTACT)) {
return domain
.getReferencedContacts()
.contains(VKey.from((Key<ContactResource>) resourceKey));
} else if (resourceKey.getKind().equals(KIND_HOST)) {
return domain.getNameservers().contains(VKey.from((Key<HostResource>) resourceKey));
} else {
throw new IllegalStateException("EPP resource key of unknown type: " + resourceKey);
}
}
}
/**
* A reducer that checks if the EPP resource to be deleted is referenced anywhere, and then
* deletes it if not and unmarks it for deletion if so.
*/
public static class DeleteEppResourceReducer
extends Reducer<DeletionRequest, Boolean, Void> {
private static final long serialVersionUID = 6569363449285506326L;
private static final DnsQueue dnsQueue = DnsQueue.create();
@NonFinalForTesting
private static AsyncTaskMetrics asyncTaskMetrics = new AsyncTaskMetrics(new SystemClock());
@Override
public void reduce(final DeletionRequest deletionRequest, ReducerInput<Boolean> values) {
final boolean hasNoActiveReferences = !Iterators.contains(values, true);
logger.atInfo().log("Processing async deletion request for %s.", deletionRequest.key());
DeletionResult result =
tm()
.transactNew(
() -> {
DeletionResult deletionResult =
attemptToDeleteResource(deletionRequest, hasNoActiveReferences);
getQueue(QUEUE_ASYNC_DELETE).deleteTask(deletionRequest.task());
return deletionResult;
});
asyncTaskMetrics.recordAsyncFlowResult(
deletionRequest.getMetricOperationType(),
result.getMetricOperationResult(),
deletionRequest.requestedTime());
String resourceNamePlural = deletionRequest.key().getKind() + "s";
getContext().incrementCounter(result.type().renderCounterText(resourceNamePlural));
logger.atInfo().log(
"Result of async deletion for resource %s: %s",
deletionRequest.key(), result.pollMessageText());
}
private DeletionResult attemptToDeleteResource(
DeletionRequest deletionRequest, boolean hasNoActiveReferences) {
DateTime now = tm().getTransactionTime();
EppResource resource =
auditedOfy().load().key(deletionRequest.key()).now().cloneProjectedAtTime(now);
// Double-check transactionally that the resource is still active and in PENDING_DELETE.
if (!doesResourceStateAllowDeletion(resource, now)) {
return DeletionResult.create(Type.ERRORED, "");
}
// Contacts and external hosts have a direct client id. For subordinate hosts it needs to be
// read off of the superordinate domain.
String resourceRegistrarId = resource.getPersistedCurrentSponsorRegistrarId();
if (resource instanceof HostResource && ((HostResource) resource).isSubordinate()) {
resourceRegistrarId =
tm().loadByKey(((HostResource) resource).getSuperordinateDomain())
.cloneProjectedAtTime(now)
.getCurrentSponsorRegistrarId();
}
boolean requestedByCurrentOwner =
resourceRegistrarId.equals(deletionRequest.requestingClientId());
boolean deleteAllowed =
hasNoActiveReferences && (requestedByCurrentOwner || deletionRequest.isSuperuser());
String resourceTypeName =
resource.getClass().getAnnotation(ExternalMessagingName.class).value();
String pollMessageText =
deleteAllowed
? String.format("Deleted %s %s.", resourceTypeName, resource.getForeignKey())
: String.format(
"Can't delete %s %s because %s.",
resourceTypeName,
resource.getForeignKey(),
requestedByCurrentOwner
? "it is referenced by a domain"
: "it was transferred prior to deletion");
HistoryEntry historyEntry =
HistoryEntry.createBuilderForResource(resource)
.setRegistrarId(deletionRequest.requestingClientId())
.setModificationTime(now)
.setType(getHistoryEntryType(resource, deleteAllowed))
.build();
PollMessage.OneTime pollMessage =
new PollMessage.OneTime.Builder()
.setRegistrarId(deletionRequest.requestingClientId())
.setMsg(pollMessageText)
.setParent(historyEntry)
.setEventTime(now)
.setResponseData(
getPollMessageResponseData(deletionRequest, resource, deleteAllowed, now))
.build();
EppResource resourceToSave;
if (deleteAllowed) {
EppResource.Builder<?, ?> resourceToSaveBuilder;
if (resource instanceof ContactResource) {
ContactResource contact = (ContactResource) resource;
// Handle pending transfers on contact deletion.
if (contact.getStatusValues().contains(StatusValue.PENDING_TRANSFER)) {
contact =
denyPendingTransfer(
contact, SERVER_CANCELLED, now, deletionRequest.requestingClientId());
}
// Wipe out PII on contact deletion.
resourceToSaveBuilder = contact.asBuilder().wipeOut();
} else {
resourceToSaveBuilder = resource.asBuilder();
}
resourceToSave = resourceToSaveBuilder.setDeletionTime(now).setStatusValues(null).build();
performDeleteTasks(resource, resourceToSave, now, historyEntry);
updateForeignKeyIndexDeletionTime(resourceToSave);
} else {
resourceToSave = resource.asBuilder().removeStatusValue(PENDING_DELETE).build();
}
auditedOfy()
.save()
.<ImmutableObject>entities(resourceToSave, historyEntry.asHistoryEntry(), pollMessage);
return DeletionResult.create(
deleteAllowed ? Type.DELETED : Type.NOT_DELETED, pollMessageText);
}
private static ImmutableList<? extends ResponseData> getPollMessageResponseData(
DeletionRequest deletionRequest,
EppResource resource,
boolean deleteAllowed,
DateTime now) {
@Nullable String clientTransactionId = deletionRequest.clientTransactionId();
String serverTransactionId = deletionRequest.serverTransactionId();
Trid trid = Trid.create(clientTransactionId, serverTransactionId);
if (resource instanceof HostResource) {
return ImmutableList.of(
HostPendingActionNotificationResponse.create(
((HostResource) resource).getHostName(), deleteAllowed, trid, now));
} else if (resource instanceof ContactResource) {
return ImmutableList.of(
ContactPendingActionNotificationResponse.create(
((ContactResource) resource).getContactId(), deleteAllowed, trid, now));
} else {
throw new IllegalStateException("EPP resource of unknown type " + Key.create(resource));
}
}
/**
* Determine the proper history entry type for the delete operation, as a function of
* whether or not the delete was successful.
*/
private HistoryEntry.Type getHistoryEntryType(EppResource resource, boolean successfulDelete) {
if (resource instanceof ContactResource) {
return successfulDelete ? CONTACT_DELETE : CONTACT_DELETE_FAILURE;
} else if (resource instanceof HostResource) {
return successfulDelete ? HOST_DELETE : HOST_DELETE_FAILURE;
} else {
throw new IllegalStateException("EPP resource of unknown type: " + Key.create(resource));
}
}
/** Perform any type-specific tasks on the resource to be deleted (and/or its dependencies). */
private void performDeleteTasks(
EppResource existingResource,
EppResource deletedResource,
DateTime deletionTime,
HistoryEntry historyEntryForDelete) {
if (existingResource instanceof ContactResource) {
handlePendingTransferOnDelete(
(ContactResource) existingResource,
(ContactResource) deletedResource,
deletionTime,
historyEntryForDelete);
} else if (existingResource instanceof HostResource) {
HostResource host = (HostResource) existingResource;
if (host.isSubordinate()) {
dnsQueue.addHostRefreshTask(host.getHostName());
tm().put(
tm().loadByKey(host.getSuperordinateDomain())
.asBuilder()
.removeSubordinateHost(host.getHostName())
.build());
}
} else {
throw new IllegalStateException(
"EPP resource of unknown type: " + Key.create(existingResource));
}
}
}
/** A class that encapsulates the values of a request to delete a contact or host. */
@AutoValue
abstract static class DeletionRequest implements Serializable {
private static final long serialVersionUID = -4612618525760839240L;
abstract Key<? extends EppResource> key();
abstract DateTime lastUpdateTime();
/**
* The client id of the registrar that requested this deletion (which might NOT be the same as
* the actual current owner of the resource).
*/
abstract String requestingClientId();
/** First half of TRID for the original request, split for serializability. */
@Nullable
abstract String clientTransactionId();
/** Second half of TRID for the original request, split for serializability. */
abstract String serverTransactionId();
abstract boolean isSuperuser();
abstract DateTime requestedTime();
abstract boolean isDeletionAllowed();
abstract TaskHandle task();
@AutoValue.Builder
abstract static class Builder {
abstract Builder setKey(Key<? extends EppResource> key);
abstract Builder setLastUpdateTime(DateTime lastUpdateTime);
abstract Builder setRequestingClientId(String requestingClientId);
abstract Builder setClientTransactionId(@Nullable String clientTransactionId);
abstract Builder setServerTransactionId(String serverTransactionId);
abstract Builder setIsSuperuser(boolean isSuperuser);
abstract Builder setRequestedTime(DateTime requestedTime);
abstract Builder setIsDeletionAllowed(boolean isDeletionAllowed);
abstract Builder setTask(TaskHandle task);
abstract DeletionRequest build();
}
static DeletionRequest createFromTask(TaskHandle task, DateTime now)
throws Exception {
ImmutableMap<String, String> params = ImmutableMap.copyOf(task.extractParams());
VKey<EppResource> resourceKey =
VKey.create(
checkNotNull(params.get(PARAM_RESOURCE_KEY), "Resource to delete not specified"));
EppResource resource =
checkNotNull(
auditedOfy().load().key(resourceKey.getOfyKey()).now(),
"Resource to delete doesn't exist");
checkState(
resource instanceof ContactResource || resource instanceof HostResource,
"Cannot delete a %s via this action",
resource.getClass().getSimpleName());
return new AutoValue_DeleteContactsAndHostsAction_DeletionRequest.Builder()
.setKey(resourceKey.getOfyKey())
.setLastUpdateTime(resource.getUpdateTimestamp().getTimestamp())
.setRequestingClientId(
checkNotNull(
params.get(PARAM_REQUESTING_CLIENT_ID), "Requesting client id not specified"))
// Note that client transaction ID is optional, in which case this sets it to null.
.setClientTransactionId(params.get(PARAM_CLIENT_TRANSACTION_ID))
.setServerTransactionId(
checkNotNull(
params.get(PARAM_SERVER_TRANSACTION_ID), "Server transaction id not specified"))
.setIsSuperuser(
Boolean.parseBoolean(
checkNotNull(params.get(PARAM_IS_SUPERUSER), "Is superuser not specified")))
.setRequestedTime(
DateTime.parse(
checkNotNull(params.get(PARAM_REQUESTED_TIME), "Requested time not specified")))
.setIsDeletionAllowed(doesResourceStateAllowDeletion(resource, now))
.setTask(task)
.build();
}
OperationType getMetricOperationType() {
if (key().getKind().equals(KIND_CONTACT)) {
return OperationType.CONTACT_DELETE;
} else if (key().getKind().equals(KIND_HOST)) {
return OperationType.HOST_DELETE;
} else {
throw new IllegalStateException(
String.format(
"Cannot determine async operation type for metric for resource %s", key()));
}
}
}
/** A class that encapsulates the values resulting from attempted contact/host deletion. */
@AutoValue
abstract static class DeletionResult {
enum Type {
DELETED("%s deleted", OperationResult.SUCCESS),
NOT_DELETED("%s not deleted", OperationResult.FAILURE),
ERRORED("%s errored out during deletion", OperationResult.ERROR);
private final String counterFormat;
private final OperationResult operationResult;
Type(String counterFormat, OperationResult operationResult) {
this.counterFormat = counterFormat;
this.operationResult = operationResult;
}
String renderCounterText(String resourceName) {
return String.format(counterFormat, resourceName);
}
}
abstract Type type();
abstract String pollMessageText();
static DeletionResult create(Type type, String pollMessageText) {
return new AutoValue_DeleteContactsAndHostsAction_DeletionResult(type, pollMessageText);
}
OperationResult getMetricOperationResult() {
return type().operationResult;
}
}
static boolean doesResourceStateAllowDeletion(EppResource resource, DateTime now) {
Key<EppResource> key = Key.create(resource);
if (isDeleted(resource, now)) {
logger.atWarning().log("Cannot asynchronously delete %s because it is already deleted.", key);
return false;
}
if (!resource.getStatusValues().contains(PENDING_DELETE)) {
logger.atWarning().log(
"Cannot asynchronously delete %s because it is not in PENDING_DELETE.", key);
return false;
}
return true;
}
}

View File

@@ -18,7 +18,6 @@ import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.net.MediaType.PLAIN_TEXT_UTF_8;
import static google.registry.flows.FlowUtils.marshalWithLenientRetry;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.persistence.transaction.TransactionManagerUtil.transactIfJpaTm;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static google.registry.util.ResourceUtils.readResourceUtf8;
import static java.nio.charset.StandardCharsets.UTF_8;
@@ -129,15 +128,13 @@ public class DeleteExpiredDomainsAction implements Runnable {
logger.atInfo().log(
"Deleting non-renewing domains with autorenew end times up through %s.", runTime);
// Note: in Datastore, this query is (and must be) non-transactional, and thus, is only
// eventually consistent.
ImmutableList<DomainBase> domainsToDelete =
transactIfJpaTm(
() ->
tm().createQueryComposer(DomainBase.class)
.where("autorenewEndTime", Comparator.LTE, runTime)
.where("deletionTime", Comparator.EQ, END_OF_TIME)
.list());
tm().transact(
() ->
tm().createQueryComposer(DomainBase.class)
.where("autorenewEndTime", Comparator.LTE, runTime)
.where("deletionTime", Comparator.EQ, END_OF_TIME)
.list());
if (domainsToDelete.isEmpty()) {
logger.atInfo().log("Found 0 domains to delete.");
response.setPayload("Found 0 domains to delete.");

View File

@@ -16,39 +16,30 @@ package google.registry.batch;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.batch.BatchModule.PARAM_DRY_RUN;
import static google.registry.config.RegistryEnvironment.PRODUCTION;
import static google.registry.mapreduce.MapreduceRunner.PARAM_DRY_RUN;
import static google.registry.mapreduce.inputs.EppResourceInputs.createEntityInput;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.appengine.tools.mapreduce.Mapper;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import com.googlecode.objectify.Key;
import google.registry.config.RegistryEnvironment;
import google.registry.flows.poll.PollFlowUtils;
import google.registry.mapreduce.MapreduceRunner;
import google.registry.model.EppResource;
import google.registry.model.EppResourceUtils;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.DomainBase;
import google.registry.model.host.HostResource;
import google.registry.model.index.EppResourceIndex;
import google.registry.model.index.ForeignKeyIndex;
import google.registry.model.poll.PollMessage;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.reporting.HistoryEntryDao;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import java.util.List;
import javax.inject.Inject;
/**
@@ -57,8 +48,8 @@ import javax.inject.Inject;
*
* <p>This only deletes contacts and hosts, NOT domains. To delete domains, use {@link
* DeleteProberDataAction} and pass it the TLD(s) that the load test domains were created on. Note
* that DeleteProberDataAction is safe enough to run in production whereas this mapreduce is not,
* but this one does not need to be runnable in production because load testing isn't run against
* that DeleteProberDataAction is safe enough to run in production whereas this action is not, but
* this one does not need to be runnable in production because load testing isn't run against
* production.
*/
@Action(
@@ -79,48 +70,31 @@ public class DeleteLoadTestDataAction implements Runnable {
private static final ImmutableSet<String> LOAD_TEST_REGISTRARS = ImmutableSet.of("proxy");
private final boolean isDryRun;
private final MapreduceRunner mrRunner;
private final Response response;
private final Clock clock;
@Inject
DeleteLoadTestDataAction(
@Parameter(PARAM_DRY_RUN) boolean isDryRun,
MapreduceRunner mrRunner,
Response response,
Clock clock) {
this.isDryRun = isDryRun;
this.mrRunner = mrRunner;
this.response = response;
this.clock = clock;
}
@Override
public void run() {
// This mapreduce doesn't guarantee that foreign key relations are preserved, so isn't safe to
// This action doesn't guarantee that foreign key relations are preserved, so isn't safe to
// run on production. On other environments, data is fully wiped out occasionally anyway, so
// having some broken data that isn't referred to isn't the end of the world.
checkState(
!RegistryEnvironment.get().equals(PRODUCTION),
"This mapreduce is not safe to run on PRODUCTION.");
"This action is not safe to run on PRODUCTION.");
if (tm().isOfy()) {
mrRunner
.setJobName("Delete load test data")
.setModuleName("backend")
.runMapOnly(
new DeleteLoadTestDataMapper(isDryRun),
ImmutableList.of(
createEntityInput(ContactResource.class), createEntityInput(HostResource.class)))
.sendLinkToMapreduceConsole(response);
} else {
tm().transact(
() -> {
LOAD_TEST_REGISTRARS.forEach(this::deletePollMessages);
tm().loadAllOfStream(ContactResource.class).forEach(this::deleteContact);
tm().loadAllOfStream(HostResource.class).forEach(this::deleteHost);
});
}
tm().transact(
() -> {
LOAD_TEST_REGISTRARS.forEach(this::deletePollMessages);
tm().loadAllOfStream(ContactResource.class).forEach(this::deleteContact);
tm().loadAllOfStream(HostResource.class).forEach(this::deleteHost);
});
}
private void deletePollMessages(String registrarId) {
@@ -184,54 +158,4 @@ public class DeleteLoadTestDataAction implements Runnable {
tm().delete(eppResource);
}
}
/** Provides the map method that runs for each existing contact and host entity. */
public static class DeleteLoadTestDataMapper extends Mapper<EppResource, Void, Void> {
private static final long serialVersionUID = -3817710674062432694L;
private final boolean isDryRun;
public DeleteLoadTestDataMapper(boolean isDryRun) {
this.isDryRun = isDryRun;
}
@Override
public final void map(EppResource resource) {
if (LOAD_TEST_REGISTRARS.contains(resource.getPersistedCurrentSponsorRegistrarId())) {
deleteResource(resource);
getContext()
.incrementCounter(
String.format("deleted %s entities", resource.getClass().getSimpleName()));
} else {
getContext().incrementCounter("skipped, not load test data");
}
}
private void deleteResource(EppResource resource) {
final Key<EppResourceIndex> eppIndex =
Key.create(EppResourceIndex.create(Key.create(resource)));
final Key<? extends ForeignKeyIndex<?>> fki = ForeignKeyIndex.createKey(resource);
int numEntitiesDeleted =
tm().transact(
() -> {
// This ancestor query selects all descendant entities.
List<Key<Object>> resourceAndDependentKeys =
auditedOfy().load().ancestor(resource).keys().list();
ImmutableSet<Key<?>> allKeys =
new ImmutableSet.Builder<Key<?>>()
.add(fki)
.add(eppIndex)
.addAll(resourceAndDependentKeys)
.build();
if (isDryRun) {
logger.atInfo().log("Would hard-delete the following entities: %s", allKeys);
} else {
auditedOfy().deleteWithoutBackup().keys(allKeys);
}
return allKeys.size();
});
getContext().incrementCounter("total entities deleted", numEntitiesDeleted);
}
}
}

View File

@@ -17,45 +17,32 @@ package google.registry.batch;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.batch.BatchModule.PARAM_DRY_RUN;
import static google.registry.config.RegistryEnvironment.PRODUCTION;
import static google.registry.mapreduce.MapreduceRunner.PARAM_DRY_RUN;
import static google.registry.model.ResourceTransferUtils.updateForeignKeyIndexDeletionTime;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_DELETE;
import static google.registry.model.tld.Registries.getTldsOfType;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static google.registry.request.RequestParameters.PARAM_TLDS;
import static org.joda.time.DateTimeZone.UTC;
import com.google.appengine.tools.mapreduce.Mapper;
import com.google.common.base.Splitter;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Iterables;
import com.google.common.collect.Sets;
import com.google.common.flogger.FluentLogger;
import com.googlecode.objectify.Key;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryEnvironment;
import google.registry.dns.DnsQueue;
import google.registry.mapreduce.MapreduceRunner;
import google.registry.mapreduce.inputs.EppResourceInputs;
import google.registry.model.CreateAutoTimestamp;
import google.registry.model.EppResourceUtils;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.index.EppResourceIndex;
import google.registry.model.index.ForeignKeyIndex;
import google.registry.model.tld.Registry;
import google.registry.model.tld.Registry.TldType;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;
import javax.inject.Inject;
import org.hibernate.CacheMode;
@@ -94,8 +81,6 @@ public class DeleteProberDataAction implements Runnable {
*/
private static final Duration SOFT_DELETE_DELAY = Duration.standardHours(1);
private static final DnsQueue dnsQueue = DnsQueue.create();
// Domains to delete must:
// 1. Be in one of the prober TLDs
// 2. Not be a nic domain
@@ -115,6 +100,8 @@ public class DeleteProberDataAction implements Runnable {
/** Number of domains to retrieve and delete per SQL transaction. */
private static final int BATCH_SIZE = 1000;
@Inject DnsQueue dnsQueue;
@Inject @Parameter(PARAM_DRY_RUN) boolean isDryRun;
/** List of TLDs to work on. If empty - will work on all TLDs that end with .test. */
@Inject @Parameter(PARAM_TLDS) ImmutableSet<String> tlds;
@@ -123,8 +110,6 @@ public class DeleteProberDataAction implements Runnable {
@Config("registryAdminClientId")
String registryAdminRegistrarId;
@Inject MapreduceRunner mrRunner;
@Inject Response response;
@Inject DeleteProberDataAction() {}
@Override
@@ -145,21 +130,7 @@ public class DeleteProberDataAction implements Runnable {
"If tlds are given, they must all exist and be TEST tlds. Given: %s, not found: %s",
tlds,
Sets.difference(tlds, deletableTlds));
ImmutableSet<String> proberRoidSuffixes =
deletableTlds.stream()
.map(tld -> Registry.get(tld).getRoidSuffix())
.collect(toImmutableSet());
if (tm().isOfy()) {
mrRunner
.setJobName("Delete prober data")
.setModuleName("backend")
.runMapOnly(
new DeleteProberDataMapper(proberRoidSuffixes, isDryRun, registryAdminRegistrarId),
ImmutableList.of(EppResourceInputs.createKeyInput(DomainBase.class)))
.sendLinkToMapreduceConsole(response);
} else {
runSqlJob(deletableTlds);
}
runSqlJob(deletableTlds);
}
private void runSqlJob(ImmutableSet<String> deletableTlds) {
@@ -231,7 +202,7 @@ public class DeleteProberDataAction implements Runnable {
"Would soft-delete the active domain: %s (%s).",
domain.getDomainName(), domain.getRepoId());
} else {
softDeleteDomain(domain, registryAdminRegistrarId, dnsQueue);
softDeleteDomain(domain);
}
softDeletedDomains.incrementAndGet();
} else {
@@ -280,8 +251,7 @@ public class DeleteProberDataAction implements Runnable {
}
// Take a DNS queue + admin registrar id as input so that it can be called from the mapper as well
private static void softDeleteDomain(
DomainBase domain, String registryAdminRegistrarId, DnsQueue localDnsQueue) {
private void softDeleteDomain(DomainBase domain) {
DomainBase deletedDomain =
domain.asBuilder().setDeletionTime(tm().getTransactionTime()).setStatusValues(null).build();
DomainHistory historyEntry =
@@ -299,119 +269,6 @@ public class DeleteProberDataAction implements Runnable {
tm().putAllWithoutBackup(ImmutableList.of(deletedDomain, historyEntry));
// updating foreign keys is a no-op in SQL
updateForeignKeyIndexDeletionTime(deletedDomain);
localDnsQueue.addDomainRefreshTask(deletedDomain.getDomainName());
}
/** Provides the map method that runs for each existing DomainBase entity. */
public static class DeleteProberDataMapper extends Mapper<Key<DomainBase>, Void, Void> {
private static final DnsQueue dnsQueue = DnsQueue.create();
private static final long serialVersionUID = -7724537393697576369L;
private final ImmutableSet<String> proberRoidSuffixes;
private final Boolean isDryRun;
private final String registryAdminRegistrarId;
public DeleteProberDataMapper(
ImmutableSet<String> proberRoidSuffixes,
Boolean isDryRun,
String registryAdminRegistrarId) {
this.proberRoidSuffixes = proberRoidSuffixes;
this.isDryRun = isDryRun;
this.registryAdminRegistrarId = registryAdminRegistrarId;
}
@Override
public final void map(Key<DomainBase> key) {
try {
String roidSuffix = Iterables.getLast(Splitter.on('-').split(key.getName()));
if (proberRoidSuffixes.contains(roidSuffix)) {
deleteDomain(key);
} else {
getContext().incrementCounter("skipped, non-prober data");
}
} catch (Throwable t) {
logger.atSevere().withCause(t).log("Error while deleting prober data for key %s.", key);
getContext().incrementCounter(String.format("error, kind %s", key.getKind()));
}
}
private void deleteDomain(final Key<DomainBase> domainKey) {
final DomainBase domain = auditedOfy().load().key(domainKey).now();
DateTime now = DateTime.now(UTC);
if (domain == null) {
// Depending on how stale Datastore indexes are, we can get keys to resources that are
// already deleted (e.g. by a recent previous invocation of this mapreduce). So ignore them.
getContext().incrementCounter("already deleted");
return;
}
String domainName = domain.getDomainName();
if (domainName.equals("nic." + domain.getTld())) {
getContext().incrementCounter("skipped, NIC domain");
return;
}
if (now.isBefore(domain.getCreationTime().plus(DOMAIN_USED_DURATION))) {
getContext().incrementCounter("skipped, domain too new");
return;
}
if (!domain.getSubordinateHosts().isEmpty()) {
logger.atWarning().log(
"Cannot delete domain %s (%s) because it has subordinate hosts.",
domainName, domainKey);
getContext().incrementCounter("skipped, had subordinate host(s)");
return;
}
// If the domain is still active, that means that the prober encountered a failure and did not
// successfully soft-delete the domain (thus leaving its DNS entry published). We soft-delete
// it now so that the DNS entry can be handled. The domain will then be hard-deleted the next
// time the mapreduce is run.
if (EppResourceUtils.isActive(domain, now)) {
if (isDryRun) {
logger.atInfo().log(
"Would soft-delete the active domain: %s (%s).", domainName, domainKey);
} else {
tm().transact(() -> softDeleteDomain(domain, registryAdminRegistrarId, dnsQueue));
}
getContext().incrementCounter("domains soft-deleted");
return;
}
// If the domain isn't active, we want to make sure it hasn't been active for "a while" before
// deleting it. This prevents accidental double-map with the same key from immediately
// deleting active domains
if (now.isBefore(domain.getDeletionTime().plus(SOFT_DELETE_DELAY))) {
getContext().incrementCounter("skipped, domain too recently soft deleted");
return;
}
final Key<EppResourceIndex> eppIndex = Key.create(EppResourceIndex.create(domainKey));
final Key<? extends ForeignKeyIndex<?>> fki = ForeignKeyIndex.createKey(domain);
int entitiesDeleted =
tm().transact(
() -> {
// This ancestor query selects all descendant HistoryEntries, BillingEvents,
// PollMessages, and TLD-specific entities, as well as the domain itself.
List<Key<Object>> domainAndDependentKeys =
auditedOfy().load().ancestor(domainKey).keys().list();
ImmutableSet<Key<?>> allKeys =
new ImmutableSet.Builder<Key<?>>()
.add(fki)
.add(eppIndex)
.addAll(domainAndDependentKeys)
.build();
if (isDryRun) {
logger.atInfo().log("Would hard-delete the following entities: %s", allKeys);
} else {
auditedOfy().deleteWithoutBackup().keys(allKeys);
}
return allKeys.size();
});
getContext().incrementCounter("domains hard-deleted");
getContext().incrementCounter("total entities hard-deleted", entitiesDeleted);
}
dnsQueue.addDomainRefreshTask(deletedDomain.getDomainName());
}
}

View File

@@ -17,32 +17,26 @@ package google.registry.batch;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.collect.Sets.difference;
import static google.registry.mapreduce.MapreduceRunner.PARAM_DRY_RUN;
import static google.registry.mapreduce.inputs.EppResourceInputs.createChildEntityInput;
import static com.google.common.collect.Sets.newHashSet;
import static google.registry.batch.BatchModule.PARAM_DRY_RUN;
import static google.registry.model.common.Cursor.CursorType.RECURRING_BILLING;
import static google.registry.model.domain.Period.Unit.YEARS;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_AUTORENEW;
import static google.registry.persistence.transaction.QueryComposer.Comparator.EQ;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.persistence.transaction.TransactionManagerUtil.transactIfJpaTm;
import static google.registry.pricing.PricingEngineProxy.getDomainRenewCost;
import static google.registry.util.CollectionUtils.union;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static google.registry.util.DateTimeUtils.earliestOf;
import static google.registry.util.DomainNameUtils.getTldFromDomainName;
import com.google.appengine.tools.mapreduce.Mapper;
import com.google.appengine.tools.mapreduce.Reducer;
import com.google.appengine.tools.mapreduce.ReducerInput;
import com.google.common.collect.ImmutableList;
import com.google.auto.value.AutoValue;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Range;
import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import google.registry.mapreduce.MapreduceRunner;
import google.registry.mapreduce.inputs.NullInput;
import google.registry.config.RegistryConfig.Config;
import google.registry.flows.domain.DomainPricingLogic;
import google.registry.model.ImmutableObject;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingEvent.Flag;
@@ -61,16 +55,17 @@ import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import java.util.List;
import java.util.Optional;
import java.util.Set;
import java.util.concurrent.TimeUnit;
import javax.inject.Inject;
import org.joda.money.Money;
import org.joda.time.DateTime;
/**
* A mapreduce that expands {@link Recurring} billing events into synthetic {@link OneTime} events.
* An action that expands {@link Recurring} billing events into synthetic {@link OneTime} events.
*
* <p>The cursor used throughout this mapreduce (overridden if necessary using the parameter {@code
* <p>The cursor used throughout this action (overridden if necessary using the parameter {@code
* cursorTime}) represents the inclusive lower bound on the range of billing times that will be
* expanded as a result of the job (the exclusive upper bound being the execution time of the job).
*/
@@ -81,13 +76,18 @@ import org.joda.time.DateTime;
public class ExpandRecurringBillingEventsAction implements Runnable {
public static final String PARAM_CURSOR_TIME = "cursorTime";
private static final String ERROR_COUNTER = "errors";
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
@Inject Clock clock;
@Inject MapreduceRunner mrRunner;
@Inject
@Config("jdbcBatchSize")
int batchSize;
@Inject @Parameter(PARAM_DRY_RUN) boolean isDryRun;
@Inject @Parameter(PARAM_CURSOR_TIME) Optional<DateTime> cursorTimeParam;
@Inject DomainPricingLogic domainPricingLogic;
@Inject Response response;
@Inject ExpandRecurringBillingEventsAction() {}
@@ -95,198 +95,152 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
public void run() {
DateTime executeTime = clock.nowUtc();
DateTime persistedCursorTime =
transactIfJpaTm(
() ->
tm().loadByKeyIfPresent(Cursor.createGlobalVKey(RECURRING_BILLING))
.orElse(Cursor.createGlobal(RECURRING_BILLING, START_OF_TIME))
.getCursorTime());
tm().transact(
() ->
tm().loadByKeyIfPresent(Cursor.createGlobalVKey(RECURRING_BILLING))
.orElse(Cursor.createGlobal(RECURRING_BILLING, START_OF_TIME))
.getCursorTime());
DateTime cursorTime = cursorTimeParam.orElse(persistedCursorTime);
checkArgument(
cursorTime.isBefore(executeTime), "Cursor time must be earlier than execution time.");
logger.atInfo().log(
"Running Recurring billing event expansion for billing time range [%s, %s).",
cursorTime, executeTime);
if (tm().isOfy()) {
mrRunner
.setJobName("Expand Recurring billing events into synthetic OneTime events.")
.setModuleName("backend")
.runMapreduce(
new ExpandRecurringBillingEventsMapper(isDryRun, cursorTime, clock.nowUtc()),
new ExpandRecurringBillingEventsReducer(isDryRun, persistedCursorTime),
// Add an extra shard that maps over a null recurring event (see the mapper for why).
ImmutableList.of(
new NullInput<>(),
createChildEntityInput(
ImmutableSet.of(DomainBase.class), ImmutableSet.of(Recurring.class))))
.sendLinkToMapreduceConsole(response);
} else {
int numBillingEventsSaved =
expandSqlBillingEventsInBatches(executeTime, cursorTime, persistedCursorTime);
}
private void expandSqlBillingEventsInBatches(
DateTime executeTime, DateTime cursorTime, DateTime persistedCursorTime) {
int totalBillingEventsSaved = 0;
long maxProcessedRecurrenceId = 0;
SqlBatchResults sqlBatchResults;
do {
final long prevMaxProcessedRecurrenceId = maxProcessedRecurrenceId;
sqlBatchResults =
jpaTm()
.transact(
() ->
jpaTm()
.query(
"FROM BillingRecurrence "
+ "WHERE eventTime <= :executeTime "
+ "AND eventTime < recurrenceEndTime "
+ "ORDER BY id ASC",
Recurring.class)
.setParameter("executeTime", executeTime)
// Need to get a list from the transaction and then convert it to a stream
// for further processing. If we get a stream directly, each elements gets
// processed downstream eagerly but Hibernate returns a
// ScrollableResultsIterator that cannot be advanced outside the
// transaction, resulting in an exception.
.getResultList())
.stream()
.map(
recurring ->
jpaTm()
.transact(
() ->
expandBillingEvent(recurring, executeTime, cursorTime, isDryRun)))
.reduce(0, Integer::sum);
if (!isDryRun) {
logger.atInfo().log("Saved OneTime billing events.", numBillingEventsSaved);
() -> {
Set<String> expandedDomains = newHashSet();
int batchBillingEventsSaved = 0;
long maxRecurrenceId = prevMaxProcessedRecurrenceId;
List<Recurring> recurrings =
jpaTm()
.query(
"FROM BillingRecurrence "
+ "WHERE eventTime <= :executeTime "
+ "AND eventTime < recurrenceEndTime "
+ "AND id > :maxProcessedRecurrenceId "
+ "AND recurrenceEndTime > :cursorTime "
+ "ORDER BY id ASC",
Recurring.class)
.setParameter("executeTime", executeTime)
.setParameter("maxProcessedRecurrenceId", prevMaxProcessedRecurrenceId)
.setParameter("cursorTime", cursorTime)
.setMaxResults(batchSize)
.getResultList();
for (Recurring recurring : recurrings) {
if (expandedDomains.contains(recurring.getTargetId())) {
// On the off chance this batch contains multiple recurrences for the same
// domain (which is actually possible if a given domain is quickly renewed
// multiple times in a row), then short-circuit after the first one is
// processed that involves actually expanding a billing event. This is
// necessary because otherwise we get an "Inserted/updated object reloaded"
// error from Hibernate when those billing events would be loaded
// inside a transaction where they were already written. Note, there is no
// actual further work to be done in this case anyway, not unless it has
// somehow been over a year since this action last ran successfully (and if
// that were somehow true, the remaining billing events would still be
// expanded on subsequent runs).
continue;
}
int billingEventsSaved =
expandBillingEvent(
recurring, executeTime, cursorTime, isDryRun, domainPricingLogic);
batchBillingEventsSaved += billingEventsSaved;
if (billingEventsSaved > 0) {
expandedDomains.add(recurring.getTargetId());
}
maxRecurrenceId = Math.max(maxRecurrenceId, recurring.getId());
}
return SqlBatchResults.create(
batchBillingEventsSaved,
maxRecurrenceId,
maxRecurrenceId > prevMaxProcessedRecurrenceId);
});
totalBillingEventsSaved += sqlBatchResults.batchBillingEventsSaved();
maxProcessedRecurrenceId = sqlBatchResults.maxProcessedRecurrenceId();
if (sqlBatchResults.batchBillingEventsSaved() > 0) {
logger.atInfo().log(
"Saved %d billing events in batch (%d total) with max recurrence id %d.",
sqlBatchResults.batchBillingEventsSaved(),
totalBillingEventsSaved,
maxProcessedRecurrenceId);
} else {
logger.atInfo().log("Generated OneTime billing events (dry run).", numBillingEventsSaved);
// If we're churning through a lot of no-op recurrences that don't need expanding (yet?),
// then only log one no-op every so often as a good balance between letting the user track
// that the action is still running while also not spamming the logs incessantly.
logger.atInfo().atMostEvery(3, TimeUnit.MINUTES).log(
"Processed up to max recurrence id %d (no billing events saved recently).",
maxProcessedRecurrenceId);
}
} while (sqlBatchResults.shouldContinue());
if (!isDryRun) {
logger.atInfo().log("Saved %d total OneTime billing events.", totalBillingEventsSaved);
} else {
logger.atInfo().log(
"Recurring event expansion %s complete for billing event range [%s, %s).",
isDryRun ? "(dry run) " : "", cursorTime, executeTime);
tm().transact(
() -> {
// Check for the unlikely scenario where the cursor has been altered during the
// expansion.
DateTime currentCursorTime =
tm().loadByKeyIfPresent(Cursor.createGlobalVKey(RECURRING_BILLING))
.orElse(Cursor.createGlobal(RECURRING_BILLING, START_OF_TIME))
.getCursorTime();
if (!currentCursorTime.equals(persistedCursorTime)) {
throw new IllegalStateException(
String.format(
"Current cursor position %s does not match persisted cursor position %s.",
currentCursorTime, persistedCursorTime));
}
if (!isDryRun) {
tm().put(Cursor.createGlobal(RECURRING_BILLING, executeTime));
}
});
}
}
/** Mapper to expand {@link Recurring} billing events into synthetic {@link OneTime} events. */
public static class ExpandRecurringBillingEventsMapper
extends Mapper<Recurring, DateTime, DateTime> {
private static final long serialVersionUID = 8376442755556228455L;
private final boolean isDryRun;
private final DateTime cursorTime;
private final DateTime executeTime;
public ExpandRecurringBillingEventsMapper(
boolean isDryRun, DateTime cursorTime, DateTime executeTime) {
this.isDryRun = isDryRun;
this.cursorTime = cursorTime;
this.executeTime = executeTime;
}
@Override
public final void map(final Recurring recurring) {
// This single emit forces the reducer to run at the end of the map job, so that a mapper
// that runs without error will advance the cursor at the end of processing (unless this was
// a dry run, in which case the cursor should not be advanced).
if (recurring == null) {
emit(cursorTime, executeTime);
return;
}
getContext().incrementCounter("Recurring billing events encountered");
// Ignore any recurring billing events that have yet to apply.
if (recurring.getEventTime().isAfter(executeTime)
// This second case occurs when a domain is transferred or deleted before first renewal.
|| recurring.getRecurrenceEndTime().isBefore(recurring.getEventTime())) {
getContext().incrementCounter("Recurring billing events ignored");
return;
}
int numBillingEventsSaved = 0;
try {
numBillingEventsSaved =
tm().transactNew(
() -> expandBillingEvent(recurring, executeTime, cursorTime, isDryRun));
} catch (Throwable t) {
getContext().incrementCounter("error: " + t.getClass().getSimpleName());
getContext().incrementCounter(ERROR_COUNTER);
throw new RuntimeException(
String.format(
"Error while expanding Recurring billing events for %d", recurring.getId()),
t);
}
if (!isDryRun) {
getContext().incrementCounter("Saved OneTime billing events", numBillingEventsSaved);
} else {
getContext()
.incrementCounter("Generated OneTime billing events (dry run)", numBillingEventsSaved);
}
"Generated %d total OneTime billing events (dry run).", totalBillingEventsSaved);
}
logger.atInfo().log(
"Recurring event expansion %s complete for billing event range [%s, %s).",
isDryRun ? "(dry run) " : "", cursorTime, executeTime);
tm().transact(
() -> {
// Check for the unlikely scenario where the cursor has been altered during the
// expansion.
DateTime currentCursorTime =
tm().loadByKeyIfPresent(Cursor.createGlobalVKey(RECURRING_BILLING))
.orElse(Cursor.createGlobal(RECURRING_BILLING, START_OF_TIME))
.getCursorTime();
if (!currentCursorTime.equals(persistedCursorTime)) {
throw new IllegalStateException(
String.format(
"Current cursor position %s does not match persisted cursor position %s.",
currentCursorTime, persistedCursorTime));
}
if (!isDryRun) {
tm().put(Cursor.createGlobal(RECURRING_BILLING, executeTime));
}
});
}
/**
* "Reducer" to advance the cursor after all map jobs have been completed. The NullInput into the
* mapper will cause the mapper to emit one timestamp pair (current cursor and execution time),
* and the cursor will be advanced (and the timestamps logged) at the end of a successful
* mapreduce.
*/
public static class ExpandRecurringBillingEventsReducer
extends Reducer<DateTime, DateTime, Void> {
@AutoValue
abstract static class SqlBatchResults {
abstract int batchBillingEventsSaved();
private final boolean isDryRun;
private final DateTime expectedPersistedCursorTime;
abstract long maxProcessedRecurrenceId();
public ExpandRecurringBillingEventsReducer(
boolean isDryRun, DateTime expectedPersistedCursorTime) {
this.isDryRun = isDryRun;
this.expectedPersistedCursorTime = expectedPersistedCursorTime;
}
abstract boolean shouldContinue();
@Override
public void reduce(final DateTime cursorTime, final ReducerInput<DateTime> executionTimeInput) {
if (getContext().getCounter(ERROR_COUNTER).getValue() > 0) {
logger.atSevere().log(
"One or more errors logged during recurring event expansion. Cursor will"
+ " not be advanced.");
return;
}
final DateTime executionTime = executionTimeInput.next();
logger.atInfo().log(
"Recurring event expansion %s complete for billing event range [%s, %s).",
isDryRun ? "(dry run) " : "", cursorTime, executionTime);
tm().transact(
() -> {
Cursor cursor =
auditedOfy().load().key(Cursor.createGlobalKey(RECURRING_BILLING)).now();
DateTime currentCursorTime =
(cursor == null ? START_OF_TIME : cursor.getCursorTime());
if (!currentCursorTime.equals(expectedPersistedCursorTime)) {
logger.atSevere().log(
"Current cursor position %s does not match expected cursor position %s.",
currentCursorTime, expectedPersistedCursorTime);
return;
}
if (!isDryRun) {
tm().put(Cursor.createGlobal(RECURRING_BILLING, executionTime));
}
});
static SqlBatchResults create(
int batchBillingEventsSaved, long maxProcessedRecurrenceId, boolean shouldContinue) {
return new AutoValue_ExpandRecurringBillingEventsAction_SqlBatchResults(
batchBillingEventsSaved, maxProcessedRecurrenceId, shouldContinue);
}
}
private static int expandBillingEvent(
Recurring recurring, DateTime executeTime, DateTime cursorTime, boolean isDryRun) {
Recurring recurring,
DateTime executeTime,
DateTime cursorTime,
boolean isDryRun,
DomainPricingLogic domainPricingLogic) {
ImmutableSet.Builder<OneTime> syntheticOneTimesBuilder = new ImmutableSet.Builder<>();
final Registry tld = Registry.get(getTldFromDomainName(recurring.getTargetId()));
// Determine the complete set of times at which this recurring event should
// occur (up to and including the runtime of the mapreduce).
// occur (up to and including the runtime of the action).
Iterable<DateTime> eventTimes =
recurring
.getRecurrenceTimeOfYear()
@@ -303,14 +257,10 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
VKey.create(
DomainBase.class, recurring.getDomainRepoId(), recurring.getParentKey().getParent());
Iterable<OneTime> oneTimesForDomain;
if (tm().isOfy()) {
oneTimesForDomain = auditedOfy().load().type(OneTime.class).ancestor(domainKey.getOfyKey());
} else {
oneTimesForDomain =
tm().createQueryComposer(OneTime.class)
.where("domainRepoId", EQ, recurring.getDomainRepoId())
.list();
}
oneTimesForDomain =
tm().createQueryComposer(OneTime.class)
.where("domainRepoId", EQ, recurring.getDomainRepoId())
.list();
// Determine the billing times that already have OneTime events persisted.
ImmutableSet<DateTime> existingBillingTimes =
@@ -349,13 +299,16 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
historyEntriesBuilder.add(historyEntry);
DateTime eventTime = billingTime.minus(tld.getAutoRenewGracePeriodLength());
// Determine the cost for a one-year renewal.
Money renewCost = getDomainRenewCost(recurring.getTargetId(), eventTime, 1);
syntheticOneTimesBuilder.add(
new OneTime.Builder()
.setBillingTime(billingTime)
.setRegistrarId(recurring.getRegistrarId())
.setCost(renewCost)
// Determine the cost for a one-year renewal.
.setCost(
domainPricingLogic
.getRenewPrice(tld, recurring.getTargetId(), eventTime, 1, recurring)
.getRenewCost())
.setEventTime(eventTime)
.setFlags(union(recurring.getFlags(), Flag.SYNTHETIC))
.setParent(historyEntry)
@@ -381,7 +334,7 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
/**
* Filters a set of {@link DateTime}s down to event times that are in scope for a particular
* mapreduce run, given the cursor time and the mapreduce execution time.
* action run, given the cursor time and the action execution time.
*/
protected static ImmutableSet<DateTime> getBillingTimesInScope(
Iterable<DateTime> eventTimes,

View File

@@ -1,376 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch;
import static com.google.appengine.api.taskqueue.QueueConstants.maxLeaseCount;
import static com.google.appengine.api.taskqueue.QueueFactory.getQueue;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_HOST_KEY;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_REQUESTED_TIME;
import static google.registry.batch.AsyncTaskEnqueuer.QUEUE_ASYNC_HOST_RENAME;
import static google.registry.batch.AsyncTaskMetrics.OperationType.DNS_REFRESH;
import static google.registry.mapreduce.inputs.EppResourceInputs.createEntityInput;
import static google.registry.model.EppResourceUtils.getLinkedDomainKeys;
import static google.registry.model.EppResourceUtils.isActive;
import static google.registry.model.EppResourceUtils.isDeleted;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.latestOf;
import static java.util.concurrent.TimeUnit.DAYS;
import static java.util.concurrent.TimeUnit.SECONDS;
import static java.util.logging.Level.INFO;
import static java.util.logging.Level.SEVERE;
import static org.joda.time.Duration.standardHours;
import com.google.appengine.api.taskqueue.LeaseOptions;
import com.google.appengine.api.taskqueue.Queue;
import com.google.appengine.api.taskqueue.TaskHandle;
import com.google.appengine.api.taskqueue.TransientFailureException;
import com.google.appengine.tools.mapreduce.Mapper;
import com.google.appengine.tools.mapreduce.Reducer;
import com.google.appengine.tools.mapreduce.ReducerInput;
import com.google.auto.value.AutoValue;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.flogger.FluentLogger;
import google.registry.batch.AsyncTaskMetrics.OperationResult;
import google.registry.dns.DnsQueue;
import google.registry.mapreduce.MapreduceRunner;
import google.registry.mapreduce.inputs.NullInput;
import google.registry.model.domain.DomainBase;
import google.registry.model.host.HostResource;
import google.registry.model.server.Lock;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import google.registry.util.NonFinalForTesting;
import google.registry.util.RequestStatusChecker;
import google.registry.util.Retrier;
import google.registry.util.SystemClock;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
import java.util.NoSuchElementException;
import java.util.Optional;
import java.util.logging.Level;
import javax.annotation.Nullable;
import javax.inject.Inject;
import javax.inject.Named;
import org.apache.http.HttpStatus;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/** Performs batched DNS refreshes for applicable domains following a host rename. */
@Action(
service = Action.Service.BACKEND,
path = "/_dr/task/refreshDnsOnHostRename",
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
public class RefreshDnsOnHostRenameAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final Duration LEASE_LENGTH = standardHours(4);
@Inject AsyncTaskMetrics asyncTaskMetrics;
@Inject Clock clock;
@Inject MapreduceRunner mrRunner;
@Inject @Named(QUEUE_ASYNC_HOST_RENAME) Queue pullQueue;
@Inject DnsQueue dnsQueue;
@Inject RequestStatusChecker requestStatusChecker;
@Inject Response response;
@Inject Retrier retrier;
@Inject RefreshDnsOnHostRenameAction() {}
@Override
public void run() {
// Check if the lock can be acquired, and if not, a previous run of this mapreduce is still
// executing, so return early.
Optional<Lock> lock =
Lock.acquire(
RefreshDnsOnHostRenameAction.class.getSimpleName(),
null,
LEASE_LENGTH,
requestStatusChecker,
false);
if (!lock.isPresent()) {
logRespondAndUnlock(INFO, "Can't acquire lock; aborting.", lock);
return;
}
// Lease the async tasks to process.
LeaseOptions options =
LeaseOptions.Builder.withCountLimit(maxLeaseCount())
.leasePeriod(LEASE_LENGTH.getStandardSeconds(), SECONDS);
List<TaskHandle> tasks = pullQueue.leaseTasks(options);
asyncTaskMetrics.recordDnsRefreshBatchSize(tasks.size());
// Check if there are no tasks to process, and if so, return early.
if (tasks.isEmpty()) {
logRespondAndUnlock(
INFO, "No DNS refresh on host rename tasks to process in pull queue; finishing.", lock);
return;
}
ImmutableList.Builder<DnsRefreshRequest> requestsBuilder = new ImmutableList.Builder<>();
ImmutableList.Builder<VKey<HostResource>> hostKeys = new ImmutableList.Builder<>();
final List<DnsRefreshRequest> requestsToDelete = new ArrayList<>();
for (TaskHandle task : tasks) {
try {
DnsRefreshRequest request = DnsRefreshRequest.createFromTask(task, clock.nowUtc());
if (request.isRefreshNeeded()) {
requestsBuilder.add(request);
hostKeys.add(request.hostKey());
} else {
// Skip hosts that are deleted.
requestsToDelete.add(request);
}
} catch (Exception e) {
logger.atSevere().withCause(e).log(
"Could not parse DNS refresh for host request, delaying task for a day: %s", task);
// Grab the lease for a whole day, so it won't continue throwing errors every five minutes.
pullQueue.modifyTaskLease(task, 1L, DAYS);
}
}
deleteTasksWithRetry(
requestsToDelete, pullQueue, asyncTaskMetrics, retrier, OperationResult.STALE);
ImmutableList<DnsRefreshRequest> refreshRequests = requestsBuilder.build();
if (refreshRequests.isEmpty()) {
logRespondAndUnlock(
INFO, "No async DNS refreshes to process because all renamed hosts are deleted.", lock);
} else {
logger.atInfo().log(
"Processing asynchronous DNS refresh for renamed hosts: %s", hostKeys.build());
if (tm().isOfy()) {
runMapreduce(refreshRequests, lock);
} else {
try {
refreshRequests.stream()
.flatMap(
request ->
getLinkedDomainKeys(request.hostKey(), request.lastUpdateTime(), null)
.stream())
.distinct()
.map(domainKey -> tm().transact(() -> tm().loadByKey(domainKey).getDomainName()))
.forEach(
domainName -> {
retrier.callWithRetry(
() -> dnsQueue.addDomainRefreshTask(domainName),
TransientFailureException.class);
logger.atInfo().log("Enqueued DNS refresh for domain '%s'.", domainName);
});
deleteTasksWithRetry(
refreshRequests,
getQueue(QUEUE_ASYNC_HOST_RENAME),
asyncTaskMetrics,
retrier,
OperationResult.SUCCESS);
} catch (Throwable t) {
String message = "Error refreshing DNS on host rename.";
logger.atSevere().withCause(t).log(message);
response.setPayload(message);
response.setStatus(HttpStatus.SC_INTERNAL_SERVER_ERROR);
} finally {
lock.get().release();
}
}
}
}
private void runMapreduce(ImmutableList<DnsRefreshRequest> refreshRequests, Optional<Lock> lock) {
try {
mrRunner
.setJobName("Enqueue DNS refreshes for domains referencing renamed hosts")
.setModuleName("backend")
.setDefaultReduceShards(1)
.runMapreduce(
new RefreshDnsOnHostRenameMapper(refreshRequests, retrier),
new RefreshDnsOnHostRenameReducer(refreshRequests, lock.get(), retrier),
// Add an extra NullInput so that the reducer always fires exactly once.
ImmutableList.of(new NullInput<>(), createEntityInput(DomainBase.class)))
.sendLinkToMapreduceConsole(response);
} catch (Throwable t) {
logRespondAndUnlock(
SEVERE, "Error starting mapreduce to refresh DNS for renamed hosts.", lock);
}
}
private void logRespondAndUnlock(Level level, String message, Optional<Lock> lock) {
logger.at(level).log(message);
response.setPayload(message);
lock.ifPresent(Lock::release);
}
/** Map over domains and refresh the DNS of those that reference the renamed hosts. */
public static class RefreshDnsOnHostRenameMapper
extends Mapper<DomainBase, Boolean, Boolean> {
private static final long serialVersionUID = -5261698524424335531L;
private static final DnsQueue dnsQueue = DnsQueue.create();
private final ImmutableList<DnsRefreshRequest> refreshRequests;
private final Retrier retrier;
RefreshDnsOnHostRenameMapper(
ImmutableList<DnsRefreshRequest> refreshRequests, Retrier retrier) {
this.refreshRequests = refreshRequests;
this.retrier = retrier;
}
@Override
public final void map(@Nullable final DomainBase domain) {
if (domain == null) {
// Emit a single value so that the reducer always runs. The key and value don't matter.
emit(true, true);
return;
}
VKey<HostResource> referencingHostKey = null;
for (DnsRefreshRequest request : refreshRequests) {
if (isActive(domain, request.lastUpdateTime())
&& domain.getNameservers().contains(request.hostKey())) {
referencingHostKey = request.hostKey();
break;
}
}
if (referencingHostKey != null) {
retrier.callWithRetry(
() -> dnsQueue.addDomainRefreshTask(domain.getDomainName()),
TransientFailureException.class);
logger.atInfo().log(
"Enqueued DNS refresh for domain %s referenced by host %s.",
domain.getDomainName(), referencingHostKey);
getContext().incrementCounter("domains refreshed");
} else {
getContext().incrementCounter("domains not refreshed");
}
// Don't catch errors -- we allow the mapreduce to terminate on any errors that can't be
// resolved by retrying the transaction. The reducer only fires if the mapper completes
// without errors, meaning that it is acceptable to delete all tasks.
}
}
/**
* A reducer that always fires exactly once.
*
* <p>This is really a reducer in name only; what it's really doing is waiting for all of the
* mapper tasks to finish, and then delete the pull queue tasks. Note that this only happens if
* the mapper completes execution without errors.
*/
public static class RefreshDnsOnHostRenameReducer extends Reducer<Boolean, Boolean, Void> {
private static final long serialVersionUID = 9077366205249562118L;
@NonFinalForTesting
private static AsyncTaskMetrics asyncTaskMetrics = new AsyncTaskMetrics(new SystemClock());
private final Lock lock;
private final Retrier retrier;
private final List<DnsRefreshRequest> refreshRequests;
RefreshDnsOnHostRenameReducer(
List<DnsRefreshRequest> refreshRequests, Lock lock, Retrier retrier) {
this.refreshRequests = refreshRequests;
this.lock = lock;
this.retrier = retrier;
}
@Override
public void reduce(Boolean key, ReducerInput<Boolean> values) {
// The reduce() method is run precisely once, because the NullInput caused the mapper to emit
// a dummy value once.
deleteTasksWithRetry(
refreshRequests,
getQueue(QUEUE_ASYNC_HOST_RENAME),
asyncTaskMetrics,
retrier,
OperationResult.SUCCESS);
lock.release();
}
}
/** Deletes a list of tasks from the given queue using a retrier. */
private static void deleteTasksWithRetry(
final List<DnsRefreshRequest> refreshRequests,
final Queue queue,
AsyncTaskMetrics asyncTaskMetrics,
Retrier retrier,
OperationResult result) {
if (refreshRequests.isEmpty()) {
return;
}
final List<TaskHandle> tasks =
refreshRequests.stream().map(DnsRefreshRequest::task).collect(toImmutableList());
retrier.callWithRetry(() -> queue.deleteTask(tasks), TransientFailureException.class);
refreshRequests.forEach(
r -> asyncTaskMetrics.recordAsyncFlowResult(DNS_REFRESH, result, r.requestedTime()));
}
/** A class that encapsulates the values of a request to refresh DNS for a renamed host. */
@AutoValue
abstract static class DnsRefreshRequest implements Serializable {
private static final long serialVersionUID = 1772812852271288622L;
abstract VKey<HostResource> hostKey();
abstract DateTime lastUpdateTime();
abstract DateTime requestedTime();
abstract boolean isRefreshNeeded();
abstract TaskHandle task();
@AutoValue.Builder
abstract static class Builder {
abstract Builder setHostKey(VKey<HostResource> hostKey);
abstract Builder setLastUpdateTime(DateTime lastUpdateTime);
abstract Builder setRequestedTime(DateTime requestedTime);
abstract Builder setIsRefreshNeeded(boolean isRefreshNeeded);
abstract Builder setTask(TaskHandle task);
abstract DnsRefreshRequest build();
}
/**
* Returns a packaged-up {@link DnsRefreshRequest} parsed from a task queue task.
*/
static DnsRefreshRequest createFromTask(TaskHandle task, DateTime now) throws Exception {
ImmutableMap<String, String> params = ImmutableMap.copyOf(task.extractParams());
VKey<HostResource> hostKey =
VKey.create(checkNotNull(params.get(PARAM_HOST_KEY), "Host to refresh not specified"));
HostResource host =
tm().transact(() -> tm().loadByKeyIfPresent(hostKey))
.orElseThrow(() -> new NoSuchElementException("Host to refresh doesn't exist"));
boolean isHostDeleted =
isDeleted(host, latestOf(now, host.getUpdateTimestamp().getTimestamp()));
if (isHostDeleted) {
logger.atInfo().log("Host %s is already deleted, not refreshing DNS.", hostKey);
}
return new AutoValue_RefreshDnsOnHostRenameAction_DnsRefreshRequest.Builder()
.setHostKey(hostKey)
.setLastUpdateTime(host.getUpdateTimestamp().getTimestamp())
.setRequestedTime(
DateTime.parse(
checkNotNull(params.get(PARAM_REQUESTED_TIME), "Requested time not specified")))
.setIsRefreshNeeded(!isHostDeleted)
.setTask(task)
.build();
}
}
}

View File

@@ -31,7 +31,7 @@ import google.registry.model.domain.DomainBase;
import google.registry.model.domain.RegistryLock;
import google.registry.model.eppcommon.StatusValue;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarContact;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.model.tld.RegistryLockDao;
import google.registry.persistence.VKey;
import google.registry.request.Action;
@@ -120,10 +120,7 @@ public class RelockDomainAction implements Runnable {
* for more details on retry behavior. */
response.setStatus(SC_NO_CONTENT);
response.setContentType(MediaType.PLAIN_TEXT_UTF_8);
// nb: DomainLockUtils relies on the JPA transaction being the outermost transaction
// if we have Datastore as the primary DB (if SQL is the primary DB, it's irrelevant)
jpaTm().transact(() -> tm().transact(this::relockDomain));
tm().transact(this::relockDomain);
}
private void relockDomain() {
@@ -296,8 +293,8 @@ public class RelockDomainAction implements Runnable {
ImmutableSet<String> registryLockEmailAddresses =
registrar.getContacts().stream()
.filter(RegistrarContact::isRegistryLockAllowed)
.map(RegistrarContact::getRegistryLockEmailAddress)
.filter(RegistrarPoc::isRegistryLockAllowed)
.map(RegistrarPoc::getRegistryLockEmailAddress)
.filter(Optional::isPresent)
.map(Optional::get)
.collect(toImmutableSet());

View File

@@ -1,128 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch;
import static google.registry.mapreduce.MapreduceRunner.PARAM_FAST;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.appengine.tools.mapreduce.Mapper;
import com.google.common.collect.ImmutableList;
import com.googlecode.objectify.Key;
import google.registry.mapreduce.MapreduceRunner;
import google.registry.mapreduce.inputs.EppResourceInputs;
import google.registry.model.EppResource;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import javax.inject.Inject;
/**
* A mapreduce that re-saves all EppResources, projecting them forward to the current time.
*
* <p>This is useful for completing data migrations on EppResource fields that are accomplished
* with @OnSave or @OnLoad annotations, and also guarantees that all EppResources will get fresh
* commit logs (for backup purposes). Additionally, pending actions such as transfers or grace
* periods that are past their effective time will be resolved.
*
* <p>Because there are no auth settings in the {@link Action} annotation, this command can only be
* run internally, or by pretending to be internal by setting the X-AppEngine-QueueName header,
* which only admin users can do.
*
* <p>If the <code>?fast=true</code> querystring parameter is passed, then entities that are not
* changed by {@link EppResource#cloneProjectedAtTime} will not be re-saved. This helps prevent
* mutation load on the DB and has the beneficial side effect of writing out smaller commit logs.
* Note that this does NOT pick up mutations caused by migrations using the {@link
* com.googlecode.objectify.annotation.OnLoad} annotation, so if you are running a one-off schema
* migration, do not use fast mode. Fast mode defaults to false for this reason, but is used by the
* monthly invocation of the mapreduce.
*/
@Action(
service = Action.Service.BACKEND,
path = "/_dr/task/resaveAllEppResources",
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
// No longer needed in SQL. Subject to future removal.
@Deprecated
@DeleteAfterMigration
public class ResaveAllEppResourcesAction implements Runnable {
@Inject MapreduceRunner mrRunner;
@Inject Response response;
@Inject
@Parameter(PARAM_FAST)
boolean isFast;
@Inject
ResaveAllEppResourcesAction() {}
/**
* The number of shards to run the map-only mapreduce on.
*
* <p>This is less than the default of 100 because we only run this action monthly and can afford
* it being slower, but we don't want to write out lots of large commit logs in a short period of
* time because they make the Cloud SQL migration tougher.
*/
private static final int NUM_SHARDS = 10;
@Override
public void run() {
mrRunner
.setJobName("Re-save all EPP resources")
.setModuleName("backend")
.setDefaultMapShards(NUM_SHARDS)
.runMapOnly(
new ResaveAllEppResourcesActionMapper(isFast),
ImmutableList.of(EppResourceInputs.createKeyInput(EppResource.class)))
.sendLinkToMapreduceConsole(response);
}
/** Mapper to re-save all EPP resources. */
public static class ResaveAllEppResourcesActionMapper
extends Mapper<Key<EppResource>, Void, Void> {
private static final long serialVersionUID = -7721628665138087001L;
private final boolean isFast;
ResaveAllEppResourcesActionMapper(boolean isFast) {
this.isFast = isFast;
}
@Override
public final void map(final Key<EppResource> resourceKey) {
boolean resaved =
tm().transact(
() -> {
EppResource originalResource = auditedOfy().load().key(resourceKey).now();
EppResource projectedResource =
originalResource.cloneProjectedAtTime(tm().getTransactionTime());
if (isFast && originalResource.equals(projectedResource)) {
return false;
} else {
auditedOfy().save().entity(projectedResource).now();
return true;
}
});
getContext()
.incrementCounter(
String.format(
"%s entities %s",
resourceKey.getKind(), resaved ? "re-saved" : "with no changes skipped"));
}
}
}

View File

@@ -1,4 +1,4 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -14,9 +14,8 @@
package google.registry.batch;
import static com.google.common.net.MediaType.PLAIN_TEXT_UTF_8;
import static google.registry.batch.BatchModule.PARAM_FAST;
import static google.registry.beam.BeamUtils.createJobName;
import static javax.servlet.http.HttpServletResponse.SC_FORBIDDEN;
import static javax.servlet.http.HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
import static javax.servlet.http.HttpServletResponse.SC_OK;
@@ -25,54 +24,66 @@ import com.google.api.services.dataflow.model.LaunchFlexTemplateParameter;
import com.google.api.services.dataflow.model.LaunchFlexTemplateRequest;
import com.google.api.services.dataflow.model.LaunchFlexTemplateResponse;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import com.google.common.net.MediaType;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryEnvironment;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import javax.inject.Inject;
/**
* Wipes out all Cloud Datastore data in a Nomulus GCP environment.
* Starts a Dataflow pipeline that resaves all EPP resources projected to the current time.
*
* <p>This class is created for the QA environment, where migration testing with production data
* will happen. A regularly scheduled wipeout is a prerequisite to using production data there.
* <p>This is useful for a few situations. First, as a fallback option for resource transfers that
* have expired pending transfers (this will resolve them), just in case the enqueued action fails.
* Second, it will reflect domain autorenews that have happened. Third, it will remove any expired
* grace periods.
*
* <p>There's also the general principle that it's good to have the data in the database remain as
* current as is reasonably possible.
*
* <p>If the <code>?isFast=true</code> query string parameter is passed as true, the pipeline will
* only attempt to load, project, and resave entities where we expect one of the previous situations
* has occurred. Otherwise, we will load, project, and resave all EPP resources.
*
* <p>This runs the {@link google.registry.beam.resave.ResaveAllEppResourcesPipeline}.
*/
@Action(
service = Action.Service.BACKEND,
path = "/_dr/task/wipeOutDatastore",
path = ResaveAllEppResourcesPipelineAction.PATH,
auth = Auth.AUTH_INTERNAL_OR_ADMIN)
@DeleteAfterMigration
public class WipeoutDatastoreAction implements Runnable {
public class ResaveAllEppResourcesPipelineAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final String PIPELINE_NAME = "bulk_delete_datastore_pipeline";
private static final ImmutableSet<RegistryEnvironment> FORBIDDEN_ENVIRONMENTS =
ImmutableSet.of(RegistryEnvironment.PRODUCTION, RegistryEnvironment.SANDBOX);
static final String PATH = "/_dr/task/resaveAllEppResourcesPipeline";
static final String PIPELINE_NAME = "resave_all_epp_resources_pipeline";
private final String projectId;
private final String jobRegion;
private final String stagingBucketUrl;
private final boolean fast;
private final Clock clock;
private final Response response;
private final Dataflow dataflow;
private final String stagingBucketUrl;
private final Clock clock;
@Inject
WipeoutDatastoreAction(
ResaveAllEppResourcesPipelineAction(
@Config("projectId") String projectId,
@Config("defaultJobRegion") String jobRegion,
@Config("beamStagingBucketUrl") String stagingBucketUrl,
@Parameter(PARAM_FAST) boolean fast,
Clock clock,
Response response,
Dataflow dataflow) {
this.projectId = projectId;
this.jobRegion = jobRegion;
this.stagingBucketUrl = stagingBucketUrl;
this.fast = fast;
this.clock = clock;
this.response = response;
this.dataflow = dataflow;
@@ -80,26 +91,19 @@ public class WipeoutDatastoreAction implements Runnable {
@Override
public void run() {
response.setContentType(PLAIN_TEXT_UTF_8);
if (FORBIDDEN_ENVIRONMENTS.contains(RegistryEnvironment.get())) {
response.setStatus(SC_FORBIDDEN);
response.setPayload("Wipeout is not allowed in " + RegistryEnvironment.get());
return;
}
response.setContentType(MediaType.PLAIN_TEXT_UTF_8);
logger.atInfo().log("Launching ResaveAllEppResourcesPipeline");
try {
LaunchFlexTemplateParameter parameters =
LaunchFlexTemplateParameter parameter =
new LaunchFlexTemplateParameter()
.setJobName(createJobName("bulk-delete-datastore-", clock))
.setJobName(createJobName("resave-all-epp-resources", clock))
.setContainerSpecGcsPath(
String.format("%s/%s_metadata.json", stagingBucketUrl, PIPELINE_NAME))
.setParameters(
ImmutableMap.of(
"kindsToDelete",
"*",
"registryEnvironment",
RegistryEnvironment.get().name()));
new ImmutableMap.Builder<String, String>()
.put(PARAM_FAST, Boolean.toString(fast))
.put("registryEnvironment", RegistryEnvironment.get().name())
.build());
LaunchFlexTemplateResponse launchResponse =
dataflow
.projects()
@@ -108,15 +112,16 @@ public class WipeoutDatastoreAction implements Runnable {
.launch(
projectId,
jobRegion,
new LaunchFlexTemplateRequest().setLaunchParameter(parameters))
new LaunchFlexTemplateRequest().setLaunchParameter(parameter))
.execute();
logger.atInfo().log("Got response: %s", launchResponse.getJob().toPrettyString());
String jobId = launchResponse.getJob().getId();
response.setStatus(SC_OK);
response.setPayload("Launched " + launchResponse.getJob().getName());
response.setPayload(String.format("Launched resaveAllEppResources pipeline: %s", jobId));
} catch (Exception e) {
String msg = String.format("Failed to launch %s.", PIPELINE_NAME);
logger.atSevere().withCause(e).log(msg);
logger.atSevere().withCause(e).log("Template Launch failed.");
response.setStatus(SC_INTERNAL_SERVER_ERROR);
response.setPayload(msg);
response.setPayload(String.format("Pipeline launch failed: %s", e.getMessage()));
}
}
}

View File

@@ -32,8 +32,8 @@ import com.google.common.net.MediaType;
import google.registry.config.RegistryConfig.Config;
import google.registry.flows.certs.CertificateChecker;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarContact;
import google.registry.model.registrar.RegistrarContact.Type;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.model.registrar.RegistrarPoc.Type;
import google.registry.request.Action;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
@@ -281,9 +281,9 @@ public class SendExpiringCertificateNotificationEmailAction implements Runnable
*/
@VisibleForTesting
ImmutableSet<InternetAddress> getEmailAddresses(Registrar registrar, Type contactType) {
ImmutableSortedSet<RegistrarContact> contacts = registrar.getContactsOfType(contactType);
ImmutableSortedSet<RegistrarPoc> contacts = registrar.getContactsOfType(contactType);
ImmutableSet.Builder<InternetAddress> recipientEmails = new ImmutableSet.Builder<>();
for (RegistrarContact contact : contacts) {
for (RegistrarPoc contact : contacts) {
try {
recipientEmails.add(new InternetAddress(contact.getEmailAddress()));
} catch (AddressException e) {

View File

@@ -23,8 +23,6 @@ import com.google.common.collect.Streams;
import google.registry.beam.common.RegistryQuery.CriteriaQuerySupplier;
import google.registry.model.UpdateAutoTimestamp;
import google.registry.model.UpdateAutoTimestamp.DisableAutoUpdateResource;
import google.registry.model.common.DatabaseMigrationStateSchedule;
import google.registry.model.replay.SqlEntity;
import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.persistence.transaction.TransactionManagerFactory;
import java.io.Serializable;
@@ -235,10 +233,6 @@ public final class RegistryJpaIO {
@ProcessElement
public void processElement(OutputReceiver<T> outputReceiver) {
// Preload the migration schedule into cache, otherwise the cache loading query may happen
// before the setDatabaseSnapshot call in the transaction below, causing it to fail.
DatabaseMigrationStateSchedule.get();
jpaTm()
.transactNoRetry(
() -> {
@@ -433,11 +427,23 @@ public final class RegistryJpaIO {
}
}
/** Returns this entity's primary key field(s) in a string. */
private String toEntityKeyString(Object entity) {
if (entity instanceof SqlEntity) {
return ((SqlEntity) entity).getPrimaryKeyString();
try {
return jpaTm()
.transact(
() ->
String.format(
"%s_%s",
entity.getClass().getSimpleName(),
jpaTm()
.getEntityManager()
.getEntityManagerFactory()
.getPersistenceUnitUtil()
.getIdentifier(entity)));
} catch (IllegalArgumentException e) {
return "Non-SqlEntity: " + entity;
}
return "Non-SqlEntity: " + String.valueOf(entity);
}
}
}

View File

@@ -1,193 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.comparedb;
import static google.registry.beam.comparedb.ValidateSqlUtils.createSqlEntityTupleTag;
import static google.registry.beam.initsql.Transforms.createTagForKind;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Verify;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.backup.VersionedEntity;
import google.registry.beam.initsql.Transforms;
import google.registry.model.EppResource;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.billing.BillingEvent;
import google.registry.model.common.Cursor;
import google.registry.model.contact.ContactHistory;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.host.HostHistory;
import google.registry.model.host.HostResource;
import google.registry.model.poll.PollMessage;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarContact;
import google.registry.model.replay.SqlEntity;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.tld.Registry;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import javax.annotation.Nullable;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.values.PCollection;
import org.apache.beam.sdk.values.PCollectionTuple;
import org.apache.beam.sdk.values.TupleTag;
import org.apache.beam.sdk.values.TupleTagList;
import org.joda.time.DateTime;
/** Utilities for loading Datastore snapshots. */
@DeleteAfterMigration
public final class DatastoreSnapshots {
private DatastoreSnapshots() {}
/**
* Datastore kinds eligible for validation. This set must be consistent with {@link
* SqlSnapshots#ALL_SQL_ENTITIES}.
*/
@VisibleForTesting
static final ImmutableSet<Class<?>> ALL_DATASTORE_KINDS =
ImmutableSet.of(
Registry.class,
Cursor.class,
Registrar.class,
ContactResource.class,
RegistrarContact.class,
HostResource.class,
HistoryEntry.class,
AllocationToken.class,
BillingEvent.Recurring.class,
BillingEvent.OneTime.class,
BillingEvent.Cancellation.class,
PollMessage.class,
DomainBase.class);
/**
* Returns the Datastore snapshot right before {@code commitLogToTime} for the user specified
* {@code kinds}. The resulting snapshot has all changes that happened before {@code
* commitLogToTime}, and none at or after {@code commitLogToTime}.
*
* <p>If {@code HistoryEntry} is included in {@code kinds}, the result will contain {@code
* PCollections} for the child entities, {@code DomainHistory}, {@code ContactHistory}, and {@code
* HostHistory}.
*/
static PCollectionTuple loadDatastoreSnapshotByKind(
Pipeline pipeline,
String exportDir,
String commitLogDir,
DateTime commitLogFromTime,
DateTime commitLogToTime,
Set<Class<?>> kinds,
Optional<DateTime> compareStartTime) {
PCollectionTuple snapshot =
pipeline.apply(
"Load Datastore snapshot.",
Transforms.loadDatastoreSnapshot(
exportDir,
commitLogDir,
commitLogFromTime,
commitLogToTime,
kinds.stream().map(Key::getKind).collect(ImmutableSet.toImmutableSet())));
PCollectionTuple perTypeSnapshots = PCollectionTuple.empty(pipeline);
for (Class<?> kind : kinds) {
PCollection<VersionedEntity> perKindSnapshot =
snapshot.get(createTagForKind(Key.getKind(kind)));
if (SqlEntity.class.isAssignableFrom(kind)) {
perTypeSnapshots =
perTypeSnapshots.and(
createSqlEntityTupleTag((Class<? extends SqlEntity>) kind),
datastoreEntityToPojo(perKindSnapshot, kind.getSimpleName(), compareStartTime));
continue;
}
Verify.verify(kind == HistoryEntry.class, "Unexpected Non-SqlEntity class: %s", kind);
PCollectionTuple historyEntriesByType = splitHistoryEntry(perKindSnapshot, compareStartTime);
for (Map.Entry<TupleTag<?>, PCollection<?>> entry :
historyEntriesByType.getAll().entrySet()) {
perTypeSnapshots = perTypeSnapshots.and(entry.getKey().getId(), entry.getValue());
}
}
return perTypeSnapshots;
}
/**
* Splits a {@link PCollection} of {@link HistoryEntry HistoryEntries} into three collections of
* its child entities by type.
*/
static PCollectionTuple splitHistoryEntry(
PCollection<VersionedEntity> historyEntries, Optional<DateTime> compareStartTime) {
DateTime nullableStartTime = compareStartTime.orElse(null);
return historyEntries.apply(
"Split HistoryEntry by Resource Type",
ParDo.of(
new DoFn<VersionedEntity, SqlEntity>() {
@ProcessElement
public void processElement(
@Element VersionedEntity historyEntry, MultiOutputReceiver out) {
Optional.ofNullable(Transforms.convertVersionedEntityToSqlEntity(historyEntry))
.filter(e -> isEntityIncludedForComparison(e, nullableStartTime))
.ifPresent(
sqlEntity ->
out.get(createSqlEntityTupleTag(sqlEntity.getClass()))
.output(sqlEntity));
}
})
.withOutputTags(
createSqlEntityTupleTag(DomainHistory.class),
TupleTagList.of(createSqlEntityTupleTag(ContactHistory.class))
.and(createSqlEntityTupleTag(HostHistory.class))));
}
/**
* Transforms a {@link PCollection} of {@link VersionedEntity VersionedEntities} to Ofy Java
* objects.
*/
static PCollection<SqlEntity> datastoreEntityToPojo(
PCollection<VersionedEntity> entities, String desc, Optional<DateTime> compareStartTime) {
DateTime nullableStartTime = compareStartTime.orElse(null);
return entities.apply(
"Datastore Entity to Pojo " + desc,
ParDo.of(
new DoFn<VersionedEntity, SqlEntity>() {
@ProcessElement
public void processElement(
@Element VersionedEntity entity, OutputReceiver<SqlEntity> out) {
Optional.ofNullable(Transforms.convertVersionedEntityToSqlEntity(entity))
.filter(e -> isEntityIncludedForComparison(e, nullableStartTime))
.ifPresent(out::output);
}
}));
}
static boolean isEntityIncludedForComparison(
SqlEntity entity, @Nullable DateTime compareStartTime) {
if (compareStartTime == null) {
return true;
}
if (entity instanceof HistoryEntry) {
return compareStartTime.isBefore(((HistoryEntry) entity).getModificationTime());
}
if (entity instanceof EppResource) {
return compareStartTime.isBefore(((EppResource) entity).getUpdateTimestamp().getTimestamp());
}
return true;
}
}

View File

@@ -1,154 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.comparedb;
import com.google.auto.value.AutoValue;
import com.google.cloud.storage.BlobId;
import com.google.cloud.storage.BlobInfo;
import dagger.Component;
import google.registry.config.CloudTasksUtilsModule;
import google.registry.config.CredentialModule;
import google.registry.config.RegistryConfig;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.gcs.GcsUtils;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.util.Clock;
import google.registry.util.UtilsModule;
import java.io.IOException;
import java.util.Comparator;
import java.util.NoSuchElementException;
import java.util.Optional;
import javax.inject.Inject;
import javax.inject.Singleton;
import org.joda.time.DateTime;
import org.joda.time.Instant;
import org.joda.time.Interval;
/** Finds the necessary information for loading the most recent Datastore snapshot. */
@DeleteAfterMigration
public class LatestDatastoreSnapshotFinder {
private final String projectId;
private final GcsUtils gcsUtils;
private final Clock clock;
@Inject
LatestDatastoreSnapshotFinder(
@Config("projectId") String projectId, GcsUtils gcsUtils, Clock clock) {
this.projectId = projectId;
this.gcsUtils = gcsUtils;
this.clock = clock;
}
/**
* Finds information of the most recent Datastore snapshot that ends strictly before {@code
* exportEndTimeUpperBound}, including the GCS folder of the exported data files and the start and
* stop times of the export. The folder of the CommitLogs is also included in the return.
*/
public DatastoreSnapshotInfo getSnapshotInfo(Instant exportEndTimeUpperBound) {
String bucketName = RegistryConfig.getDatastoreBackupsBucket().substring("gs://".length());
/**
* Find the bucket-relative path to the overall metadata file of the last Datastore export.
* Since Datastore export is saved daily, we may need to look back to yesterday. If found, the
* return value is like
* "2021-11-19T06:00:00_76493/2021-11-19T06:00:00_76493.overall_export_metadata".
*/
Optional<String> metaFilePathOptional =
findNewestExportMetadataFileBeforeTime(bucketName, exportEndTimeUpperBound, 5);
if (!metaFilePathOptional.isPresent()) {
throw new NoSuchElementException("No exports found over the past 2 days.");
}
String metaFilePath = metaFilePathOptional.get();
String metaFileFolder = metaFilePath.substring(0, metaFilePath.indexOf('/'));
Instant exportStartTime = Instant.parse(metaFileFolder.replace('_', '.') + 'Z');
BlobInfo blobInfo = gcsUtils.getBlobInfo(BlobId.of(bucketName, metaFilePath));
Instant exportEndTime = new Instant(blobInfo.getCreateTime());
return DatastoreSnapshotInfo.create(
String.format("gs://%s/%s", bucketName, metaFileFolder),
getCommitLogDir(),
new Interval(exportStartTime, exportEndTime));
}
public String getCommitLogDir() {
return "gs://" + projectId + "-commits";
}
/**
* Finds the latest Datastore export that ends strictly before {@code endTimeUpperBound} and
* returns the bucket-relative path of the overall export metadata file, in the given bucket. The
* search goes back for up to {@code lookBackDays} days in time, including today.
*
* <p>The overall export metadata file is the last file created during a Datastore export. All
* data has been exported by the creation time of this file. The name of this file, like that of
* all files in the same export, begins with the timestamp when the export starts.
*
* <p>An example return value: {@code
* 2021-11-19T06:00:00_76493/2021-11-19T06:00:00_76493.overall_export_metadata}.
*/
private Optional<String> findNewestExportMetadataFileBeforeTime(
String bucketName, Instant endTimeUpperBound, int lookBackDays) {
DateTime today = clock.nowUtc();
for (int day = 0; day < lookBackDays; day++) {
String dateString = today.minusDays(day).toString("yyyy-MM-dd");
try {
Optional<String> metaFilePath =
gcsUtils.listFolderObjects(bucketName, dateString).stream()
.filter(s -> s.endsWith("overall_export_metadata"))
.map(s -> dateString + s)
.sorted(Comparator.<String>naturalOrder().reversed())
.findFirst();
if (metaFilePath.isPresent()) {
BlobInfo blobInfo = gcsUtils.getBlobInfo(BlobId.of(bucketName, metaFilePath.get()));
Instant exportEndTime = new Instant(blobInfo.getCreateTime());
if (exportEndTime.isBefore(endTimeUpperBound)) {
return metaFilePath;
}
}
} catch (IOException ioe) {
throw new RuntimeException(ioe);
}
}
return Optional.empty();
}
/** Holds information about a Datastore snapshot. */
@AutoValue
public abstract static class DatastoreSnapshotInfo {
public abstract String exportDir();
public abstract String commitLogDir();
public abstract Interval exportInterval();
static DatastoreSnapshotInfo create(
String exportDir, String commitLogDir, Interval exportOperationInterval) {
return new AutoValue_LatestDatastoreSnapshotFinder_DatastoreSnapshotInfo(
exportDir, commitLogDir, exportOperationInterval);
}
}
@Singleton
@Component(
modules = {
CredentialModule.class,
ConfigModule.class,
CloudTasksUtilsModule.class,
UtilsModule.class
})
interface LatestDatastoreSnapshotFinderFinderComponent {
LatestDatastoreSnapshotFinder datastoreSnapshotInfoFinder();
}
}

View File

@@ -1,539 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.comparedb;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.beam.comparedb.ValidateSqlUtils.createSqlEntityTupleTag;
import static google.registry.beam.comparedb.ValidateSqlUtils.getMedianIdForHistoryTable;
import com.google.auto.value.AutoValue;
import com.google.common.base.Strings;
import com.google.common.base.Verify;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSetMultimap;
import com.google.common.collect.Streams;
import google.registry.beam.common.RegistryJpaIO;
import google.registry.beam.common.RegistryJpaIO.Read;
import google.registry.model.EppResource;
import google.registry.model.UpdateAutoTimestamp;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.billing.BillingEvent;
import google.registry.model.bulkquery.BulkQueryEntities;
import google.registry.model.bulkquery.DomainBaseLite;
import google.registry.model.bulkquery.DomainHistoryHost;
import google.registry.model.bulkquery.DomainHistoryLite;
import google.registry.model.bulkquery.DomainHost;
import google.registry.model.common.Cursor;
import google.registry.model.contact.ContactHistory;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.domain.DomainHistory.DomainHistoryId;
import google.registry.model.domain.GracePeriod;
import google.registry.model.domain.GracePeriod.GracePeriodHistory;
import google.registry.model.domain.secdns.DelegationSignerData;
import google.registry.model.domain.secdns.DomainDsDataHistory;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.host.HostHistory;
import google.registry.model.host.HostResource;
import google.registry.model.poll.PollMessage;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarContact;
import google.registry.model.replay.SqlEntity;
import google.registry.model.reporting.DomainTransactionRecord;
import google.registry.model.tld.Registry;
import google.registry.persistence.transaction.CriteriaQueryBuilder;
import google.registry.util.DateTimeUtils;
import java.io.Serializable;
import java.util.Optional;
import javax.persistence.Entity;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.Flatten;
import org.apache.beam.sdk.transforms.GroupByKey;
import org.apache.beam.sdk.transforms.MapElements;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.transforms.SerializableFunction;
import org.apache.beam.sdk.values.KV;
import org.apache.beam.sdk.values.PCollection;
import org.apache.beam.sdk.values.PCollectionList;
import org.apache.beam.sdk.values.PCollectionTuple;
import org.apache.beam.sdk.values.TypeDescriptor;
import org.apache.beam.sdk.values.TypeDescriptors;
import org.joda.time.DateTime;
/**
* Utilities for loading SQL snapshots.
*
* <p>For {@link DomainBase} and {@link DomainHistory}, this class assumes the presence of the
* {@link google.registry.persistence.PersistenceModule.JpaTransactionManagerType#BULK_QUERY
* bulk-query-capable JpaTransactionManager}, and takes advantage of it for higher throughput.
*
* <p>For now this class is meant for use during the database migration period only. Therefore, it
* contains optimizations specifically for the production database at the current size, e.g.,
* parallel queries for select tables.
*/
@DeleteAfterMigration
public final class SqlSnapshots {
private SqlSnapshots() {}
/**
* SQL entity types that are eligible for validation. This set must be consistent with {@link
* DatastoreSnapshots#ALL_DATASTORE_KINDS}.
*/
static final ImmutableSet<Class<? extends SqlEntity>> ALL_SQL_ENTITIES =
ImmutableSet.of(
Registry.class,
Cursor.class,
Registrar.class,
ContactResource.class,
RegistrarContact.class,
HostResource.class,
AllocationToken.class,
BillingEvent.Recurring.class,
BillingEvent.OneTime.class,
BillingEvent.Cancellation.class,
PollMessage.class,
DomainBase.class,
ContactHistory.class,
HostHistory.class,
DomainHistory.class);
/**
* Loads a SQL snapshot for the given {@code sqlEntityTypes}.
*
* <p>If {@code snapshotId} is present, all queries use the specified database snapshot,
* guaranteeing a consistent result.
*/
public static PCollectionTuple loadCloudSqlSnapshotByType(
Pipeline pipeline,
ImmutableSet<Class<? extends SqlEntity>> sqlEntityTypes,
Optional<String> snapshotId,
Optional<DateTime> compareStartTime) {
PCollectionTuple perTypeSnapshots = PCollectionTuple.empty(pipeline);
for (Class<? extends SqlEntity> clazz : sqlEntityTypes) {
if (clazz == DomainBase.class) {
perTypeSnapshots =
perTypeSnapshots.and(
createSqlEntityTupleTag(DomainBase.class),
loadAndAssembleDomainBase(pipeline, snapshotId, compareStartTime));
continue;
}
if (clazz == DomainHistory.class) {
perTypeSnapshots =
perTypeSnapshots.and(
createSqlEntityTupleTag(DomainHistory.class),
loadAndAssembleDomainHistory(pipeline, snapshotId, compareStartTime));
continue;
}
if (clazz == ContactHistory.class) {
perTypeSnapshots =
perTypeSnapshots.and(
createSqlEntityTupleTag(ContactHistory.class),
loadContactHistory(pipeline, snapshotId, compareStartTime));
continue;
}
if (clazz == HostHistory.class) {
perTypeSnapshots =
perTypeSnapshots.and(
createSqlEntityTupleTag(HostHistory.class),
loadHostHistory(
pipeline, snapshotId, compareStartTime.orElse(DateTimeUtils.START_OF_TIME)));
continue;
}
if (EppResource.class.isAssignableFrom(clazz) && compareStartTime.isPresent()) {
perTypeSnapshots =
perTypeSnapshots.and(
createSqlEntityTupleTag(clazz),
pipeline.apply(
"SQL Load " + clazz.getSimpleName(),
buildEppResourceQueryWithTimeFilter(
clazz, SqlEntity.class, snapshotId, compareStartTime.get())
.withSnapshot(snapshotId.orElse(null))));
continue;
}
perTypeSnapshots =
perTypeSnapshots.and(
createSqlEntityTupleTag(clazz),
pipeline.apply(
"SQL Load " + clazz.getSimpleName(),
RegistryJpaIO.read(
() -> CriteriaQueryBuilder.create(clazz).build(), SqlEntity.class::cast)
.withSnapshot(snapshotId.orElse(null))));
}
return perTypeSnapshots;
}
/**
* Bulk-loads parts of {@link DomainBase} and assembles them in the pipeline.
*
* @see BulkQueryEntities
*/
public static PCollection<SqlEntity> loadAndAssembleDomainBase(
Pipeline pipeline, Optional<String> snapshotId, Optional<DateTime> compareStartTime) {
PCollection<KV<String, Serializable>> baseObjects =
readAllAndAssignKey(
pipeline,
DomainBaseLite.class,
DomainBaseLite::getRepoId,
snapshotId,
compareStartTime);
PCollection<KV<String, Serializable>> gracePeriods =
readAllAndAssignKey(
pipeline,
GracePeriod.class,
GracePeriod::getDomainRepoId,
snapshotId,
compareStartTime);
PCollection<KV<String, Serializable>> delegationSigners =
readAllAndAssignKey(
pipeline,
DelegationSignerData.class,
DelegationSignerData::getDomainRepoId,
snapshotId,
compareStartTime);
PCollection<KV<String, Serializable>> domainHosts =
readAllAndAssignKey(
pipeline, DomainHost.class, DomainHost::getDomainRepoId, snapshotId, compareStartTime);
DateTime nullableCompareStartTime = compareStartTime.orElse(null);
return PCollectionList.of(
ImmutableList.of(baseObjects, gracePeriods, delegationSigners, domainHosts))
.apply("SQL Merge DomainBase parts", Flatten.pCollections())
.apply("Group by Domain Parts by RepoId", GroupByKey.create())
.apply(
"Assemble DomainBase",
ParDo.of(
new DoFn<KV<String, Iterable<Serializable>>, SqlEntity>() {
@ProcessElement
public void processElement(
@Element KV<String, Iterable<Serializable>> kv,
OutputReceiver<SqlEntity> outputReceiver) {
TypedClassifier partsByType = new TypedClassifier(kv.getValue());
ImmutableSet<DomainBaseLite> baseObjects =
partsByType.getAllOf(DomainBaseLite.class);
if (nullableCompareStartTime != null) {
Verify.verify(
baseObjects.size() <= 1,
"Found duplicate DomainBaseLite object per repoId: " + kv.getKey());
if (baseObjects.isEmpty()) {
return;
}
}
Verify.verify(
baseObjects.size() == 1,
"Expecting one DomainBaseLite object per repoId: " + kv.getKey());
outputReceiver.output(
BulkQueryEntities.assembleDomainBase(
baseObjects.iterator().next(),
partsByType.getAllOf(GracePeriod.class),
partsByType.getAllOf(DelegationSignerData.class),
partsByType.getAllOf(DomainHost.class).stream()
.map(DomainHost::getHostVKey)
.collect(ImmutableSet.toImmutableSet())));
}
}));
}
/**
* Loads all {@link ContactHistory} entities from the database.
*
* <p>This method uses two queries to load data in parallel. This is a performance optimization
* specifically for the production database.
*/
static PCollection<SqlEntity> loadContactHistory(
Pipeline pipeline, Optional<String> snapshotId, Optional<DateTime> compareStartTime) {
PartitionedQuery partitionedQuery =
buildPartitonedHistoryQuery(ContactHistory.class, compareStartTime);
PCollection<SqlEntity> part1 =
pipeline.apply(
"SQL Load ContactHistory first half",
RegistryJpaIO.read(
partitionedQuery.firstHalfQuery(),
partitionedQuery.parameters(),
false,
SqlEntity.class::cast)
.withSnapshot(snapshotId.orElse(null)));
PCollection<SqlEntity> part2 =
pipeline.apply(
"SQL Load ContactHistory second half",
RegistryJpaIO.read(
partitionedQuery.secondHalfQuery(),
partitionedQuery.parameters(),
false,
SqlEntity.class::cast)
.withSnapshot(snapshotId.orElse(null)));
return PCollectionList.of(part1)
.and(part2)
.apply("Combine ContactHistory parts", Flatten.pCollections());
}
/** Loads all {@link HostHistory} entities from the database. */
static PCollection<SqlEntity> loadHostHistory(
Pipeline pipeline, Optional<String> snapshotId, DateTime compareStartTime) {
return pipeline.apply(
"SQL Load HostHistory",
RegistryJpaIO.read(
"select c from HostHistory c where :compareStartTime < modificationTime",
ImmutableMap.of("compareStartTime", compareStartTime),
false,
SqlEntity.class::cast)
.withSnapshot(snapshotId.orElse(null)));
}
/**
* Bulk-loads all parts of {@link DomainHistory} and assembles them in the pipeline.
*
* <p>This method uses two queries to load {@link DomainBaseLite} in parallel. This is a
* performance optimization specifically for the production database.
*
* @see BulkQueryEntities
*/
static PCollection<SqlEntity> loadAndAssembleDomainHistory(
Pipeline pipeline, Optional<String> snapshotId, Optional<DateTime> compareStartTime) {
PartitionedQuery partitionedQuery =
buildPartitonedHistoryQuery(DomainHistoryLite.class, compareStartTime);
PCollection<KV<String, Serializable>> baseObjectsPart1 =
queryAndAssignKey(
pipeline,
"first half",
partitionedQuery.firstHalfQuery(),
partitionedQuery.parameters(),
DomainHistoryLite.class,
compose(DomainHistoryLite::getDomainHistoryId, DomainHistoryId::toString),
snapshotId);
PCollection<KV<String, Serializable>> baseObjectsPart2 =
queryAndAssignKey(
pipeline,
"second half",
partitionedQuery.secondHalfQuery(),
partitionedQuery.parameters(),
DomainHistoryLite.class,
compose(DomainHistoryLite::getDomainHistoryId, DomainHistoryId::toString),
snapshotId);
PCollection<KV<String, Serializable>> gracePeriods =
readAllAndAssignKey(
pipeline,
GracePeriodHistory.class,
compose(GracePeriodHistory::getDomainHistoryId, DomainHistoryId::toString),
snapshotId,
compareStartTime);
PCollection<KV<String, Serializable>> delegationSigners =
readAllAndAssignKey(
pipeline,
DomainDsDataHistory.class,
compose(DomainDsDataHistory::getDomainHistoryId, DomainHistoryId::toString),
snapshotId,
compareStartTime);
PCollection<KV<String, Serializable>> domainHosts =
readAllAndAssignKey(
pipeline,
DomainHistoryHost.class,
compose(DomainHistoryHost::getDomainHistoryId, DomainHistoryId::toString),
snapshotId,
compareStartTime);
PCollection<KV<String, Serializable>> transactionRecords =
readAllAndAssignKey(
pipeline,
DomainTransactionRecord.class,
compose(DomainTransactionRecord::getDomainHistoryId, DomainHistoryId::toString),
snapshotId,
compareStartTime);
DateTime nullableCompareStartTime = compareStartTime.orElse(null);
return PCollectionList.of(
ImmutableList.of(
baseObjectsPart1,
baseObjectsPart2,
gracePeriods,
delegationSigners,
domainHosts,
transactionRecords))
.apply("Merge DomainHistory parts", Flatten.pCollections())
.apply("Group by DomainHistory Parts by DomainHistoryId string", GroupByKey.create())
.apply(
"Assemble DomainHistory",
ParDo.of(
new DoFn<KV<String, Iterable<Serializable>>, SqlEntity>() {
@ProcessElement
public void processElement(
@Element KV<String, Iterable<Serializable>> kv,
OutputReceiver<SqlEntity> outputReceiver) {
TypedClassifier partsByType = new TypedClassifier(kv.getValue());
ImmutableSet<DomainHistoryLite> baseObjects =
partsByType.getAllOf(DomainHistoryLite.class);
if (nullableCompareStartTime != null) {
Verify.verify(
baseObjects.size() <= 1,
"Found duplicate DomainHistoryLite object per domainHistoryId: "
+ kv.getKey());
if (baseObjects.isEmpty()) {
return;
}
}
Verify.verify(
baseObjects.size() == 1,
"Expecting one DomainHistoryLite object per domainHistoryId: "
+ kv.getKey());
outputReceiver.output(
BulkQueryEntities.assembleDomainHistory(
baseObjects.iterator().next(),
partsByType.getAllOf(DomainDsDataHistory.class),
partsByType.getAllOf(DomainHistoryHost.class).stream()
.map(DomainHistoryHost::getHostVKey)
.collect(ImmutableSet.toImmutableSet()),
partsByType.getAllOf(GracePeriodHistory.class),
partsByType.getAllOf(DomainTransactionRecord.class)));
}
}));
}
static <R, T> PCollection<KV<String, Serializable>> readAllAndAssignKey(
Pipeline pipeline,
Class<R> type,
SerializableFunction<R, String> keyFunction,
Optional<String> snapshotId,
Optional<DateTime> compareStartTime) {
Read<R, R> queryObject;
if (compareStartTime.isPresent() && EppResource.class.isAssignableFrom(type)) {
queryObject =
buildEppResourceQueryWithTimeFilter(type, type, snapshotId, compareStartTime.get());
} else {
queryObject =
RegistryJpaIO.read(() -> CriteriaQueryBuilder.create(type).build())
.withSnapshot(snapshotId.orElse(null));
}
return pipeline
.apply("SQL Load " + type.getSimpleName(), queryObject)
.apply(
"Assign Key to " + type.getSimpleName(),
MapElements.into(
TypeDescriptors.kvs(
TypeDescriptors.strings(), TypeDescriptor.of(Serializable.class)))
.via(obj -> KV.of(keyFunction.apply(obj), (Serializable) obj)));
}
static <R, T> PCollection<KV<String, Serializable>> queryAndAssignKey(
Pipeline pipeline,
String diffrentiator,
String jplQuery,
ImmutableMap<String, Object> queryParameters,
Class<R> type,
SerializableFunction<R, String> keyFunction,
Optional<String> snapshotId) {
return pipeline
.apply(
"SQL Load " + type.getSimpleName() + " " + diffrentiator,
RegistryJpaIO.read(jplQuery, queryParameters, false, type::cast)
.withSnapshot(snapshotId.orElse(null)))
.apply(
"Assign Key to " + type.getSimpleName() + " " + diffrentiator,
MapElements.into(
TypeDescriptors.kvs(
TypeDescriptors.strings(), TypeDescriptor.of(Serializable.class)))
.via(obj -> KV.of(keyFunction.apply(obj), (Serializable) obj)));
}
// TODO(b/205988530): don't use beam serializablefunction, make one that extends Java's Function.
private static <R, I, T> SerializableFunction<R, T> compose(
SerializableFunction<R, I> f1, SerializableFunction<I, T> f2) {
return r -> f2.apply(f1.apply(r));
}
static <R, T> Read<R, T> buildEppResourceQueryWithTimeFilter(
Class<R> entityType,
Class<T> castOutputAsType,
Optional<String> snapshotId,
DateTime compareStartTime) {
String tableName = getJpaEntityName(entityType);
String jpql =
String.format("select c from %s c where :compareStartTime < updateTimestamp", tableName);
return RegistryJpaIO.read(
jpql,
ImmutableMap.of("compareStartTime", UpdateAutoTimestamp.create(compareStartTime)),
false,
(R x) -> castOutputAsType.cast(x))
.withSnapshot(snapshotId.orElse(null));
}
static PartitionedQuery buildPartitonedHistoryQuery(
Class<?> entityType, Optional<DateTime> compareStartTime) {
String tableName = getJpaEntityName(entityType);
Verify.verify(
!Strings.isNullOrEmpty(tableName), "Invalid entity type %s", entityType.getSimpleName());
long medianId =
getMedianIdForHistoryTable(tableName)
.orElseThrow(() -> new IllegalStateException("Not a valid database: no " + tableName));
String firstHalfQuery = String.format("select c from %s c where id <= :historyId", tableName);
String secondHalfQuery = String.format("select c from %s c where id > :historyId", tableName);
if (compareStartTime.isPresent()) {
String timeFilter = " and :compareStartTime < modificationTime";
firstHalfQuery += timeFilter;
secondHalfQuery += timeFilter;
return PartitionedQuery.createPartitionedQuery(
firstHalfQuery,
secondHalfQuery,
ImmutableMap.of("historyId", medianId, "compareStartTime", compareStartTime.get()));
} else {
return PartitionedQuery.createPartitionedQuery(
firstHalfQuery, secondHalfQuery, ImmutableMap.of("historyId", medianId));
}
}
private static String getJpaEntityName(Class entityType) {
Entity entityAnnotation = (Entity) entityType.getAnnotation(Entity.class);
checkState(
entityAnnotation != null, "Unexpected non-entity type %s", entityType.getSimpleName());
return Strings.isNullOrEmpty(entityAnnotation.name())
? entityType.getSimpleName()
: entityAnnotation.name();
}
/** Contains two queries that partition the target table in two. */
@AutoValue
abstract static class PartitionedQuery {
abstract String firstHalfQuery();
abstract String secondHalfQuery();
abstract ImmutableMap<String, Object> parameters();
public static PartitionedQuery createPartitionedQuery(
String firstHalfQuery, String secondHalfQuery, ImmutableMap<String, Object> parameters) {
return new AutoValue_SqlSnapshots_PartitionedQuery(
firstHalfQuery, secondHalfQuery, parameters);
}
}
/** Container that receives mixed-typed data and groups them by {@link Class}. */
static class TypedClassifier {
private final ImmutableSetMultimap<Class<?>, Object> classifiedEntities;
TypedClassifier(Iterable<Serializable> inputs) {
this.classifiedEntities =
Streams.stream(inputs)
.collect(ImmutableSetMultimap.toImmutableSetMultimap(Object::getClass, x -> x));
}
<T> ImmutableSet<T> getAllOf(Class<T> clazz) {
return classifiedEntities.get(clazz).stream()
.map(clazz::cast)
.collect(ImmutableSet.toImmutableSet());
}
}
}

View File

@@ -1,206 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.comparedb;
import static com.google.common.base.Verify.verify;
import static org.apache.beam.sdk.values.TypeDescriptors.strings;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Strings;
import com.google.common.flogger.FluentLogger;
import google.registry.beam.common.RegistryPipelineOptions;
import google.registry.beam.common.RegistryPipelineWorkerInitializer;
import google.registry.beam.comparedb.LatestDatastoreSnapshotFinder.DatastoreSnapshotInfo;
import google.registry.beam.comparedb.ValidateSqlUtils.CompareSqlEntity;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.DomainHistory;
import google.registry.model.replay.SqlEntity;
import google.registry.persistence.PersistenceModule.JpaTransactionManagerType;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.util.SystemClock;
import java.io.Serializable;
import java.util.Optional;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.coders.SerializableCoder;
import org.apache.beam.sdk.io.TextIO;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.apache.beam.sdk.transforms.Flatten;
import org.apache.beam.sdk.transforms.GroupByKey;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.transforms.WithKeys;
import org.apache.beam.sdk.values.PCollectionList;
import org.apache.beam.sdk.values.PCollectionTuple;
import org.apache.beam.sdk.values.TupleTag;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/**
* Validates the asynchronous data replication process between Datastore and Cloud SQL.
*
* <p>This pipeline is to be launched by {@link google.registry.tools.ValidateDatastoreCommand} or
* {@link google.registry.tools.ValidateSqlCommand}.
*/
@DeleteAfterMigration
public class ValidateDatabasePipeline {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
/** Specifies the extra CommitLogs to load before the start of a Database export. */
private static final Duration COMMITLOG_START_TIME_MARGIN = Duration.standardMinutes(10);
private final ValidateDatabasePipelineOptions options;
private final LatestDatastoreSnapshotFinder datastoreSnapshotFinder;
public ValidateDatabasePipeline(
ValidateDatabasePipelineOptions options,
LatestDatastoreSnapshotFinder datastoreSnapshotFinder) {
this.options = options;
this.datastoreSnapshotFinder = datastoreSnapshotFinder;
}
@VisibleForTesting
void run(Pipeline pipeline) {
DateTime latestCommitLogTime = DateTime.parse(options.getLatestCommitLogTimestamp());
DatastoreSnapshotInfo mostRecentExport =
datastoreSnapshotFinder.getSnapshotInfo(latestCommitLogTime.toInstant());
logger.atInfo().log(
"Comparing datastore export at %s and commitlog timestamp %s.",
mostRecentExport.exportDir(), latestCommitLogTime);
Optional<String> outputPath =
Optional.ofNullable(options.getDiffOutputGcsBucket())
.map(
bucket ->
String.format(
"gs://%s/validate_database/%s/diffs.txt",
bucket, new SystemClock().nowUtc()));
outputPath.ifPresent(path -> logger.atInfo().log("Discrepancies will be logged to %s", path));
setupPipeline(
pipeline,
Optional.ofNullable(options.getSqlSnapshotId()),
mostRecentExport,
latestCommitLogTime,
Optional.ofNullable(options.getComparisonStartTimestamp()).map(DateTime::parse),
outputPath);
pipeline.run();
}
static void setupPipeline(
Pipeline pipeline,
Optional<String> sqlSnapshotId,
DatastoreSnapshotInfo mostRecentExport,
DateTime latestCommitLogTime,
Optional<DateTime> compareStartTime,
Optional<String> diffOutputPath) {
pipeline
.getCoderRegistry()
.registerCoderForClass(SqlEntity.class, SerializableCoder.of(Serializable.class));
PCollectionTuple datastoreSnapshot =
DatastoreSnapshots.loadDatastoreSnapshotByKind(
pipeline,
mostRecentExport.exportDir(),
mostRecentExport.commitLogDir(),
mostRecentExport.exportInterval().getStart().minus(COMMITLOG_START_TIME_MARGIN),
// Increase by 1ms since we want to include commitLogs latestCommitLogTime but
// this parameter is exclusive.
latestCommitLogTime.plusMillis(1),
DatastoreSnapshots.ALL_DATASTORE_KINDS,
compareStartTime);
PCollectionTuple cloudSqlSnapshot =
SqlSnapshots.loadCloudSqlSnapshotByType(
pipeline, SqlSnapshots.ALL_SQL_ENTITIES, sqlSnapshotId, compareStartTime);
verify(
datastoreSnapshot.getAll().keySet().equals(cloudSqlSnapshot.getAll().keySet()),
"Expecting the same set of types in both snapshots.");
PCollectionList<String> diffLogs = PCollectionList.empty(pipeline);
for (Class<? extends SqlEntity> clazz : SqlSnapshots.ALL_SQL_ENTITIES) {
TupleTag<SqlEntity> tag = ValidateSqlUtils.createSqlEntityTupleTag(clazz);
verify(
datastoreSnapshot.has(tag), "Missing %s in Datastore snapshot.", clazz.getSimpleName());
verify(cloudSqlSnapshot.has(tag), "Missing %s in Cloud SQL snapshot.", clazz.getSimpleName());
diffLogs =
diffLogs.and(
PCollectionList.of(datastoreSnapshot.get(tag))
.and(cloudSqlSnapshot.get(tag))
.apply(
"Combine from both snapshots: " + clazz.getSimpleName(),
Flatten.pCollections())
.apply(
"Assign primary key to merged " + clazz.getSimpleName(),
WithKeys.of(ValidateDatabasePipeline::getPrimaryKeyString)
.withKeyType(strings()))
.apply("Group by primary key " + clazz.getSimpleName(), GroupByKey.create())
.apply("Compare " + clazz.getSimpleName(), ParDo.of(new CompareSqlEntity())));
}
if (diffOutputPath.isPresent()) {
diffLogs
.apply("Gather diff logs", Flatten.pCollections())
.apply(
"Output diffs",
TextIO.write()
.to(diffOutputPath.get())
/**
* Output to a single file for ease of use since diffs should be few. If this
* assumption turns out not to be false, user should abort the pipeline and
* investigate why.
*/
.withoutSharding()
.withDelimiter((Strings.repeat("-", 80) + "\n").toCharArray()));
}
}
private static String getPrimaryKeyString(SqlEntity sqlEntity) {
// SqlEntity.getPrimaryKeyString only works with entities registered with Hibernate.
// We are using the BulkQueryJpaTransactionManager, which does not recognize DomainBase and
// DomainHistory. See BulkQueryEntities.java for more information.
if (sqlEntity instanceof DomainBase) {
return "DomainBase_" + ((DomainBase) sqlEntity).getRepoId();
}
if (sqlEntity instanceof DomainHistory) {
return "DomainHistory_" + ((DomainHistory) sqlEntity).getDomainHistoryId().toString();
}
return sqlEntity.getPrimaryKeyString();
}
public static void main(String[] args) {
ValidateDatabasePipelineOptions options =
PipelineOptionsFactory.fromArgs(args)
.withValidation()
.as(ValidateDatabasePipelineOptions.class);
RegistryPipelineOptions.validateRegistryPipelineOptions(options);
// Defensively set important options.
options.setIsolationOverride(TransactionIsolationLevel.TRANSACTION_REPEATABLE_READ);
options.setJpaTransactionManagerType(JpaTransactionManagerType.BULK_QUERY);
// Set up JPA in the pipeline harness (the locally executed part of the main() method). Reuse
// code in RegistryPipelineWorkerInitializer, which only applies to pipeline worker VMs.
new RegistryPipelineWorkerInitializer().beforeProcessing(options);
LatestDatastoreSnapshotFinder datastoreSnapshotFinder =
DaggerLatestDatastoreSnapshotFinder_LatestDatastoreSnapshotFinderFinderComponent.create()
.datastoreSnapshotInfoFinder();
new ValidateDatabasePipeline(options, datastoreSnapshotFinder).run(Pipeline.create(options));
}
}

View File

@@ -1,55 +0,0 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.comparedb;
import google.registry.beam.common.RegistryPipelineOptions;
import google.registry.model.annotations.DeleteAfterMigration;
import javax.annotation.Nullable;
import org.apache.beam.sdk.options.Description;
import org.apache.beam.sdk.options.Validation;
/** BEAM pipeline options for {@link ValidateDatabasePipeline}. */
@DeleteAfterMigration
public interface ValidateDatabasePipelineOptions extends RegistryPipelineOptions {
@Description(
"The id of the SQL snapshot to be compared with Datastore. "
+ "If null, the current state of the SQL database is used.")
@Nullable
String getSqlSnapshotId();
void setSqlSnapshotId(String snapshotId);
@Description("The latest CommitLogs to load, in ISO8601 format.")
@Validation.Required
String getLatestCommitLogTimestamp();
void setLatestCommitLogTimestamp(String commitLogEndTimestamp);
@Description(
"For history entries and EPP resources, only those modified strictly after this time are "
+ "included in comparison. Value is in ISO8601 format. "
+ "Other entity types are not affected.")
@Nullable
String getComparisonStartTimestamp();
void setComparisonStartTimestamp(String comparisonStartTimestamp);
@Description("The GCS bucket where discrepancies found during comparison should be logged.")
@Nullable
String getDiffOutputGcsBucket();
void setDiffOutputGcsBucket(String gcsBucket);
}

View File

@@ -1,427 +0,0 @@
// Copyright 2021 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.comparedb;
import static com.google.common.base.Verify.verify;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import com.google.common.base.Preconditions;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Maps;
import com.google.common.collect.Maps.EntryTransformer;
import google.registry.beam.initsql.Transforms;
import google.registry.config.RegistryEnvironment;
import google.registry.model.EppResource;
import google.registry.model.ImmutableObject;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.billing.BillingEvent;
import google.registry.model.contact.ContactBase;
import google.registry.model.contact.ContactHistory;
import google.registry.model.domain.DomainContent;
import google.registry.model.domain.DomainHistory;
import google.registry.model.eppcommon.AuthInfo;
import google.registry.model.host.HostHistory;
import google.registry.model.poll.PollMessage;
import google.registry.model.registrar.Registrar;
import google.registry.model.replay.SqlEntity;
import google.registry.model.reporting.HistoryEntry;
import google.registry.util.DiffUtils;
import java.io.Serializable;
import java.lang.reflect.Field;
import java.math.BigInteger;
import java.util.Collection;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.stream.Collectors;
import org.apache.beam.sdk.metrics.Counter;
import org.apache.beam.sdk.metrics.Metrics;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.values.KV;
import org.apache.beam.sdk.values.TupleTag;
import org.checkerframework.checker.nullness.qual.Nullable;
/** Helpers for use by {@link ValidateDatabasePipeline}. */
@DeleteAfterMigration
final class ValidateSqlUtils {
private ValidateSqlUtils() {}
private static final ImmutableSet<String> PROBER_CELLS = ImmutableSet.of("IQ", "LG", "TL");
private static final ImmutableSet<String> PROBER_TYPES =
ImmutableSet.of("ANYT", "ANYTES", "CANARY");
/**
* Query template for finding the median value of the {@code history_revision_id} column in one of
* the History tables.
*
* <p>The {@link ValidateDatabasePipeline} uses this query to parallelize the query to some of the
* history tables. Although the {@code repo_id} column is the leading column in the primary keys
* of these tables, in practice and with production data, division by {@code history_revision_id}
* works slightly faster for unknown reasons.
*/
private static final String MEDIAN_ID_QUERY_TEMPLATE =
"SELECT history_revision_id FROM ( "
+ " SELECT"
+ " ROW_NUMBER() OVER (ORDER BY history_revision_id ASC) AS rownumber,"
+ " history_revision_id"
+ " FROM \"%TABLE%\""
+ ") AS foo\n"
+ "WHERE rownumber in (select count(*) / 2 + 1 from \"%TABLE%\")";
static Optional<Long> getMedianIdForHistoryTable(String tableName) {
Preconditions.checkArgument(
tableName.endsWith("History"), "Table must be one of the History tables.");
String sqlText = MEDIAN_ID_QUERY_TEMPLATE.replace("%TABLE%", tableName);
List results =
jpaTm()
.transact(() -> jpaTm().getEntityManager().createNativeQuery(sqlText).getResultList());
verify(results.size() < 2, "MidPoint query should have at most one result.");
if (results.isEmpty()) {
return Optional.empty();
}
return Optional.of(((BigInteger) results.get(0)).longValue());
}
static TupleTag<SqlEntity> createSqlEntityTupleTag(Class<? extends SqlEntity> actualType) {
return new TupleTag<SqlEntity>(actualType.getSimpleName()) {};
}
static class CompareSqlEntity extends DoFn<KV<String, Iterable<SqlEntity>>, String> {
private final HashMap<String, Counter> totalCounters = new HashMap<>();
private final HashMap<String, Counter> missingCounters = new HashMap<>();
private final HashMap<String, Counter> unequalCounters = new HashMap<>();
private final HashMap<String, Counter> badEntityCounters = new HashMap<>();
private final HashMap<String, Counter> duplicateEntityCounters = new HashMap<>();
private String getCounterKey(Class<?> clazz) {
return PollMessage.class.isAssignableFrom(clazz) ? "PollMessage" : clazz.getSimpleName();
}
private synchronized void ensureCounterExists(String counterKey) {
if (totalCounters.containsKey(counterKey)) {
return;
}
totalCounters.put(counterKey, Metrics.counter("CompareDB", "Total Compared: " + counterKey));
missingCounters.put(
counterKey, Metrics.counter("CompareDB", "Missing In One DB: " + counterKey));
unequalCounters.put(counterKey, Metrics.counter("CompareDB", "Not Equal:" + counterKey));
badEntityCounters.put(counterKey, Metrics.counter("CompareDB", "Bad Entities:" + counterKey));
duplicateEntityCounters.put(
counterKey, Metrics.counter("CompareDB", "Duplicate Entities:" + counterKey));
}
String duplicateEntityLog(String key, ImmutableList<SqlEntity> entities) {
return String.format("%s: %d entities.", key, entities.size());
}
String unmatchedEntityLog(String key, SqlEntity entry) {
// For a PollMessage only found in Datastore, key is not enough to query for it.
return String.format("Missing in one DB:\n%s", entry instanceof PollMessage ? entry : key);
}
/**
* A rudimentary debugging helper that prints the first pair of unequal entities in each worker.
* This will be removed when we start exporting such entities to GCS.
*/
String unEqualEntityLog(String key, SqlEntity entry0, SqlEntity entry1) {
Map<String, Object> fields0 = ((ImmutableObject) entry0).toDiffableFieldMap();
Map<String, Object> fields1 = ((ImmutableObject) entry1).toDiffableFieldMap();
return key + " " + DiffUtils.prettyPrintEntityDeepDiff(fields0, fields1);
}
String badEntitiesLog(String key, SqlEntity entry0, SqlEntity entry1) {
Map<String, Object> fields0 = ((ImmutableObject) entry0).toDiffableFieldMap();
Map<String, Object> fields1 = ((ImmutableObject) entry1).toDiffableFieldMap();
return String.format(
"Failed to parse one or both entities for key %s:\n%s\n",
key, DiffUtils.prettyPrintEntityDeepDiff(fields0, fields1));
}
@ProcessElement
public void processElement(
@Element KV<String, Iterable<SqlEntity>> kv, OutputReceiver<String> out) {
ImmutableList<SqlEntity> entities = ImmutableList.copyOf(kv.getValue());
verify(!entities.isEmpty(), "Can't happen: no value for key %s.", kv.getKey());
String counterKey = getCounterKey(entities.get(0).getClass());
ensureCounterExists(counterKey);
totalCounters.get(counterKey).inc();
if (entities.size() > 2) {
// Duplicates may happen with Cursors if imported across projects. Its key in Datastore, the
// id field, encodes the project name and is not fixed by the importing job.
duplicateEntityCounters.get(counterKey).inc();
out.output(duplicateEntityLog(kv.getKey(), entities) + "\n");
return;
}
if (entities.size() == 1) {
if (isSpecialCaseProberEntity(entities.get(0))) {
return;
}
missingCounters.get(counterKey).inc();
out.output(unmatchedEntityLog(kv.getKey(), entities.get(0)) + "\n");
return;
}
SqlEntity entity0 = entities.get(0);
SqlEntity entity1 = entities.get(1);
if (isSpecialCaseProberEntity(entity0) && isSpecialCaseProberEntity(entity1)) {
// Ignore prober-related data: their deletions are not propagated from Datastore to SQL.
// When code reaches here, in most cases it involves one soft deleted entity in Datastore
// and an SQL entity with its pre-deletion status.
return;
}
try {
entity0 = normalizeEntity(entity0);
entity1 = normalizeEntity(entity1);
} catch (Exception e) {
badEntityCounters.get(counterKey).inc();
out.output(badEntitiesLog(kv.getKey(), entity0, entity1));
return;
}
Map<String, Object> fields0 =
Maps.transformEntries(
((ImmutableObject) entity0).toDiffableFieldMap(), new DiffableFieldNormalizer());
Map<String, Object> fields1 =
Maps.transformEntries(
((ImmutableObject) entity1).toDiffableFieldMap(), new DiffableFieldNormalizer());
if (!Objects.equals(fields0, fields1)) {
unequalCounters.get(counterKey).inc();
out.output(kv.getKey() + " " + DiffUtils.prettyPrintEntityDeepDiff(fields0, fields1));
}
}
}
/**
* Normalizes trivial differences between objects persisted in Datastore and SQL.
*
* <p>This class works on a map generated by {@link ImmutableObject#toDiffableFieldMap}, and
* performs the following changes:
*
* <ul>
* <li>If a value is an empty {@link Collection}, it is converted to null
* <li>For each {@link google.registry.model.eppcommon.Address} object, empty strings are
* removed from its {@code string} field, which is a {@link List}.
* </ul>
*/
static class DiffableFieldNormalizer
implements EntryTransformer<String, Object, Object>, Serializable {
@Override
public Object transformEntry(String key, @Nullable Object value) {
if (value instanceof Collection && ((Collection<?>) value).isEmpty()) {
return null;
}
if (key.equals("street") && value instanceof List) {
return ((List<?>) value)
.stream().filter(v -> v != null && !Objects.equals("", v)).collect(Collectors.toList());
}
// Short-term hack: LinkedHashMap<String, ?> represents a child ImmutableObject instance.
if (value instanceof LinkedHashMap
&& ((LinkedHashMap<?, ?>) value).keySet().stream().anyMatch(e -> e instanceof String)) {
return Maps.transformEntries((Map<String, Object>) value, this);
}
return value;
}
}
static SqlEntity normalizeEntity(SqlEntity sqlEntity) {
if (sqlEntity instanceof EppResource) {
return normalizeEppResource(sqlEntity);
}
if (sqlEntity instanceof HistoryEntry) {
return (SqlEntity) normalizeHistoryEntry((HistoryEntry) sqlEntity);
}
return sqlEntity;
}
/**
* Normalizes an {@link EppResource} instance for comparison.
*
* <p>This method may modify the input object using reflection instead of making a copy with
* {@code eppResource.asBuilder().build()}, because when {@code eppResource} is a {@link
* google.registry.model.domain.DomainBase}, the {@code build} method accesses the Database, which
* we want to avoid.
*/
static SqlEntity normalizeEppResource(SqlEntity eppResource) {
try {
Field authField =
eppResource instanceof DomainContent
? DomainContent.class.getDeclaredField("authInfo")
: eppResource instanceof ContactBase
? ContactBase.class.getDeclaredField("authInfo")
: null;
if (authField != null) {
authField.setAccessible(true);
AuthInfo authInfo = (AuthInfo) authField.get(eppResource);
// When AuthInfo is missing, the authInfo field is null if the object is loaded from
// Datastore, or a PasswordAuth with null properties if loaded from SQL. In the second case
// we set the authInfo field to null.
if (authInfo != null
&& authInfo.getPw() != null
&& authInfo.getPw().getRepoId() == null
&& authInfo.getPw().getValue() == null) {
authField.set(eppResource, null);
}
}
Field field = EppResource.class.getDeclaredField("revisions");
field.setAccessible(true);
field.set(eppResource, null);
return eppResource;
} catch (Exception e) {
throw new RuntimeException(e);
}
}
/**
* Normalizes a {@link HistoryEntry} for comparison.
*
* <p>This method modifies the input using reflection because relevant builder methods performs
* unwanted checks and changes.
*/
static HistoryEntry normalizeHistoryEntry(HistoryEntry historyEntry) {
// History objects from Datastore do not have details of their EppResource objects
// (domainContent, contactBase, hostBase).
try {
if (historyEntry instanceof DomainHistory) {
Field domainContent = DomainHistory.class.getDeclaredField("domainContent");
domainContent.setAccessible(true);
domainContent.set(historyEntry, null);
// Convert empty domainTransactionRecords to null for comparison.
Field domainTransactionRecords =
HistoryEntry.class.getDeclaredField("domainTransactionRecords");
domainTransactionRecords.setAccessible(true);
Set<?> domainTransactionRecordsValue = (Set<?>) domainTransactionRecords.get(historyEntry);
if (domainTransactionRecordsValue != null && domainTransactionRecordsValue.isEmpty()) {
domainTransactionRecords.set(historyEntry, null);
}
// DomainHistory in Datastore does not have the following properties either:
Field nsHosts = DomainHistory.class.getDeclaredField("nsHosts");
nsHosts.setAccessible(true);
nsHosts.set(historyEntry, null);
Field dsDataHistories = DomainHistory.class.getDeclaredField("dsDataHistories");
dsDataHistories.setAccessible(true);
dsDataHistories.set(historyEntry, null);
Field gracePeriodHistories = DomainHistory.class.getDeclaredField("gracePeriodHistories");
gracePeriodHistories.setAccessible(true);
gracePeriodHistories.set(historyEntry, null);
} else if (historyEntry instanceof ContactHistory) {
Field contactBase = ContactHistory.class.getDeclaredField("contactBase");
contactBase.setAccessible(true);
contactBase.set(historyEntry, null);
} else if (historyEntry instanceof HostHistory) {
Field hostBase = HostHistory.class.getDeclaredField("hostBase");
hostBase.setAccessible(true);
hostBase.set(historyEntry, null);
}
return historyEntry;
} catch (Exception e) {
throw new RuntimeException(e);
}
}
/**
* Returns {@code true} if {@code entity} is created by the prober and needs special treatment.
*
* <p>{@link EppResource} entities created by the prober are deleted by a cron job that bypasses
* the CommitLog mechanism. As a result, their deletions are not propagated to SQL, creating two
* types of mismatches: an entity exists in both databases but differs in lastUpdateTime and
* deletionTime; an entity only exists in the SQL database.
*
* <p>In production, there are few placeholder {@link Registrar registrars} that do not exist in
* Datastore. They were manually created to in SQL to solve a one-time problem (see b/187946868
* for details). They can be ignored in the database comparison.
*/
static boolean isSpecialCaseProberEntity(Object entity) {
if (entity instanceof EppResource) {
EppResource host = (EppResource) entity;
if (host.getPersistedCurrentSponsorRegistrarId().startsWith("prober-")) {
return true;
}
}
if (entity instanceof HistoryEntry) {
HistoryEntry historyEntry = (HistoryEntry) entity;
if (historyEntry.getRegistrarId().startsWith("prober-")) {
// Not all prober entities have "prober-" as registrar prefix.
return true;
}
if (Objects.equals(historyEntry.getReason(), "Deletion of prober data")) {
// Soft-delete event in Datastore that is not propagated to SQL.
return true;
}
}
if (entity instanceof DomainHistory) {
DomainHistory domainHistory = (DomainHistory) entity;
if (domainHistory.getDomainContent().isPresent()
&& domainHistory.getDomainContent().get().getDomainName().startsWith("prober-")) {
// Asynchronously replicated event in SQL.
return true;
}
if (domainHistory.getDomainRepoId() != null) {
// Some synthetic events only have domainRepoId.
String repoId = domainHistory.getDomainRepoId();
if (Transforms.IGNORED_DOMAINS.contains(repoId)) {
return true;
}
String suffix = repoId.substring(repoId.indexOf('-') + 1);
String cell = suffix.substring(0, 2);
suffix = suffix.substring(2);
if (PROBER_CELLS.contains(cell) && PROBER_TYPES.contains(suffix)) {
return true;
}
}
}
if (entity instanceof ContactHistory) {
if (Transforms.IGNORED_CONTACTS.contains(((ContactHistory) entity).getContactRepoId())) {
return true;
}
}
if (entity instanceof HostHistory) {
if (Transforms.IGNORED_HOSTS.contains(((HostHistory) entity).getHostRepoId())) {
return true;
}
}
if (entity instanceof BillingEvent) {
BillingEvent event = (BillingEvent) entity;
if (event.getRegistrarId().startsWith("prober-")) {
return true;
}
}
if (entity instanceof PollMessage) {
if (((PollMessage) entity).getRegistrarId().startsWith("prober-")) {
return true;
}
}
if (RegistryEnvironment.get().equals(RegistryEnvironment.PRODUCTION)
&& entity instanceof Registrar) {
Registrar registrar = (Registrar) entity;
if (registrar.getRegistrarId().startsWith("prober-wj-")) {
return true;
}
}
return false;
}
}

View File

@@ -1,336 +0,0 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.datastore;
import static com.google.common.base.Preconditions.checkState;
import static org.apache.beam.sdk.values.TypeDescriptors.kvs;
import static org.apache.beam.sdk.values.TypeDescriptors.strings;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.flogger.FluentLogger;
import com.google.datastore.v1.Entity;
import google.registry.config.RegistryEnvironment;
import google.registry.model.annotations.DeleteAfterMigration;
import java.util.Iterator;
import java.util.Map;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.extensions.gcp.options.GcpOptions;
import org.apache.beam.sdk.io.gcp.datastore.DatastoreIO;
import org.apache.beam.sdk.options.Default;
import org.apache.beam.sdk.options.Description;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.apache.beam.sdk.options.Validation;
import org.apache.beam.sdk.transforms.Create;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.GroupByKey;
import org.apache.beam.sdk.transforms.MapElements;
import org.apache.beam.sdk.transforms.PTransform;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.transforms.Reshuffle;
import org.apache.beam.sdk.transforms.View;
import org.apache.beam.sdk.values.KV;
import org.apache.beam.sdk.values.PBegin;
import org.apache.beam.sdk.values.PCollection;
import org.apache.beam.sdk.values.PCollectionTuple;
import org.apache.beam.sdk.values.PCollectionView;
import org.apache.beam.sdk.values.TupleTag;
import org.apache.beam.sdk.values.TupleTagList;
/**
* A BEAM pipeline that deletes Datastore entities in bulk.
*
* <p>This pipeline provides an alternative to the <a
* href="https://cloud.google.com/datastore/docs/bulk-delete">GCP builtin template</a> that performs
* the same task. It solves the following performance and usability problems in the builtin
* template:
*
* <ul>
* <li>When deleting all data (by using the {@code select __key__} or {@code select *} queries),
* the builtin template cannot parallelize the query, therefore has to query with a single
* worker.
* <li>When deleting all data, the builtin template also attempts to delete Datastore internal
* tables which would cause permission-denied errors, which in turn MAY cause the pipeline to
* abort before all data has been deleted.
* <li>With the builtin template, it is possible to delete multiple entity types in one pipeline
* ONLY if the user can come up with a single literal query that covers all of them. This is
* not the case with most Nomulus entity types.
* </ul>
*
* <p>A user of this pipeline must specify the types of entities to delete using the {@code
* --kindsToDelete} command line argument. To delete specific entity types, give a comma-separated
* string of their kind names; to delete all data, give {@code "*"}.
*
* <p>When deleting all data, it is recommended for the user to specify the number of user entity
* types in the Datastore using the {@code --numOfKindsHint} argument. If the default value for this
* parameter is too low, performance will suffer.
*/
@DeleteAfterMigration
public class BulkDeleteDatastorePipeline {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
// This tool is not for use in our critical projects.
private static final ImmutableSet<String> FORBIDDEN_PROJECTS =
ImmutableSet.of("domain-registry", "domain-registry-sandbox");
private final BulkDeletePipelineOptions options;
BulkDeleteDatastorePipeline(BulkDeletePipelineOptions options) {
this.options = options;
}
public void run() {
Pipeline pipeline = Pipeline.create(options);
setupPipeline(pipeline);
pipeline.run();
}
@SuppressWarnings("deprecation") // org.apache.beam.sdk.transforms.Reshuffle
private void setupPipeline(Pipeline pipeline) {
checkState(
!FORBIDDEN_PROJECTS.contains(options.getProject()),
"Bulk delete is forbidden in %s",
options.getProject());
// Pre-allocated tags to label entities by kind. In the case of delete-all, we must use a guess.
TupleTagList deletionTags;
PCollection<String> kindsToDelete;
if (options.getKindsToDelete().equals("*")) {
deletionTags = getDeletionTags(options.getNumOfKindsHint());
kindsToDelete =
pipeline.apply("DiscoverEntityKinds", discoverEntityKinds(options.getProject()));
} else {
ImmutableList<String> kindsToDeleteParam = parseKindsToDelete(options);
checkState(
!kindsToDeleteParam.contains("*"),
"The --kindsToDelete argument should not contain both '*' and other kinds.");
deletionTags = getDeletionTags(kindsToDeleteParam.size());
kindsToDelete = pipeline.apply("UseProvidedKinds", Create.of(kindsToDeleteParam));
}
// Map each kind to a tag. The "SplitByKind" stage below will group entities by kind using
// this mapping. In practice, this has been effective at avoiding entity group contentions.
PCollectionView<Map<String, TupleTag<Entity>>> kindToTagMapping =
mapKindsToDeletionTags(kindsToDelete, deletionTags).apply("GetKindsToTagMap", View.asMap());
PCollectionTuple entities =
kindsToDelete
.apply("GenerateQueries", ParDo.of(new GenerateQueries()))
.apply("ReadEntities", DatastoreV1.read().withProjectId(options.getProject()))
.apply(
"SplitByKind",
ParDo.of(new SplitEntities(kindToTagMapping))
.withSideInputs(kindToTagMapping)
.withOutputTags(getOneDeletionTag("placeholder"), deletionTags));
for (TupleTag<?> tag : deletionTags.getAll()) {
entities
.get((TupleTag<Entity>) tag)
// Reshuffle calls GroupByKey which is one way to trigger load rebalance in the pipeline.
// Using the deprecated "Reshuffle" for convenience given the short life of this tool.
.apply("RebalanceLoad", Reshuffle.viaRandomKey())
.apply(
"DeleteEntities_" + tag.getId(),
DatastoreIO.v1().deleteEntity().withProjectId(options.getProject()));
}
}
private static String toKeyOnlyQueryForKind(String kind) {
return "select __key__ from `" + kind + "`";
}
/**
* Returns a {@link TupleTag} that retains the generic type parameter and may be used in a
* multi-output {@link ParDo} (e.g. {@link SplitEntities}).
*
* <p>This method is NOT needed in tests when creating tags for assertions. Simply create them
* with {@code new TupleTag<Entity>(String)}.
*/
@VisibleForTesting
static TupleTag<Entity> getOneDeletionTag(String id) {
// The trailing {} is needed to retain generic param type.
return new TupleTag<Entity>(id) {};
}
@VisibleForTesting
static ImmutableList<String> parseKindsToDelete(BulkDeletePipelineOptions options) {
return ImmutableList.copyOf(
Splitter.on(",").omitEmptyStrings().trimResults().split(options.getKindsToDelete().trim()));
}
/**
* Returns a list of {@code n} {@link TupleTag TupleTags} numbered from {@code 0} to {@code n-1}.
*/
@VisibleForTesting
static TupleTagList getDeletionTags(int n) {
ImmutableList.Builder<TupleTag<?>> builder = new ImmutableList.Builder<>();
for (int i = 0; i < n; i++) {
builder.add(getOneDeletionTag(String.valueOf(i)));
}
return TupleTagList.of(builder.build());
}
/** Returns a {@link PTransform} that finds all entity kinds in Datastore. */
@VisibleForTesting
static PTransform<PBegin, PCollection<String>> discoverEntityKinds(String project) {
return new PTransform<PBegin, PCollection<String>>() {
@Override
public PCollection<String> expand(PBegin input) {
// Use the __kind__ table to discover entity kinds. Data in the more informational
// __Stat_Kind__ table may be up to 48-hour stale.
return input
.apply(
"LoadEntityMetaData",
DatastoreIO.v1()
.read()
.withProjectId(project)
.withLiteralGqlQuery("select * from __kind__"))
.apply(
"GetKindNames",
ParDo.of(
new DoFn<Entity, String>() {
@ProcessElement
public void processElement(
@Element Entity entity, OutputReceiver<String> out) {
String kind = entity.getKey().getPath(0).getName();
if (kind.startsWith("_")) {
return;
}
out.output(kind);
}
}));
}
};
}
@VisibleForTesting
static PCollection<KV<String, TupleTag<Entity>>> mapKindsToDeletionTags(
PCollection<String> kinds, TupleTagList tags) {
// The first two stages send all strings in the 'kinds' PCollection to one worker which
// performs the mapping in the last stage.
return kinds
.apply(
"AssignSingletonKeyToKinds",
MapElements.into(kvs(strings(), strings())).via(kind -> KV.of("", kind)))
.apply("GatherKindsIntoCollection", GroupByKey.create())
.apply("MapKindsToTag", ParDo.of(new MapKindsToTags(tags)));
}
/** Transforms each {@code kind} string into a Datastore query for that kind. */
@VisibleForTesting
static class GenerateQueries extends DoFn<String, String> {
@ProcessElement
public void processElement(@Element String kind, OutputReceiver<String> out) {
out.output(toKeyOnlyQueryForKind(kind));
}
}
private static class MapKindsToTags
extends DoFn<KV<String, Iterable<String>>, KV<String, TupleTag<Entity>>> {
private final TupleTagList tupleTags;
MapKindsToTags(TupleTagList tupleTags) {
this.tupleTags = tupleTags;
}
@ProcessElement
public void processElement(
@Element KV<String, Iterable<String>> kv,
OutputReceiver<KV<String, TupleTag<Entity>>> out) {
// Sort kinds so that mapping is deterministic.
ImmutableSortedSet<String> sortedKinds = ImmutableSortedSet.copyOf(kv.getValue());
Iterator<String> kinds = sortedKinds.iterator();
Iterator<TupleTag<?>> tags = tupleTags.getAll().iterator();
while (kinds.hasNext() && tags.hasNext()) {
out.output(KV.of(kinds.next(), (TupleTag<Entity>) tags.next()));
}
if (kinds.hasNext()) {
logger.atWarning().log(
"There are more kinds to delete (%s) than our estimate (%s). "
+ "Performance may suffer.",
sortedKinds.size(), tupleTags.size());
}
// Round robin assignment so that mapping is deterministic
while (kinds.hasNext()) {
tags = tupleTags.getAll().iterator();
while (kinds.hasNext() && tags.hasNext()) {
out.output(KV.of(kinds.next(), (TupleTag<Entity>) tags.next()));
}
}
}
}
/**
* {@link DoFn} that splits one {@link PCollection} of mixed kinds into multiple single-kind
* {@code PCollections}.
*/
@VisibleForTesting
static class SplitEntities extends DoFn<Entity, Entity> {
private final PCollectionView<Map<String, TupleTag<Entity>>> kindToTagMapping;
SplitEntities(PCollectionView<Map<String, TupleTag<Entity>>> kindToTagMapping) {
super();
this.kindToTagMapping = kindToTagMapping;
}
@ProcessElement
public void processElement(ProcessContext context) {
Entity entity = context.element();
com.google.datastore.v1.Key entityKey = entity.getKey();
String kind = entityKey.getPath(entityKey.getPathCount() - 1).getKind();
TupleTag<Entity> tag = context.sideInput(kindToTagMapping).get(kind);
context.output(tag, entity);
}
}
public static void main(String[] args) {
BulkDeletePipelineOptions options =
PipelineOptionsFactory.fromArgs(args).withValidation().as(BulkDeletePipelineOptions.class);
BulkDeleteDatastorePipeline pipeline = new BulkDeleteDatastorePipeline(options);
pipeline.run();
System.exit(0);
}
public interface BulkDeletePipelineOptions extends GcpOptions {
@Description("The Registry environment.")
RegistryEnvironment getRegistryEnvironment();
void setRegistryEnvironment(RegistryEnvironment environment);
@Description(
"The Datastore KINDs to be deleted. The format may be:\n"
+ "\t- The list of kinds to be deleted as a comma-separated string, or\n"
+ "\t- '*', which causes all kinds to be deleted.")
@Validation.Required
String getKindsToDelete();
void setKindsToDelete(String kinds);
@Description(
"An estimate of the number of KINDs to be deleted. "
+ "This is recommended if --kindsToDelete is '*' and the default value is too low.")
@Default.Integer(30)
int getNumOfKindsHint();
void setNumOfKindsHint(int numOfKindsHint);
}
}

View File

@@ -1,768 +0,0 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// This class is adapted from the Apache BEAM SDK. The original license may
// be found at <a href="https://github.com/apache/beam/blob/master/LICENSE">
// this link</a>.
package google.registry.beam.datastore;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Verify.verify;
import static com.google.datastore.v1.PropertyFilter.Operator.EQUAL;
import static com.google.datastore.v1.PropertyOrder.Direction.DESCENDING;
import static com.google.datastore.v1.QueryResultBatch.MoreResultsType.NOT_FINISHED;
import static com.google.datastore.v1.client.DatastoreHelper.makeAndFilter;
import static com.google.datastore.v1.client.DatastoreHelper.makeFilter;
import static com.google.datastore.v1.client.DatastoreHelper.makeOrder;
import static com.google.datastore.v1.client.DatastoreHelper.makeValue;
import com.google.api.client.http.HttpRequestInitializer;
import com.google.auth.Credentials;
import com.google.auth.http.HttpCredentialsAdapter;
import com.google.auto.value.AutoValue;
import com.google.cloud.hadoop.util.ChainingHttpRequestInitializer;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import com.google.datastore.v1.Entity;
import com.google.datastore.v1.EntityResult;
import com.google.datastore.v1.GqlQuery;
import com.google.datastore.v1.PartitionId;
import com.google.datastore.v1.Query;
import com.google.datastore.v1.QueryResultBatch;
import com.google.datastore.v1.RunQueryRequest;
import com.google.datastore.v1.RunQueryResponse;
import com.google.datastore.v1.client.Datastore;
import com.google.datastore.v1.client.DatastoreException;
import com.google.datastore.v1.client.DatastoreFactory;
import com.google.datastore.v1.client.DatastoreHelper;
import com.google.datastore.v1.client.DatastoreOptions;
import com.google.datastore.v1.client.QuerySplitter;
import com.google.protobuf.Int32Value;
import com.google.rpc.Code;
import google.registry.model.annotations.DeleteAfterMigration;
import java.io.Serializable;
import java.util.List;
import java.util.NoSuchElementException;
import javax.annotation.Nullable;
import org.apache.beam.sdk.extensions.gcp.options.GcpOptions;
import org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer;
import org.apache.beam.sdk.metrics.Counter;
import org.apache.beam.sdk.metrics.Metrics;
import org.apache.beam.sdk.options.PipelineOptions;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.PTransform;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.transforms.Reshuffle;
import org.apache.beam.sdk.transforms.display.DisplayData;
import org.apache.beam.sdk.transforms.display.HasDisplayData;
import org.apache.beam.sdk.util.BackOff;
import org.apache.beam.sdk.util.BackOffUtils;
import org.apache.beam.sdk.util.FluentBackoff;
import org.apache.beam.sdk.util.Sleeper;
import org.apache.beam.sdk.values.KV;
import org.apache.beam.sdk.values.PCollection;
import org.joda.time.Duration;
/**
* Contains an adaptation of {@link org.apache.beam.sdk.io.gcp.datastore.DatastoreV1.Read}. See
* {@link MultiRead} for details.
*/
@DeleteAfterMigration
public class DatastoreV1 {
// A package-private constructor to prevent direct instantiation from outside of this package
DatastoreV1() {}
/**
* Non-retryable errors. See https://cloud.google.com/datastore/docs/concepts/errors#Error_Codes .
*/
private static final ImmutableSet<Code> NON_RETRYABLE_ERRORS =
ImmutableSet.of(
Code.FAILED_PRECONDITION,
Code.INVALID_ARGUMENT,
Code.PERMISSION_DENIED,
Code.UNAUTHENTICATED);
/**
* Returns an empty {@link MultiRead} builder. Configure the source {@code projectId}, {@code
* query}, and optionally {@code namespace} and {@code numQuerySplits} using {@link
* MultiRead#withProjectId}, {@link MultiRead#withNamespace}, {@link
* MultiRead#withNumQuerySplits}.
*/
public static MultiRead read() {
return new AutoValue_DatastoreV1_MultiRead.Builder().setNumQuerySplits(0).build();
}
/**
* A {@link PTransform} that executes every Cloud SQL queries in a {@link PCollection } and reads
* their result rows as {@code Entity} objects.
*
* <p>This class is adapted from {@link org.apache.beam.sdk.io.gcp.datastore.DatastoreV1.Read}. It
* uses literal GQL queries in the input {@link PCollection} instead of a constant query provided
* to the builder. Only the {@link #expand} method is modified from the original. Everything else
* including comments have been copied verbatim.
*/
@AutoValue
public abstract static class MultiRead
extends PTransform<PCollection<String>, PCollection<Entity>> {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
/** An upper bound on the number of splits for a query. */
public static final int NUM_QUERY_SPLITS_MAX = 50000;
/** A lower bound on the number of splits for a query. */
static final int NUM_QUERY_SPLITS_MIN = 12;
/** Default bundle size of 64MB. */
static final long DEFAULT_BUNDLE_SIZE_BYTES = 64L * 1024L * 1024L;
/**
* Maximum number of results to request per query.
*
* <p>Must be set, or it may result in an I/O error when querying Cloud Datastore.
*/
static final int QUERY_BATCH_LIMIT = 500;
public abstract @Nullable String getProjectId();
public abstract @Nullable String getNamespace();
public abstract int getNumQuerySplits();
public abstract @Nullable String getLocalhost();
@Override
public abstract String toString();
abstract Builder toBuilder();
@AutoValue.Builder
abstract static class Builder {
abstract Builder setProjectId(String projectId);
abstract Builder setNamespace(String namespace);
abstract Builder setNumQuerySplits(int numQuerySplits);
abstract Builder setLocalhost(String localhost);
abstract MultiRead build();
}
/**
* Computes the number of splits to be performed on the given query by querying the estimated
* size from Cloud Datastore.
*/
static int getEstimatedNumSplits(Datastore datastore, Query query, @Nullable String namespace) {
int numSplits;
try {
long estimatedSizeBytes = getEstimatedSizeBytes(datastore, query, namespace);
logger.atInfo().log("Estimated size for the query is %d bytes.", estimatedSizeBytes);
numSplits =
(int)
Math.min(
NUM_QUERY_SPLITS_MAX,
Math.round(((double) estimatedSizeBytes) / DEFAULT_BUNDLE_SIZE_BYTES));
} catch (Exception e) {
logger.atWarning().withCause(e).log(
"Failed the fetch estimatedSizeBytes for query: %s", query);
// Fallback in case estimated size is unavailable.
numSplits = NUM_QUERY_SPLITS_MIN;
}
return Math.max(numSplits, NUM_QUERY_SPLITS_MIN);
}
/**
* Cloud Datastore system tables with statistics are periodically updated. This method fetches
* the latest timestamp (in microseconds) of statistics update using the {@code __Stat_Total__}
* table.
*/
private static long queryLatestStatisticsTimestamp(
Datastore datastore, @Nullable String namespace) throws DatastoreException {
Query.Builder query = Query.newBuilder();
// Note: namespace either being null or empty represents the default namespace, in which
// case we treat it as not provided by the user.
if (Strings.isNullOrEmpty(namespace)) {
query.addKindBuilder().setName("__Stat_Total__");
} else {
query.addKindBuilder().setName("__Stat_Ns_Total__");
}
query.addOrder(makeOrder("timestamp", DESCENDING));
query.setLimit(Int32Value.newBuilder().setValue(1));
RunQueryRequest request = makeRequest(query.build(), namespace);
RunQueryResponse response = datastore.runQuery(request);
QueryResultBatch batch = response.getBatch();
if (batch.getEntityResultsCount() == 0) {
throw new NoSuchElementException("Datastore total statistics unavailable");
}
Entity entity = batch.getEntityResults(0).getEntity();
return entity.getProperties().get("timestamp").getTimestampValue().getSeconds() * 1000000;
}
/** Retrieve latest table statistics for a given kind, namespace, and datastore. */
private static Entity getLatestTableStats(
String ourKind, @Nullable String namespace, Datastore datastore) throws DatastoreException {
long latestTimestamp = queryLatestStatisticsTimestamp(datastore, namespace);
logger.atInfo().log("Latest stats timestamp for kind %s is %s.", ourKind, latestTimestamp);
Query.Builder queryBuilder = Query.newBuilder();
if (Strings.isNullOrEmpty(namespace)) {
queryBuilder.addKindBuilder().setName("__Stat_Kind__");
} else {
queryBuilder.addKindBuilder().setName("__Stat_Ns_Kind__");
}
queryBuilder.setFilter(
makeAndFilter(
makeFilter("kind_name", EQUAL, makeValue(ourKind).build()).build(),
makeFilter("timestamp", EQUAL, makeValue(latestTimestamp).build()).build()));
RunQueryRequest request = makeRequest(queryBuilder.build(), namespace);
long now = System.currentTimeMillis();
RunQueryResponse response = datastore.runQuery(request);
logger.atFine().log(
"Query for per-kind statistics took %d ms.", System.currentTimeMillis() - now);
QueryResultBatch batch = response.getBatch();
if (batch.getEntityResultsCount() == 0) {
throw new NoSuchElementException(
"Datastore statistics for kind " + ourKind + " unavailable");
}
return batch.getEntityResults(0).getEntity();
}
/**
* Get the estimated size of the data returned by the given query.
*
* <p>Cloud Datastore provides no way to get a good estimate of how large the result of a query
* entity kind being queried, using the __Stat_Kind__ system table, assuming exactly 1 kind is
* specified in the query.
*
* <p>See https://cloud.google.com/datastore/docs/concepts/stats.
*/
static long getEstimatedSizeBytes(Datastore datastore, Query query, @Nullable String namespace)
throws DatastoreException {
String ourKind = query.getKind(0).getName();
Entity entity = getLatestTableStats(ourKind, namespace, datastore);
return entity.getProperties().get("entity_bytes").getIntegerValue();
}
private static PartitionId.Builder forNamespace(@Nullable String namespace) {
PartitionId.Builder partitionBuilder = PartitionId.newBuilder();
// Namespace either being null or empty represents the default namespace.
// Datastore Client libraries expect users to not set the namespace proto field in
// either of these cases.
if (!Strings.isNullOrEmpty(namespace)) {
partitionBuilder.setNamespaceId(namespace);
}
return partitionBuilder;
}
/** Builds a {@link RunQueryRequest} from the {@code query} and {@code namespace}. */
static RunQueryRequest makeRequest(Query query, @Nullable String namespace) {
return RunQueryRequest.newBuilder()
.setQuery(query)
.setPartitionId(forNamespace(namespace))
.build();
}
/** Builds a {@link RunQueryRequest} from the {@code GqlQuery} and {@code namespace}. */
private static RunQueryRequest makeRequest(GqlQuery gqlQuery, @Nullable String namespace) {
return RunQueryRequest.newBuilder()
.setGqlQuery(gqlQuery)
.setPartitionId(forNamespace(namespace))
.build();
}
/**
* A helper function to get the split queries, taking into account the optional {@code
* namespace}.
*/
private static List<Query> splitQuery(
Query query,
@Nullable String namespace,
Datastore datastore,
QuerySplitter querySplitter,
int numSplits)
throws DatastoreException {
// If namespace is set, include it in the split request so splits are calculated accordingly.
return querySplitter.getSplits(query, forNamespace(namespace).build(), numSplits, datastore);
}
/**
* Translates a Cloud Datastore gql query string to {@link Query}.
*
* <p>Currently, the only way to translate a gql query string to a Query is to run the query
* against Cloud Datastore and extract the {@code Query} from the response. To prevent reading
* any data, we set the {@code LIMIT} to 0 but if the gql query already has a limit set, we
* catch the exception with {@code INVALID_ARGUMENT} error code and retry the translation
* without the zero limit.
*
* <p>Note: This may result in reading actual data from Cloud Datastore but the service has a
* cap on the number of entities returned for a single rpc request, so this should not be a
* problem in practice.
*/
private static Query translateGqlQueryWithLimitCheck(
String gql, Datastore datastore, String namespace) throws DatastoreException {
String gqlQueryWithZeroLimit = gql + " LIMIT 0";
try {
Query translatedQuery = translateGqlQuery(gqlQueryWithZeroLimit, datastore, namespace);
// Clear the limit that we set.
return translatedQuery.toBuilder().clearLimit().build();
} catch (DatastoreException e) {
// Note: There is no specific error code or message to detect if the query already has a
// limit, so we just check for INVALID_ARGUMENT and assume that that the query might have
// a limit already set.
if (e.getCode() == Code.INVALID_ARGUMENT) {
logger.atWarning().log(
"Failed to translate Gql query '%s': %s", gqlQueryWithZeroLimit, e.getMessage());
logger.atWarning().log(
"User query might have a limit already set, so trying without zero limit.");
// Retry without the zero limit.
return translateGqlQuery(gql, datastore, namespace);
} else {
throw e;
}
}
}
/** Translates a gql query string to {@link Query}. */
private static Query translateGqlQuery(String gql, Datastore datastore, String namespace)
throws DatastoreException {
logger.atInfo().log("Translating gql %s", gql);
GqlQuery gqlQuery = GqlQuery.newBuilder().setQueryString(gql).setAllowLiterals(true).build();
RunQueryRequest req = makeRequest(gqlQuery, namespace);
return datastore.runQuery(req).getQuery();
}
/**
* Returns a new {@link MultiRead} that reads from the Cloud Datastore for the specified
* project.
*/
public MultiRead withProjectId(String projectId) {
checkArgument(projectId != null, "projectId can not be null");
return toBuilder().setProjectId(projectId).build();
}
/** Returns a new {@link MultiRead} that reads from the given namespace. */
public MultiRead withNamespace(String namespace) {
return toBuilder().setNamespace(namespace).build();
}
/**
* Returns a new {@link MultiRead} that reads by splitting the given {@code query} into {@code
* numQuerySplits}.
*
* <p>The semantics for the query splitting is defined below:
*
* <ul>
* <li>Any value less than or equal to 0 will be ignored, and the number of splits will be
* chosen dynamically at runtime based on the query data size.
* <li>Any value greater than {@link MultiRead#NUM_QUERY_SPLITS_MAX} will be capped at {@code
* NUM_QUERY_SPLITS_MAX}.
* <li>If the {@code query} has a user limit set, then {@code numQuerySplits} will be ignored
* and no split will be performed.
* <li>Under certain cases Cloud Datastore is unable to split query to the requested number of
* splits. In such cases we just use whatever the Cloud Datastore returns.
* </ul>
*/
public MultiRead withNumQuerySplits(int numQuerySplits) {
return toBuilder()
.setNumQuerySplits(Math.min(Math.max(numQuerySplits, 0), NUM_QUERY_SPLITS_MAX))
.build();
}
/**
* Returns a new {@link MultiRead} that reads from a Datastore Emulator running at the given
* localhost address.
*/
public MultiRead withLocalhost(String localhost) {
return toBuilder().setLocalhost(localhost).build();
}
/** Returns Number of entities available for reading. */
public long getNumEntities(
PipelineOptions options, String ourKind, @Nullable String namespace) {
try {
V1Options v1Options = V1Options.from(getProjectId(), getNamespace(), getLocalhost());
V1DatastoreFactory datastoreFactory = new V1DatastoreFactory();
Datastore datastore =
datastoreFactory.getDatastore(
options, v1Options.getProjectId(), v1Options.getLocalhost());
Entity entity = getLatestTableStats(ourKind, namespace, datastore);
return entity.getProperties().get("count").getIntegerValue();
} catch (Exception e) {
return -1;
}
}
@Override
public PCollection<Entity> expand(PCollection<String> gqlQueries) {
checkArgument(getProjectId() != null, "projectId cannot be null");
V1Options v1Options = V1Options.from(getProjectId(), getNamespace(), getLocalhost());
/*
* This composite transform involves the following steps:
* 1. Apply a {@link ParDo} that translates each query in {@code gqlQueries} into a {@code
* query}.
*
* 2. A {@link ParDo} splits the resulting query into {@code numQuerySplits} and
* assign each split query a unique {@code Integer} as the key. The resulting output is
* of the type {@code PCollection<KV<Integer, Query>>}.
*
* If the value of {@code numQuerySplits} is less than or equal to 0, then the number of
* splits will be computed dynamically based on the size of the data for the {@code query}.
*
* 3. The resulting {@code PCollection} is sharded using a {@link GroupByKey} operation. The
* queries are extracted from they {@code KV<Integer, Iterable<Query>>} and flattened to
* output a {@code PCollection<Query>}.
*
* 4. In the third step, a {@code ParDo} reads entities for each query and outputs
* a {@code PCollection<Entity>}.
*/
PCollection<Query> inputQuery =
gqlQueries.apply(ParDo.of(new GqlQueryTranslateFn(v1Options)));
return inputQuery
.apply("Split", ParDo.of(new SplitQueryFn(v1Options, getNumQuerySplits())))
.apply("Reshuffle", Reshuffle.viaRandomKey())
.apply("Read", ParDo.of(new ReadFn(v1Options)));
}
@Override
public void populateDisplayData(DisplayData.Builder builder) {
super.populateDisplayData(builder);
builder
.addIfNotNull(DisplayData.item("projectId", getProjectId()).withLabel("ProjectId"))
.addIfNotNull(DisplayData.item("namespace", getNamespace()).withLabel("Namespace"));
}
private static class V1Options implements HasDisplayData, Serializable {
private final String project;
private final @Nullable String namespace;
private final @Nullable String localhost;
private V1Options(String project, @Nullable String namespace, @Nullable String localhost) {
this.project = project;
this.namespace = namespace;
this.localhost = localhost;
}
public static V1Options from(
String projectId, @Nullable String namespace, @Nullable String localhost) {
return new V1Options(projectId, namespace, localhost);
}
public String getProjectId() {
return project;
}
public @Nullable String getNamespace() {
return namespace;
}
public @Nullable String getLocalhost() {
return localhost;
}
@Override
public void populateDisplayData(DisplayData.Builder builder) {
builder
.addIfNotNull(DisplayData.item("projectId", getProjectId()).withLabel("ProjectId"))
.addIfNotNull(DisplayData.item("namespace", getNamespace()).withLabel("Namespace"));
}
}
/** A DoFn that translates a Cloud Datastore gql query string to {@code Query}. */
static class GqlQueryTranslateFn extends DoFn<String, Query> {
private final V1Options v1Options;
private transient Datastore datastore;
private final V1DatastoreFactory datastoreFactory;
GqlQueryTranslateFn(V1Options options) {
this(options, new V1DatastoreFactory());
}
GqlQueryTranslateFn(V1Options options, V1DatastoreFactory datastoreFactory) {
this.v1Options = options;
this.datastoreFactory = datastoreFactory;
}
@StartBundle
public void startBundle(StartBundleContext c) {
datastore =
datastoreFactory.getDatastore(
c.getPipelineOptions(), v1Options.getProjectId(), v1Options.getLocalhost());
}
@ProcessElement
public void processElement(ProcessContext c) throws Exception {
String gqlQuery = c.element();
logger.atInfo().log("User query: '%s'.", gqlQuery);
Query query =
translateGqlQueryWithLimitCheck(gqlQuery, datastore, v1Options.getNamespace());
logger.atInfo().log("User gql query translated to Query(%s).", query);
c.output(query);
}
}
/**
* A {@link DoFn} that splits a given query into multiple sub-queries, assigns them unique keys
* and outputs them as {@link KV}.
*/
private static class SplitQueryFn extends DoFn<Query, Query> {
private final V1Options options;
// number of splits to make for a given query
private final int numSplits;
private final V1DatastoreFactory datastoreFactory;
// Datastore client
private transient Datastore datastore;
// Query splitter
private transient QuerySplitter querySplitter;
public SplitQueryFn(V1Options options, int numSplits) {
this(options, numSplits, new V1DatastoreFactory());
}
private SplitQueryFn(V1Options options, int numSplits, V1DatastoreFactory datastoreFactory) {
this.options = options;
this.numSplits = numSplits;
this.datastoreFactory = datastoreFactory;
}
@StartBundle
public void startBundle(StartBundleContext c) {
datastore =
datastoreFactory.getDatastore(
c.getPipelineOptions(), options.getProjectId(), options.getLocalhost());
querySplitter = datastoreFactory.getQuerySplitter();
}
@ProcessElement
public void processElement(ProcessContext c) {
Query query = c.element();
// If query has a user set limit, then do not split.
if (query.hasLimit()) {
c.output(query);
return;
}
int estimatedNumSplits;
// Compute the estimated numSplits if numSplits is not specified by the user.
if (numSplits <= 0) {
estimatedNumSplits = getEstimatedNumSplits(datastore, query, options.getNamespace());
} else {
estimatedNumSplits = numSplits;
}
logger.atInfo().log("Splitting the query into %d splits.", estimatedNumSplits);
List<Query> querySplits;
try {
querySplits =
splitQuery(
query, options.getNamespace(), datastore, querySplitter, estimatedNumSplits);
} catch (Exception e) {
logger.atWarning().log("Unable to parallelize the given query: %s", query, e);
querySplits = ImmutableList.of(query);
}
// assign unique keys to query splits.
for (Query subquery : querySplits) {
c.output(subquery);
}
}
@Override
public void populateDisplayData(DisplayData.Builder builder) {
super.populateDisplayData(builder);
builder.include("options", options);
if (numSplits > 0) {
builder.add(
DisplayData.item("numQuerySplits", numSplits)
.withLabel("Requested number of Query splits"));
}
}
}
/** A {@link DoFn} that reads entities from Cloud Datastore for each query. */
private static class ReadFn extends DoFn<Query, Entity> {
private final V1Options options;
private final V1DatastoreFactory datastoreFactory;
// Datastore client
private transient Datastore datastore;
private final Counter rpcErrors = Metrics.counter(ReadFn.class, "datastoreRpcErrors");
private final Counter rpcSuccesses = Metrics.counter(ReadFn.class, "datastoreRpcSuccesses");
private static final int MAX_RETRIES = 5;
private static final FluentBackoff RUNQUERY_BACKOFF =
FluentBackoff.DEFAULT
.withMaxRetries(MAX_RETRIES)
.withInitialBackoff(Duration.standardSeconds(5));
public ReadFn(V1Options options) {
this(options, new V1DatastoreFactory());
}
private ReadFn(V1Options options, V1DatastoreFactory datastoreFactory) {
this.options = options;
this.datastoreFactory = datastoreFactory;
}
@StartBundle
public void startBundle(StartBundleContext c) {
datastore =
datastoreFactory.getDatastore(
c.getPipelineOptions(), options.getProjectId(), options.getLocalhost());
}
private RunQueryResponse runQueryWithRetries(RunQueryRequest request) throws Exception {
Sleeper sleeper = Sleeper.DEFAULT;
BackOff backoff = RUNQUERY_BACKOFF.backoff();
while (true) {
try {
RunQueryResponse response = datastore.runQuery(request);
rpcSuccesses.inc();
return response;
} catch (DatastoreException exception) {
rpcErrors.inc();
if (NON_RETRYABLE_ERRORS.contains(exception.getCode())) {
throw exception;
}
if (!BackOffUtils.next(sleeper, backoff)) {
logger.atSevere().log("Aborting after %d retries.", MAX_RETRIES);
throw exception;
}
}
}
}
/** Read and output entities for the given query. */
@ProcessElement
public void processElement(ProcessContext context) throws Exception {
Query query = context.element();
String namespace = options.getNamespace();
int userLimit = query.hasLimit() ? query.getLimit().getValue() : Integer.MAX_VALUE;
boolean moreResults = true;
QueryResultBatch currentBatch = null;
while (moreResults) {
Query.Builder queryBuilder = query.toBuilder();
queryBuilder.setLimit(
Int32Value.newBuilder().setValue(Math.min(userLimit, QUERY_BATCH_LIMIT)));
if (currentBatch != null && !currentBatch.getEndCursor().isEmpty()) {
queryBuilder.setStartCursor(currentBatch.getEndCursor());
}
RunQueryRequest request = makeRequest(queryBuilder.build(), namespace);
RunQueryResponse response = runQueryWithRetries(request);
currentBatch = response.getBatch();
// MORE_RESULTS_AFTER_LIMIT is not implemented yet:
// https://groups.google.com/forum/#!topic/gcd-discuss/iNs6M1jA2Vw, so
// use result count to determine if more results might exist.
int numFetch = currentBatch.getEntityResultsCount();
if (query.hasLimit()) {
verify(
userLimit >= numFetch,
"Expected userLimit %s >= numFetch %s, because query limit %s must be <= userLimit",
userLimit,
numFetch,
query.getLimit());
userLimit -= numFetch;
}
// output all the entities from the current batch.
for (EntityResult entityResult : currentBatch.getEntityResultsList()) {
context.output(entityResult.getEntity());
}
// Check if we have more entities to be read.
moreResults =
// User-limit does not exist (so userLimit == MAX_VALUE) and/or has not been satisfied
(userLimit > 0)
// All indications from the API are that there are/may be more results.
&& ((numFetch == QUERY_BATCH_LIMIT)
|| (currentBatch.getMoreResults() == NOT_FINISHED));
}
}
@Override
public void populateDisplayData(DisplayData.Builder builder) {
super.populateDisplayData(builder);
builder.include("options", options);
}
}
}
/**
* A wrapper factory class for Cloud Datastore singleton classes {@link DatastoreFactory} and
* {@link QuerySplitter}
*
* <p>{@link DatastoreFactory} and {@link QuerySplitter} are not java serializable, hence wrapping
* them under this class, which implements {@link Serializable}.
*/
private static class V1DatastoreFactory implements Serializable {
/** Builds a Cloud Datastore client for the given pipeline options and project. */
public Datastore getDatastore(PipelineOptions pipelineOptions, String projectId) {
return getDatastore(pipelineOptions, projectId, null);
}
/**
* Builds a Cloud Datastore client for the given pipeline options, project and an optional
* locahost.
*/
public Datastore getDatastore(
PipelineOptions pipelineOptions, String projectId, @Nullable String localhost) {
Credentials credential = pipelineOptions.as(GcpOptions.class).getGcpCredential();
HttpRequestInitializer initializer;
if (credential != null) {
initializer =
new ChainingHttpRequestInitializer(
new HttpCredentialsAdapter(credential), new RetryHttpRequestInitializer());
} else {
initializer = new RetryHttpRequestInitializer();
}
DatastoreOptions.Builder builder =
new DatastoreOptions.Builder().projectId(projectId).initializer(initializer);
if (localhost != null) {
builder.localHost(localhost);
} else {
builder.host("batch-datastore.googleapis.com");
}
return DatastoreFactory.get().create(builder.build());
}
/** Builds a Cloud Datastore {@link QuerySplitter}. */
public QuerySplitter getQuerySplitter() {
return DatastoreHelper.getQuerySplitter();
}
}
}

View File

@@ -1,119 +0,0 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Strings.isNullOrEmpty;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Streams;
import google.registry.model.annotations.DeleteAfterMigration;
import org.joda.time.DateTime;
/**
* Helpers for determining the fully qualified paths to Nomulus backup files. A backup consists of a
* Datastore export and Nomulus CommitLogs that overlap with the export.
*/
@DeleteAfterMigration
public final class BackupPaths {
private BackupPaths() {}
private static final String WILDCARD_CHAR = "*";
private static final String EXPORT_PATTERN_TEMPLATE = "%s/all_namespaces/kind_%s/output-%s";
public static final String COMMIT_LOG_NAME_PREFIX = "commit_diff_until_";
private static final String COMMIT_LOG_PATTERN_TEMPLATE = "%s/" + COMMIT_LOG_NAME_PREFIX + "*";
/**
* Pattern of the per-project file with Cloud SQL connection information. To get a concrete path,
* user needs to provide the name of the environment, alpha, crash, sandbox, or production. This
* file is meant for applications without access to secrets stored in Datastore.
*
* <p>In production, this is an base-64 encoded encrypted file with one line, which contains
* space-separated values of Cloud SQL instance name, login, and password.
*
* <p>A plain text may be used for tests to a local database. Replace Cloud SQL instance name with
* JDBC URL.
*/
private static final String SQL_CONN_INFO_FILE_PATTERN =
"gs://domain-registry-dev-deploy/cloudsql-credentials/%s/admin_credential.enc";
private static final ImmutableSet<String> ALLOWED_ENV =
ImmutableSet.of("alpha", "crash", "sandbox", "production");
/**
* Returns a regex pattern that matches all Datastore export files of a given {@code kind}.
*
* @param exportDir path to the top directory of a Datastore export
* @param kind the 'kind' of the Datastore entity
*/
public static String getExportFileNamePattern(String exportDir, String kind) {
checkArgument(!isNullOrEmpty(exportDir), "Null or empty exportDir.");
checkArgument(!isNullOrEmpty(kind), "Null or empty kind.");
return String.format(EXPORT_PATTERN_TEMPLATE, exportDir, kind, WILDCARD_CHAR);
}
/**
* Returns an {@link ImmutableList} of regex patterns that match all Datastore export files of the
* given {@code kinds}.
*
* @param exportDir path to the top directory of a Datastore export
* @param kinds all entity 'kinds' to be matched
*/
public static ImmutableList<String> getExportFilePatterns(
String exportDir, Iterable<String> kinds) {
checkNotNull(kinds, "kinds");
return Streams.stream(kinds)
.map(kind -> getExportFileNamePattern(exportDir, kind))
.collect(ImmutableList.toImmutableList());
}
/**
* Returns the fully qualified path of a Datastore export file with the given {@code kind} and
* {@code shard}.
*
* @param exportDir path to the top directory of a Datastore export
* @param kind the 'kind' of the Datastore entity
* @param shard an integer suffix of the file name
*/
public static String getExportFileNameByShard(String exportDir, String kind, int shard) {
checkArgument(!isNullOrEmpty(exportDir), "Null or empty exportDir.");
checkArgument(!isNullOrEmpty(kind), "Null or empty kind.");
checkArgument(shard >= 0, "Negative shard %s not allowed.", shard);
return String.format(EXPORT_PATTERN_TEMPLATE, exportDir, kind, shard);
}
/** Returns an {@link ImmutableList} of regex patterns that match all CommitLog files. */
public static ImmutableList<String> getCommitLogFilePatterns(String commitLogDir) {
return ImmutableList.of(String.format(COMMIT_LOG_PATTERN_TEMPLATE, commitLogDir));
}
/** Gets the Commit timestamp from a CommitLog file name. */
public static DateTime getCommitLogTimestamp(String fileName) {
checkArgument(!isNullOrEmpty(fileName), "Null or empty fileName.");
int start = fileName.lastIndexOf(COMMIT_LOG_NAME_PREFIX);
checkArgument(start >= 0, "Illegal file name %s.", fileName);
return DateTime.parse(fileName.substring(start + COMMIT_LOG_NAME_PREFIX.length()));
}
public static ImmutableList<String> getCloudSQLCredentialFilePatterns(String environmentName) {
checkArgument(
ALLOWED_ENV.contains(environmentName), "Invalid environment name %s", environmentName);
return ImmutableList.of(String.format(SQL_CONN_INFO_FILE_PATTERN, environmentName));
}
}

View File

@@ -1,78 +0,0 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import com.google.appengine.api.datastore.Entity;
import google.registry.model.annotations.DeleteAfterMigration;
import java.util.Objects;
/** Helper for manipulating {@code DomainBase} when migrating from Datastore to SQL database */
@DeleteAfterMigration
final class DomainBaseUtil {
private DomainBaseUtil() {}
/**
* Removes properties that contain foreign keys from a Datastore {@link Entity} that represents an
* Ofy {@link google.registry.model.domain.DomainBase}. This breaks the cycle of foreign key
* constraints between entity kinds, allowing {@code DomainBases} to be inserted into the SQL
* database. See {@link InitSqlPipeline} for a use case, where the full {@code DomainBases} are
* written again during the last stage of the pipeline.
*
* <p>The returned object may be in bad state. Specifically, {@link
* google.registry.model.eppcommon.StatusValue#INACTIVE} is not added after name servers are
* removed. This only impacts tests that manipulate Datastore entities directly.
*
* <p>This operation is performed on an Datastore {@link Entity} instead of Ofy Java object
* because Objectify requires access to a Datastore service when converting an Ofy object to a
* Datastore {@code Entity}. If we insist on working with Objectify objects, we face a few
* unsatisfactory options:
*
* <ul>
* <li>Connect to our production Datastore, which incurs unnecessary security and code health
* risk.
* <li>Connect to a separate real Datastore instance, which is a waster and overkill.
* <li>Use an in-memory test Datastore, which is a project health risk in that the test
* Datastore would be added to Nomulus' production binary unless we create a separate
* project for this pipeline.
* </ul>
*
* <p>Given our use case, operating on Datastore entities is the best option.
*
* @throws IllegalArgumentException if input does not represent a DomainBase
*/
static Entity removeBillingAndPollAndHosts(Entity domainBase) {
checkNotNull(domainBase, "domainBase");
checkArgument(
Objects.equals(domainBase.getKind(), "DomainBase"),
"Expecting DomainBase, got %s",
domainBase.getKind());
Entity clone = domainBase.clone();
clone.removeProperty("autorenewBillingEvent");
clone.removeProperty("autorenewPollMessage");
clone.removeProperty("deletePollMessage");
clone.removeProperty("nsHosts");
domainBase.getProperties().keySet().stream()
.filter(s -> s.startsWith("transferData."))
.forEach(s -> clone.removeProperty(s));
domainBase.getProperties().keySet().stream()
.filter(s -> s.startsWith("gracePeriods."))
.forEach(s -> clone.removeProperty(s));
return clone;
}
}

View File

@@ -1,242 +0,0 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import static com.google.common.base.Preconditions.checkArgument;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import google.registry.backup.VersionedEntity;
import google.registry.beam.common.RegistryJpaIO;
import google.registry.beam.initsql.Transforms.RemoveDomainBaseForeignKeys;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.billing.BillingEvent;
import google.registry.model.common.Cursor;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.DomainBase;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.host.HostResource;
import google.registry.model.poll.PollMessage;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarContact;
import google.registry.model.reporting.HistoryEntry;
import google.registry.model.tld.Registry;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import java.io.Serializable;
import java.util.Collection;
import java.util.Optional;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.PipelineResult;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.transforms.Wait;
import org.apache.beam.sdk.values.PCollection;
import org.apache.beam.sdk.values.PCollectionTuple;
import org.apache.beam.sdk.values.TupleTag;
import org.joda.time.DateTime;
/**
* A BEAM pipeline that populates a SQL database with data from a Datastore backup.
*
* <p>This pipeline migrates EPP resources and related entities that cross-reference each other. To
* avoid violating foreign key constraints, writes to SQL are ordered by entity kinds. In addition,
* the {@link DomainBase} kind is written twice (see details below). The write order is presented
* below. Although some kinds can be written concurrently, e.g. {@code ContactResource} and {@code
* RegistrarContact}, we do not expect any performance benefit since the limiting resource is the
* number of JDBC connections. Google internal users may refer to <a
* href="http://go/registry-r3-init-sql">the design doc</a> for more information.
*
* <ol>
* <li>{@link Registry}: Assumes that {@code PremiumList} and {@code ReservedList} have been set
* up in the SQL database.
* <li>{@link Cursor}: Logically can depend on {@code Registry}, but without foreign key.
* <li>{@link Registrar}: Logically depends on {@code Registry}, Foreign key not modeled yet.
* <li>{@link ContactResource}: references {@code Registrar}
* <li>{@link RegistrarContact}: references {@code Registrar}.
* <li>Cleansed {@link DomainBase}: with references to {@code BillingEvent}, {@code Recurring},
* {@code Cancellation} and {@code HostResource} removed, still references {@code Registrar}
* and {@code ContactResource}. The removal breaks circular Foreign Key references.
* <li>{@link HostResource}: references {@code DomainBase}.
* <li>{@link HistoryEntry}: maps to one of three SQL entity types and may reference {@code
* Registrar}, {@code ContactResource}, {@code HostResource}, and {@code DomainBase}.
* <li>{@link AllocationToken}: references {@code HistoryEntry}.
* <li>{@link BillingEvent.Recurring}: references {@code Registrar}, {@code DomainBase} and {@code
* HistoryEntry}.
* <li>{@link BillingEvent.OneTime}: references {@code Registrar}, {@code DomainBase}, {@code
* BillingEvent.Recurring}, {@code HistoryEntry} and {@code AllocationToken}.
* <li>{@link BillingEvent.Cancellation}: references {@code Registrar}, {@code DomainBase}, {@code
* BillingEvent.Recurring}, {@code BillingEvent.OneTime}, and {@code HistoryEntry}.
* <li>{@link PollMessage}: references {@code Registrar}, {@code DomainBase}, {@code
* ContactResource}, {@code HostResource}, and {@code HistoryEntry}.
* <li>{@link DomainBase}, original copy from Datastore.
* </ol>
*
* <p>This pipeline expects that the source Datastore has at least one entity in each of the types
* above. This assumption allows us to construct a simpler pipeline graph that can be visually
* examined, and is true in all intended use cases. However, tests must not violate this assumption
* when setting up data, otherwise they may run into foreign key constraint violations. The reason
* is that this pipeline uses the {@link Wait} transform to order the persistence by entity type.
* However, the wait is skipped if the target type has no data, resulting in subsequent entity types
* starting prematurely. E.g., if a Datastore has no {@code RegistrarContact} entities, the pipeline
* may start writing {@code DomainBase} entities before all {@code Registry}, {@code Registrar} and
* {@code ContactResource} entities have been persisted.
*/
@DeleteAfterMigration
public class InitSqlPipeline implements Serializable {
/**
* Datastore kinds to be written to the SQL database before the cleansed version of {@link
* DomainBase}.
*/
private static final ImmutableList<Class<?>> PHASE_ONE_ORDERED =
ImmutableList.of(
Registry.class,
Cursor.class,
Registrar.class,
ContactResource.class,
RegistrarContact.class);
/**
* Datastore kinds to be written to the SQL database after the cleansed version of {@link
* DomainBase}.
*/
private static final ImmutableList<Class<?>> PHASE_TWO_ORDERED =
ImmutableList.of(
HostResource.class,
HistoryEntry.class,
AllocationToken.class,
BillingEvent.Recurring.class,
BillingEvent.OneTime.class,
BillingEvent.Cancellation.class,
PollMessage.class,
DomainBase.class);
private final InitSqlPipelineOptions options;
InitSqlPipeline(InitSqlPipelineOptions options) {
this.options = options;
}
PipelineResult run() {
return run(Pipeline.create(options));
}
@VisibleForTesting
PipelineResult run(Pipeline pipeline) {
setupPipeline(pipeline);
return pipeline.run();
}
@VisibleForTesting
void setupPipeline(Pipeline pipeline) {
options.setIsolationOverride(TransactionIsolationLevel.TRANSACTION_READ_UNCOMMITTED);
PCollectionTuple datastoreSnapshot =
pipeline.apply(
"Load Datastore snapshot",
Transforms.loadDatastoreSnapshot(
options.getDatastoreExportDir(),
options.getCommitLogDir(),
DateTime.parse(options.getCommitLogStartTimestamp()),
DateTime.parse(options.getCommitLogEndTimestamp()),
ImmutableSet.<String>builder()
.add("DomainBase")
.addAll(toKindStrings(PHASE_ONE_ORDERED))
.addAll(toKindStrings(PHASE_TWO_ORDERED))
.build()));
// Set up the pipeline to write entity kinds from PHASE_ONE_ORDERED to SQL. Return a object
// that signals the completion of the phase.
PCollection<Void> blocker =
scheduleOnePhaseWrites(datastoreSnapshot, PHASE_ONE_ORDERED, Optional.empty(), null);
blocker =
writeToSql(
"DomainBase without circular foreign keys",
removeDomainBaseForeignKeys(datastoreSnapshot)
.apply("Wait on phase one", Wait.on(blocker)));
// Set up the pipeline to write entity kinds from PHASE_TWO_ORDERED to SQL. This phase won't
// start until all cleansed DomainBases have been written (started by line above).
scheduleOnePhaseWrites(
datastoreSnapshot, PHASE_TWO_ORDERED, Optional.of(blocker), "DomainBaseNoFkeys");
}
private PCollection<VersionedEntity> removeDomainBaseForeignKeys(
PCollectionTuple datastoreSnapshot) {
PCollection<VersionedEntity> domainBases =
datastoreSnapshot.get(Transforms.createTagForKind("DomainBase"));
return domainBases.apply(
"Remove circular foreign keys from DomainBase",
ParDo.of(new RemoveDomainBaseForeignKeys()));
}
/**
* Sets up the pipeline to write entities in {@code entityClasses} to SQL. Entities are written
* one kind at a time based on each kind's position in {@code entityClasses}. Concurrency exists
* within each kind.
*
* @param datastoreSnapshot the Datastore snapshot of all data to be migrated to SQL
* @param entityClasses the entity types in write order
* @param blockingPCollection the pipeline stage that blocks this phase
* @param blockingTag description of the stage (if exists) that blocks this phase. Needed for
* generating unique transform ids
* @return the output {@code PCollection} from the writing of the last entity kind. Other parts of
* the pipeline can {@link Wait} on this object
*/
private PCollection<Void> scheduleOnePhaseWrites(
PCollectionTuple datastoreSnapshot,
Collection<Class<?>> entityClasses,
Optional<PCollection<Void>> blockingPCollection,
String blockingTag) {
checkArgument(!entityClasses.isEmpty(), "Each phase must have at least one kind.");
ImmutableList<TupleTag<VersionedEntity>> tags =
toKindStrings(entityClasses).stream()
.map(Transforms::createTagForKind)
.collect(ImmutableList.toImmutableList());
PCollection<Void> prev = blockingPCollection.orElse(null);
String prevTag = blockingTag;
for (TupleTag<VersionedEntity> tag : tags) {
PCollection<VersionedEntity> curr = datastoreSnapshot.get(tag);
if (prev != null) {
curr = curr.apply("Wait on " + prevTag, Wait.on(prev));
}
prev = writeToSql(tag.getId(), curr);
prevTag = tag.getId();
}
return prev;
}
private PCollection<Void> writeToSql(String transformId, PCollection<VersionedEntity> data) {
return data.apply(
"Write to Sql: " + transformId,
RegistryJpaIO.<VersionedEntity>write()
.withName(transformId)
.withBatchSize(options.getSqlWriteBatchSize())
.withShards(options.getSqlWriteShards())
.withJpaConverter(Transforms::convertVersionedEntityToSqlEntity)
.disableUpdateAutoTimestamp());
}
private static ImmutableList<String> toKindStrings(Collection<Class<?>> entityClasses) {
return entityClasses.stream().map(Key::getKind).collect(ImmutableList.toImmutableList());
}
public static void main(String[] args) {
InitSqlPipelineOptions options =
PipelineOptionsFactory.fromArgs(args).withValidation().as(InitSqlPipelineOptions.class);
new InitSqlPipeline(options).run();
}
}

View File

@@ -1,47 +0,0 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import google.registry.beam.common.RegistryPipelineOptions;
import google.registry.model.annotations.DeleteAfterMigration;
import org.apache.beam.sdk.options.Description;
import org.apache.beam.sdk.options.Validation;
/** Pipeline options for {@link InitSqlPipeline} */
@DeleteAfterMigration
public interface InitSqlPipelineOptions extends RegistryPipelineOptions {
@Description("The root directory of the export to load.")
String getDatastoreExportDir();
void setDatastoreExportDir(String datastoreExportDir);
@Description("The directory that contains all CommitLog files.")
String getCommitLogDir();
void setCommitLogDir(String commitLogDir);
@Description("The earliest CommitLogs to load, in ISO8601 format.")
@Validation.Required
String getCommitLogStartTimestamp();
void setCommitLogStartTimestamp(String commitLogStartTimestamp);
@Description("The latest CommitLogs to load, in ISO8601 format.")
@Validation.Required
String getCommitLogEndTimestamp();
void setCommitLogEndTimestamp(String commitLogEndTimestamp);
}

View File

@@ -1,17 +0,0 @@
## Summary
This package contains a BEAM pipeline that populates a Cloud SQL database from a
Datastore backup. The pipeline uses an unsynchronized Datastore export and
overlapping CommitLogs generated by the Nomulus server to recreate a consistent
Datastore snapshot, and writes the data to a Cloud SQL instance.
## Pipeline Visualization
The golden flow graph of the InitSqlPipeline is saved both as a text-base
[DOT file](../../../../../../test/resources/google/registry/beam/initsql/pipeline_golden.dot)
and a
[.png file](../../../../../../test/resources/google/registry/beam/initsql/pipeline_golden.png).
A test compares the flow graph of the current pipeline with the golden graph,
and will fail if changes are detected. When this happens, run the Gradle task
':core:updateInitSqlPipelineGraph' to update the golden files and review the
changes.

View File

@@ -1,485 +0,0 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.beam.initsql.BackupPaths.getCommitLogTimestamp;
import static google.registry.beam.initsql.BackupPaths.getExportFilePatterns;
import static google.registry.model.ofy.ObjectifyService.auditedOfy;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import static google.registry.util.DomainNameUtils.canonicalizeDomainName;
import static java.util.Comparator.comparing;
import static org.apache.beam.sdk.values.TypeDescriptors.kvs;
import static org.apache.beam.sdk.values.TypeDescriptors.strings;
import avro.shaded.com.google.common.collect.Iterators;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.EntityTranslator;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Streams;
import google.registry.backup.CommitLogImports;
import google.registry.backup.VersionedEntity;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.model.billing.BillingEvent.Flag;
import google.registry.model.billing.BillingEvent.Reason;
import google.registry.model.domain.DomainBase;
import google.registry.model.replay.DatastoreAndSqlEntity;
import google.registry.model.replay.SqlEntity;
import google.registry.model.reporting.HistoryEntry;
import google.registry.tools.LevelDbLogReader;
import java.io.Serializable;
import java.util.Collection;
import java.util.Iterator;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.function.Supplier;
import javax.annotation.Nullable;
import org.apache.beam.sdk.coders.StringUtf8Coder;
import org.apache.beam.sdk.io.Compression;
import org.apache.beam.sdk.io.FileIO;
import org.apache.beam.sdk.io.FileIO.ReadableFile;
import org.apache.beam.sdk.io.fs.EmptyMatchTreatment;
import org.apache.beam.sdk.io.fs.MatchResult.Metadata;
import org.apache.beam.sdk.transforms.Create;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.Flatten;
import org.apache.beam.sdk.transforms.GroupByKey;
import org.apache.beam.sdk.transforms.MapElements;
import org.apache.beam.sdk.transforms.PTransform;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.transforms.ProcessFunction;
import org.apache.beam.sdk.values.KV;
import org.apache.beam.sdk.values.PBegin;
import org.apache.beam.sdk.values.PCollection;
import org.apache.beam.sdk.values.PCollectionList;
import org.apache.beam.sdk.values.PCollectionTuple;
import org.apache.beam.sdk.values.TupleTag;
import org.apache.beam.sdk.values.TupleTagList;
import org.apache.beam.sdk.values.TypeDescriptor;
import org.joda.time.DateTime;
/**
* {@link PTransform Pipeline transforms} used in pipelines that load from both Datastore export
* files and Nomulus CommitLog files.
*/
@DeleteAfterMigration
public final class Transforms {
private Transforms() {}
/**
* The commitTimestamp assigned to all entities loaded from a Datastore export file. The exact
* value does not matter, but it must be lower than the timestamps of real CommitLog records.
*/
@VisibleForTesting static final long EXPORT_ENTITY_TIME_STAMP = START_OF_TIME.getMillis();
/**
* Returns a {@link TupleTag} that can be used to retrieve entities of the given {@code kind} from
* the Datastore snapshot returned by {@link #loadDatastoreSnapshot}.
*/
public static TupleTag<VersionedEntity> createTagForKind(String kind) {
// When used with PCollectionTuple the result must retain generic type information.
// Both the Generic param and the empty bracket below are important.
return new TupleTag<VersionedEntity>(Transforms.class.getSimpleName() + ":" + kind) {};
}
/**
* Composite {@link PTransform transform} that loads the Datastore snapshot right before {@code
* commitLogToTime} for caller specified {@code kinds}. The resulting snapshot has all changes
* that happened before {@code commitLogToTime}, and none at or after {@code commitLogToTime}.
*
* <p>Caller must provide the location of a Datastore export that started AFTER {@code
* commitLogFromTime} and completed BEFORE {@code commitLogToTime}, as well as the root directory
* of all CommitLog files.
*
* <p>Selection of {@code commitLogFromTime} and {@code commitLogToTime} should follow the
* guidelines below to ensure that all incremental changes concurrent with the export are covered:
*
* <ul>
* <li>Two or more CommitLogs should exist between {@code commitLogFromTime} and the starting
* time of the Datastore export. This ensures that the earlier CommitLog file was complete
* before the export started.
* <li>Two or more CommitLogs should exit between the export completion time and {@code
* commitLogToTime}.
* </ul>
*
* <p>The output from the returned transform is a {@link PCollectionTuple} consisting of {@link
* VersionedEntity VersionedEntities} grouped into {@link PCollection PCollections} by {@code
* kind}.
*/
public static PTransform<PBegin, PCollectionTuple> loadDatastoreSnapshot(
String exportDir,
String commitLogDir,
DateTime commitLogFromTime,
DateTime commitLogToTime,
Set<String> kinds) {
checkArgument(kinds != null && !kinds.isEmpty(), "At least one kind is expected.");
// Create tags to collect entities by kind in final step.
final ImmutableMap<String, TupleTag<VersionedEntity>> outputTags =
kinds.stream()
.collect(ImmutableMap.toImmutableMap(kind -> kind, Transforms::createTagForKind));
// Arbitrarily select one tag as mainOutTag and put the remaining ones in a TupleTagList.
// This separation is required by ParDo's config API.
Iterator<TupleTag<VersionedEntity>> tagsIt = outputTags.values().iterator();
final TupleTag<VersionedEntity> mainOutputTag = tagsIt.next();
final TupleTagList additionalTags = TupleTagList.of(ImmutableList.copyOf(tagsIt));
return new PTransform<PBegin, PCollectionTuple>() {
@Override
public PCollectionTuple expand(PBegin input) {
PCollection<VersionedEntity> exportedEntities =
input
.apply("Get export file patterns", getDatastoreExportFilePatterns(exportDir, kinds))
.apply("Find export files", getFilesByPatterns())
.apply("Load export data", loadExportDataFromFiles());
PCollection<VersionedEntity> commitLogEntities =
input
.apply("Get commitlog file patterns", getCommitLogFilePatterns(commitLogDir))
.apply("Find commitlog files", getFilesByPatterns())
.apply(
"Filter commitLog by time",
filterCommitLogsByTime(commitLogFromTime, commitLogToTime))
.apply("Load commitlog data", loadCommitLogsFromFiles(kinds));
return PCollectionList.of(exportedEntities)
.and(commitLogEntities)
.apply("Merge exports and CommitLogs", Flatten.pCollections())
.apply(
"Key entities by Datastore Keys",
// Converting to KV<String, VE> instead of KV<Key, VE> b/c default coder for Key
// (SerializableCoder) is not deterministic and cannot be used with GroupBy.
MapElements.into(kvs(strings(), TypeDescriptor.of(VersionedEntity.class)))
.via((VersionedEntity e) -> KV.of(e.key().toString(), e)))
.apply("Gather entities by key", GroupByKey.create())
.apply(
"Output latest version per entity",
ParDo.of(
new DoFn<KV<String, Iterable<VersionedEntity>>, VersionedEntity>() {
@ProcessElement
public void processElement(
@Element KV<String, Iterable<VersionedEntity>> kv,
MultiOutputReceiver out) {
Optional<VersionedEntity> latest =
Streams.stream(kv.getValue())
.sorted(comparing(VersionedEntity::commitTimeMills).reversed())
.findFirst();
// Throw to abort (after default retries). Investigate, fix, and rerun.
checkState(
latest.isPresent(), "Unexpected key with no data", kv.getKey());
if (latest.get().isDelete()) {
return;
}
String kind = latest.get().getEntity().get().getKind();
out.get(outputTags.get(kind)).output(latest.get());
}
})
.withOutputTags(mainOutputTag, additionalTags));
}
};
}
/**
* Returns a {@link PTransform transform} that can generate a collection of patterns that match
* all Datastore CommitLog files.
*/
public static PTransform<PBegin, PCollection<String>> getCommitLogFilePatterns(
String commitLogDir) {
return toStringPCollection(BackupPaths.getCommitLogFilePatterns(commitLogDir));
}
/**
* Returns a {@link PTransform transform} that can generate a collection of patterns that match
* all Datastore export files of the given {@code kinds}.
*/
public static PTransform<PBegin, PCollection<String>> getDatastoreExportFilePatterns(
String exportDir, Collection<String> kinds) {
return toStringPCollection(getExportFilePatterns(exportDir, kinds));
}
public static PTransform<PBegin, PCollection<String>> getCloudSqlConnectionInfoFilePatterns(
String gcpProjectName) {
return toStringPCollection(BackupPaths.getCloudSQLCredentialFilePatterns(gcpProjectName));
}
/**
* Returns a {@link PTransform} from file name patterns to file {@link Metadata Metadata records}.
*/
public static PTransform<PCollection<String>, PCollection<Metadata>> getFilesByPatterns() {
return new PTransform<PCollection<String>, PCollection<Metadata>>() {
@Override
public PCollection<Metadata> expand(PCollection<String> input) {
return input.apply(FileIO.matchAll().withEmptyMatchTreatment(EmptyMatchTreatment.ALLOW));
}
};
}
/**
* Returns CommitLog files with timestamps between {@code fromTime} (inclusive) and {@code
* endTime} (exclusive).
*/
public static PTransform<PCollection<? extends Metadata>, PCollection<Metadata>>
filterCommitLogsByTime(DateTime fromTime, DateTime toTime) {
return ParDo.of(new FilterCommitLogFileByTime(fromTime, toTime));
}
/** Returns a {@link PTransform} from file {@link Metadata} to {@link VersionedEntity}. */
public static PTransform<PCollection<Metadata>, PCollection<VersionedEntity>>
loadExportDataFromFiles() {
return processFiles(
new BackupFileReader(
file ->
Iterators.transform(
LevelDbLogReader.from(file.open()),
(byte[] bytes) -> VersionedEntity.from(EXPORT_ENTITY_TIME_STAMP, bytes))));
}
/** Returns a {@link PTransform} from file {@link Metadata} to {@link VersionedEntity}. */
public static PTransform<PCollection<Metadata>, PCollection<VersionedEntity>>
loadCommitLogsFromFiles(Set<String> kinds) {
return processFiles(
new BackupFileReader(
file ->
CommitLogImports.loadEntities(file.open()).stream()
.filter(e -> kinds.contains(e.key().getKind()))
.iterator()));
}
// Production data repair configs go below. See b/185954992. Note that the CommitLog replay
// process does not filter out the ignored entities listed below, a mistake that we do not fix
// for operational convenience. Instead, the Database comparison tool will filter them out. See
// ValidateSqlUtils.java for more information.
// Prober domains in bad state, without associated contacts, hosts, billings, and non-synthesized
// history. They can be safely ignored.
public static final ImmutableSet<String> IGNORED_DOMAINS =
ImmutableSet.of("6AF6D2-IQCANT", "2-IQANYT");
// Prober hosts referencing phantom registrars. They and their associated history entries can be
// safely ignored.
public static final ImmutableSet<String> IGNORED_HOSTS =
ImmutableSet.of(
"4E21_WJ0TEST-GOOGLE",
"4E21_WJ1TEST-GOOGLE",
"4E21_WJ2TEST-GOOGLE",
"4E21_WJ3TEST-GOOGLE");
// Prober contacts referencing phantom registrars. They and their associated history entries can
// be safely ignored.
public static final ImmutableSet<String> IGNORED_CONTACTS =
ImmutableSet.of(
"1_WJ0TEST-GOOGLE", "1_WJ1TEST-GOOGLE", "1_WJ2TEST-GOOGLE", "1_WJ3TEST-GOOGLE");
private static boolean isMigratable(Entity entity) {
// Checks specific to production data. See b/185954992 for details.
// The names of these bad entities in production do not conflict with other environments. For
// simplicities sake we apply them regardless of the source of the data.
if (entity.getKind().equals("DomainBase")
&& IGNORED_DOMAINS.contains(entity.getKey().getName())) {
return false;
}
if (entity.getKind().equals("ContactResource")) {
String roid = entity.getKey().getName();
return !IGNORED_CONTACTS.contains(roid);
}
if (entity.getKind().equals("HostResource")) {
String roid = entity.getKey().getName();
return !IGNORED_HOSTS.contains(roid);
}
if (entity.getKind().equals("HistoryEntry")) {
// DOMAIN_APPLICATION_CREATE is deprecated type and should not be migrated.
// The Enum name DOMAIN_APPLICATION_CREATE no longer exists in Java and cannot
// be deserialized.
if (Objects.equals(entity.getProperty("type"), "DOMAIN_APPLICATION_CREATE")) {
return false;
}
// Remove production bad data: Histories of ignored EPP resources:
com.google.appengine.api.datastore.Key parentKey = entity.getKey().getParent();
if (parentKey.getKind().equals("ContactResource")) {
String contactRoid = parentKey.getName();
return !IGNORED_CONTACTS.contains(contactRoid);
}
if (parentKey.getKind().equals("HostResource")) {
String hostRoid = parentKey.getName();
return !IGNORED_HOSTS.contains(hostRoid);
}
if (parentKey.getKind().equals("DomainBase")) {
String domainRoid = parentKey.getName();
return !IGNORED_DOMAINS.contains(domainRoid);
}
}
return true;
}
@VisibleForTesting
static Entity repairBadData(Entity entity) {
if (entity.getKind().equals("Cancellation")
&& Objects.equals(entity.getProperty("reason"), "AUTO_RENEW")) {
// AUTO_RENEW has been moved from 'reason' to flags. Change reason to RENEW and add the
// AUTO_RENEW flag. Note: all affected entities have empty flags so we can simply assign
// instead of append. See b/185954992.
entity.setUnindexedProperty("reason", Reason.RENEW.name());
entity.setUnindexedProperty("flags", ImmutableList.of(Flag.AUTO_RENEW.name()));
} else if (entity.getKind().equals("DomainBase")) {
// Canonicalize old domain/host names from 2016 and earlier before we were enforcing this.
entity.setIndexedProperty(
"fullyQualifiedDomainName",
canonicalizeDomainName((String) entity.getProperty("fullyQualifiedDomainName")));
} else if (entity.getKind().equals("HostResource")) {
entity.setIndexedProperty(
"fullyQualifiedHostName",
canonicalizeDomainName((String) entity.getProperty("fullyQualifiedHostName")));
}
return entity;
}
private static SqlEntity toSqlEntity(Object ofyEntity) {
if (ofyEntity instanceof HistoryEntry) {
HistoryEntry ofyHistory = (HistoryEntry) ofyEntity;
return (SqlEntity) ofyHistory.toChildHistoryEntity();
}
return ((DatastoreAndSqlEntity) ofyEntity).toSqlEntity().get();
}
/**
* Converts a {@link VersionedEntity} to an JPA entity for persistence.
*
* @return An object to be persisted to SQL, or null if the input is not to be migrated. (Not
* using Optional in return because as a one-use method, we do not want to invest the effort
* to make Optional work with BEAM)
*/
@Nullable
public static SqlEntity convertVersionedEntityToSqlEntity(VersionedEntity dsEntity) {
return dsEntity
.getEntity()
.filter(Transforms::isMigratable)
.map(Transforms::repairBadData)
.map(e -> auditedOfy().toPojo(e))
.map(Transforms::toSqlEntity)
.orElse(null);
}
/** Interface for serializable {@link Supplier suppliers}. */
public interface SerializableSupplier<T> extends Supplier<T>, Serializable {}
/**
* Returns a {@link PTransform} that produces a {@link PCollection} containing all elements in the
* given {@link Iterable}.
*/
private static PTransform<PBegin, PCollection<String>> toStringPCollection(
Iterable<String> strings) {
return Create.of(strings).withCoder(StringUtf8Coder.of());
}
/**
* Returns a {@link PTransform} from file {@link Metadata} to {@link VersionedEntity} using
* caller-provided {@code transformer}.
*/
private static PTransform<PCollection<Metadata>, PCollection<VersionedEntity>> processFiles(
DoFn<ReadableFile, VersionedEntity> transformer) {
return new PTransform<PCollection<Metadata>, PCollection<VersionedEntity>>() {
@Override
public PCollection<VersionedEntity> expand(PCollection<Metadata> input) {
return input
.apply(FileIO.readMatches().withCompression(Compression.UNCOMPRESSED))
.apply(transformer.getClass().getSimpleName(), ParDo.of(transformer));
}
};
}
private static class FilterCommitLogFileByTime extends DoFn<Metadata, Metadata> {
private final DateTime fromTime;
private final DateTime toTime;
FilterCommitLogFileByTime(DateTime fromTime, DateTime toTime) {
checkNotNull(fromTime, "fromTime");
checkNotNull(toTime, "toTime");
checkArgument(
fromTime.isBefore(toTime),
"Invalid time range: fromTime (%s) is before endTime (%s)",
fromTime,
toTime);
this.fromTime = fromTime;
this.toTime = toTime;
}
@ProcessElement
public void processElement(@Element Metadata fileMeta, OutputReceiver<Metadata> out) {
DateTime timestamp = getCommitLogTimestamp(fileMeta.resourceId().toString());
if (isBeforeOrAt(fromTime, timestamp) && timestamp.isBefore(toTime)) {
out.output(fileMeta);
}
}
}
/**
* Reads from a Datastore backup file and converts its content into {@link VersionedEntity
* VersionedEntities}.
*
* <p>The input file may be either a LevelDb file from a Datastore export or a CommitLog file
* generated by the Nomulus server. In either case, the file contains variable-length records and
* must be read sequentially from the beginning. If the read fails, the file needs to be retried
* from the beginning.
*/
private static class BackupFileReader extends DoFn<ReadableFile, VersionedEntity> {
private final ProcessFunction<ReadableFile, Iterator<VersionedEntity>> reader;
private BackupFileReader(ProcessFunction<ReadableFile, Iterator<VersionedEntity>> reader) {
this.reader = reader;
}
@ProcessElement
public void processElement(@Element ReadableFile file, OutputReceiver<VersionedEntity> out) {
try {
reader.apply(file).forEachRemaining(out::output);
} catch (Exception e) {
// Let the pipeline use default retry strategy on the whole file. For GCP Dataflow this
// means retrying up to 4 times (may include other files grouped with this one), and failing
// the pipeline if no success.
throw new RuntimeException(e);
}
}
}
/**
* Removes BillingEvents, {@link google.registry.model.poll.PollMessage PollMessages} and {@link
* google.registry.model.host.HostResource} from a {@link DomainBase}. These are circular foreign
* key constraints that prevent migration of {@code DomainBase} to SQL databases.
*
* <p>See {@link InitSqlPipeline} for more information.
*/
static class RemoveDomainBaseForeignKeys extends DoFn<VersionedEntity, VersionedEntity> {
@ProcessElement
public void processElement(
@Element VersionedEntity domainBase, OutputReceiver<VersionedEntity> out) {
checkArgument(
domainBase.getEntity().isPresent(), "Unexpected delete entity %s", domainBase.key());
Entity outputEntity =
DomainBaseUtil.removeBillingAndPollAndHosts(domainBase.getEntity().get());
out.output(
VersionedEntity.from(
domainBase.commitTimeMills(),
EntityTranslator.convertToPb(outputEntity).toByteArray()));
}
}
}

View File

@@ -18,6 +18,7 @@ import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.beam.BeamUtils.getQueryFromFile;
import static org.apache.beam.sdk.values.TypeDescriptors.strings;
import com.google.common.flogger.FluentLogger;
import google.registry.beam.common.RegistryJpaIO;
import google.registry.beam.common.RegistryJpaIO.Read;
import google.registry.beam.invoicing.BillingEvent.InvoiceGroupingKey;
@@ -35,15 +36,14 @@ import java.time.LocalTime;
import java.time.YearMonth;
import java.time.ZoneId;
import java.time.format.DateTimeFormatter;
import java.util.Objects;
import java.util.Optional;
import java.util.regex.Pattern;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.PipelineResult;
import org.apache.beam.sdk.coders.SerializableCoder;
import org.apache.beam.sdk.coders.StringUtf8Coder;
import org.apache.beam.sdk.io.FileIO;
import org.apache.beam.sdk.io.TextIO;
import org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.apache.beam.sdk.transforms.Contextful;
import org.apache.beam.sdk.transforms.Count;
@@ -53,6 +53,7 @@ import org.apache.beam.sdk.transforms.PTransform;
import org.apache.beam.sdk.values.KV;
import org.apache.beam.sdk.values.PCollection;
import org.apache.beam.sdk.values.TypeDescriptor;
import org.joda.money.CurrencyUnit;
/**
* Definition of a Dataflow Flex pipeline template, which generates a given month's invoices.
@@ -73,6 +74,8 @@ public class InvoicingPipeline implements Serializable {
private static final Pattern SQL_COMMENT_REGEX =
Pattern.compile("^\\s*--.*\\n", Pattern.MULTILINE);
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final InvoicingPipelineOptions options;
InvoicingPipeline(InvoicingPipelineOptions options) {
@@ -87,57 +90,53 @@ public class InvoicingPipeline implements Serializable {
void setupPipeline(Pipeline pipeline) {
options.setIsolationOverride(TransactionIsolationLevel.TRANSACTION_READ_COMMITTED);
PCollection<BillingEvent> billingEvents =
options.getDatabase().equals("DATASTORE")
? readFromBigQuery(options, pipeline)
: readFromCloudSql(options, pipeline);
PCollection<BillingEvent> billingEvents = readFromCloudSql(options, pipeline);
saveInvoiceCsv(billingEvents, options);
saveDetailedCsv(billingEvents, options);
}
static PCollection<BillingEvent> readFromBigQuery(
InvoicingPipelineOptions options, Pipeline pipeline) {
return pipeline.apply(
"Read BillingEvents from Bigquery",
BigQueryIO.read(BillingEvent::parseFromRecord)
.fromQuery(makeQuery(options.getYearMonth(), options.getProject()))
.withCoder(SerializableCoder.of(BillingEvent.class))
.usingStandardSql()
.withoutValidation()
.withTemplateCompatibility());
}
static PCollection<BillingEvent> readFromCloudSql(
InvoicingPipelineOptions options, Pipeline pipeline) {
Read<Object[], BillingEvent> read =
RegistryJpaIO.read(
makeCloudSqlQuery(options.getYearMonth()), false, InvoicingPipeline::parseRow);
makeCloudSqlQuery(options.getYearMonth()), false, row -> parseRow(row).orElse(null));
return pipeline.apply("Read BillingEvents from Cloud SQL", read);
PCollection<BillingEvent> billingEventsWithNulls =
pipeline.apply("Read BillingEvents from Cloud SQL", read);
// Remove null billing events
return billingEventsWithNulls.apply(Filter.by(Objects::nonNull));
}
private static BillingEvent parseRow(Object[] row) {
private static Optional<BillingEvent> parseRow(Object[] row) {
google.registry.model.billing.BillingEvent.OneTime oneTime =
(google.registry.model.billing.BillingEvent.OneTime) row[0];
Registrar registrar = (Registrar) row[1];
return BillingEvent.create(
oneTime.getId(),
DateTimeUtils.toZonedDateTime(oneTime.getBillingTime(), ZoneId.of("UTC")),
DateTimeUtils.toZonedDateTime(oneTime.getEventTime(), ZoneId.of("UTC")),
registrar.getRegistrarId(),
registrar.getBillingIdentifier().toString(),
registrar.getPoNumber().orElse(""),
DomainNameUtils.getTldFromDomainName(oneTime.getTargetId()),
oneTime.getReason().toString(),
oneTime.getTargetId(),
oneTime.getDomainRepoId(),
Optional.ofNullable(oneTime.getPeriodYears()).orElse(0),
oneTime.getCost().getCurrencyUnit().toString(),
oneTime.getCost().getAmount().doubleValue(),
String.join(
" ", oneTime.getFlags().stream().map(Flag::toString).collect(toImmutableSet())));
CurrencyUnit currency = oneTime.getCost().getCurrencyUnit();
if (!registrar.getBillingAccountMap().containsKey(currency)) {
logger.atSevere().log(
"Registrar %s does not have a product account key for the currency unit: %s",
registrar.getRegistrarId(), currency);
return Optional.empty();
}
return Optional.of(
BillingEvent.create(
oneTime.getId(),
DateTimeUtils.toZonedDateTime(oneTime.getBillingTime(), ZoneId.of("UTC")),
DateTimeUtils.toZonedDateTime(oneTime.getEventTime(), ZoneId.of("UTC")),
registrar.getRegistrarId(),
registrar.getBillingAccountMap().get(currency),
registrar.getPoNumber().orElse(""),
DomainNameUtils.getTldFromDomainName(oneTime.getTargetId()),
oneTime.getReason().toString(),
oneTime.getTargetId(),
oneTime.getDomainRepoId(),
Optional.ofNullable(oneTime.getPeriodYears()).orElse(0),
oneTime.getCost().getCurrencyUnit().toString(),
oneTime.getCost().getAmount().doubleValue(),
String.join(
" ", oneTime.getFlags().stream().map(Flag::toString).collect(toImmutableSet()))));
}
/** Transform that converts a {@code BillingEvent} into an invoice CSV row. */

View File

@@ -30,11 +30,6 @@ public interface InvoicingPipelineOptions extends RegistryPipelineOptions {
void setInvoiceFilePrefix(String value);
@Description("The database to read data from.")
String getDatabase();
void setDatabase(String value);
@Description("The GCS bucket URL for invoices and detailed reports to be uploaded.")
String getBillingBucketUrl();

View File

@@ -18,7 +18,6 @@ import static com.google.common.base.Preconditions.checkState;
import static com.google.common.base.Verify.verify;
import static google.registry.model.common.Cursor.getCursorTimeOrStartOfTime;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.persistence.transaction.TransactionManagerUtil.transactIfJpaTm;
import static google.registry.rde.RdeModule.BRDA_QUEUE;
import static google.registry.rde.RdeModule.RDE_UPLOAD_QUEUE;
import static java.nio.charset.StandardCharsets.UTF_8;
@@ -249,9 +248,6 @@ public class RdeIO {
// Now that we're done, output roll the cursor forward.
if (key.manual()) {
logger.atInfo().log("Manual operation; not advancing cursor or enqueuing upload task.");
// Temporary measure to run RDE in beam in parallel with the daily MapReduce based RDE runs.
} else if (tm().isOfy()) {
logger.atInfo().log("Ofy is primary TM; not advancing cursor or enqueuing upload task.");
} else {
outputReceiver.output(KV.of(key, revision));
}
@@ -273,16 +269,15 @@ public class RdeIO {
@ProcessElement
public void processElement(
@Element KV<PendingDeposit, Integer> input, PipelineOptions options) {
tm().transact(
() -> {
PendingDeposit key = input.getKey();
Registry registry = Registry.get(key.tld());
Optional<Cursor> cursor =
transactIfJpaTm(
() ->
tm().loadByKeyIfPresent(
Cursor.createVKey(key.cursor(), registry.getTldStr())));
tm().transact(
() ->
tm().loadByKeyIfPresent(
Cursor.createScopedVKey(key.cursor(), registry)));
DateTime position = getCursorTimeOrStartOfTime(cursor);
checkState(key.interval() != null, "Interval must be present");
DateTime newPosition = key.watermark().plus(key.interval());
@@ -295,7 +290,7 @@ public class RdeIO {
"Partial ordering of RDE deposits broken: %s %s",
position,
key);
tm().put(Cursor.create(key.cursor(), newPosition, registry));
tm().put(Cursor.createScoped(key.cursor(), newPosition, registry));
logger.atInfo().log(
"Rolled forward %s on %s cursor to %s.", key.cursor(), key.tld(), newPosition);
RdeRevision.saveRevision(key.tld(), key.watermark(), key.mode(), input.getValue());

View File

@@ -15,6 +15,7 @@
package google.registry.beam.rde;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.beam.rde.RdePipeline.TupleTags.DOMAIN_FRAGMENTS;
import static google.registry.beam.rde.RdePipeline.TupleTags.EXTERNAL_HOST_FRAGMENTS;
@@ -28,10 +29,12 @@ import static google.registry.model.reporting.HistoryEntryDao.RESOURCE_TYPES_TO_
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static org.apache.beam.sdk.values.TypeDescriptors.kvs;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Sets;
import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import com.google.common.io.BaseEncoding;
import dagger.BindsInstance;
import dagger.Component;
@@ -198,6 +201,8 @@ public class RdePipeline implements Serializable {
HostHistory.class,
"hostBase");
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
@Inject
RdePipeline(RdePipelineOptions options, GcsUtils gcsUtils, CloudTasksUtils cloudTasksUtils) {
this.options = options;
@@ -357,6 +362,33 @@ public class RdePipeline implements Serializable {
.setCoder(KvCoder.of(StringUtf8Coder.of(), VarLongCoder.of()));
}
private <T extends HistoryEntry> EppResource loadResourceByHistoryEntryId(
Class<T> historyEntryClazz, String repoId, Iterable<Long> revisionIds) {
ImmutableList<Long> ids = ImmutableList.copyOf(revisionIds);
// The size should always be 1 because we are only getting one repo ID -> revision ID pair per
// repo ID from the source transform (the JPA query in the method above). But for some reason
// after CoGroupByKey (joining the revision IDs and the pending deposits on repo IDs), in
// #removedUnreferencedResources, duplicate revision IDs are sometimes introduced. Here we
// attempt to deduplicate the iterable. If it contains multiple revision IDs that are NOT the
// same, we have a more serious problem as we cannot be sure which one to use. We should use the
// highest revision ID, but we don't even know where it comes from, as the query should
// definitively only give us one revision ID per repo ID. In this case we have to abort and
// require manual intervention.
if (ids.size() != 1) {
ImmutableSet<Long> dedupedIds = ImmutableSet.copyOf(ids);
checkState(
dedupedIds.size() == 1,
"Multiple unique revision IDs detected for %s repo ID %s: %s",
EPP_RESOURCE_FIELD_NAME.get(historyEntryClazz),
repoId,
ids);
logger.atSevere().log(
"Duplicate revision IDs detected for %s repo ID %s: %s",
EPP_RESOURCE_FIELD_NAME.get(historyEntryClazz), repoId, ids);
}
return loadResourceByHistoryEntryId(historyEntryClazz, repoId, ids.get(0));
}
private <T extends HistoryEntry> EppResource loadResourceByHistoryEntryId(
Class<T> historyEntryClazz, String repoId, long revisionId) {
try {
@@ -516,7 +548,7 @@ public class RdePipeline implements Serializable {
loadResourceByHistoryEntryId(
ContactHistory.class,
kv.getKey(),
kv.getValue().getOnly(REVISION_ID));
kv.getValue().getAll(REVISION_ID));
DepositFragment fragment = marshaller.marshalContact(contact);
ImmutableSet<KV<PendingDeposit, DepositFragment>> fragments =
Streams.stream(kv.getValue().getAll(PENDING_DEPOSIT))
@@ -549,8 +581,8 @@ public class RdePipeline implements Serializable {
loadResourceByHistoryEntryId(
HostHistory.class,
kv.getKey(),
kv.getValue().getOnly(REVISION_ID));
// When a host is subordinate, we need to find it's superordinate domain and
kv.getValue().getAll(REVISION_ID));
// When a host is subordinate, we need to find its superordinate domain and
// include it in the deposit as well.
if (host.isSubordinate()) {
subordinateHostCounter.inc();
@@ -627,7 +659,7 @@ public class RdePipeline implements Serializable {
loadResourceByHistoryEntryId(
DomainHistory.class,
kv.getKey(),
kv.getValue().getOnly(REVISION_ID));
kv.getValue().getAll(REVISION_ID));
ImmutableSet.Builder<KV<PendingDeposit, DepositFragment>> results =
new ImmutableSet.Builder<>();
for (KV<String, CoGbkResult> hostToPendingDeposits :
@@ -637,7 +669,7 @@ public class RdePipeline implements Serializable {
loadResourceByHistoryEntryId(
HostHistory.class,
hostToPendingDeposits.getKey(),
hostToPendingDeposits.getValue().getOnly(REVISION_ID));
hostToPendingDeposits.getValue().getAll(REVISION_ID));
DepositFragment fragment =
marshaller.marshalSubordinateHost(host, superordinateDomain);
Streams.stream(hostToPendingDeposits.getValue().getAll(PENDING_DEPOSIT))
@@ -696,6 +728,7 @@ public class RdePipeline implements Serializable {
* CoGbkResult}s are used.
*/
protected abstract static class TupleTags {
protected static final TupleTag<KV<PendingDeposit, DepositFragment>> DOMAIN_FRAGMENTS =
new TupleTag<KV<PendingDeposit, DepositFragment>>() {};
@@ -729,10 +762,12 @@ public class RdePipeline implements Serializable {
UtilsModule.class
})
interface RdePipelineComponent {
RdePipeline rdePipeline();
@Component.Builder
interface Builder {
@BindsInstance
Builder options(RdePipelineOptions options);

View File

@@ -0,0 +1,174 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.resave;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static org.apache.beam.sdk.values.TypeDescriptors.integers;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import google.registry.beam.common.RegistryJpaIO;
import google.registry.beam.common.RegistryJpaIO.Read;
import google.registry.model.EppResource;
import google.registry.model.contact.ContactResource;
import google.registry.model.domain.DomainBase;
import google.registry.model.host.HostResource;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.persistence.transaction.CriteriaQueryBuilder;
import google.registry.util.DateTimeUtils;
import java.io.Serializable;
import java.util.concurrent.ThreadLocalRandom;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.PipelineResult;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.GroupIntoBatches;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.transforms.WithKeys;
import org.apache.beam.sdk.util.ShardedKey;
import org.apache.beam.sdk.values.KV;
import org.joda.time.DateTime;
/**
* A Dataflow Flex pipeline that resaves changed EPP resources in SQL.
*
* <p>Due to the way that Hibernate works, if an entity is unchanged by {@link
* EppResource#cloneProjectedAtTime(DateTime)} it will not actually be re-persisted to the database.
* Thus, the only actual changes occur when objects are changed by projecting them to now, such as
* when a pending transfer is resolved.
*/
public class ResaveAllEppResourcesPipeline implements Serializable {
private static final ImmutableSet<Class<? extends EppResource>> EPP_RESOURCE_CLASSES =
ImmutableSet.of(ContactResource.class, DomainBase.class, HostResource.class);
/**
* There exist three possible situations where we know we'll want to project domains to the
* current point in time:
*
* <ul>
* <li>A pending domain transfer has expired.
* <li>A domain is past its expiration time without being deleted (this means it autorenewed).
* <li>A domain has expired grace periods.
* </ul>
*
* <p>This command contains all three scenarios so that we can avoid querying the Domain table
* multiple times, and to avoid projecting and resaving the same domain multiple times.
*/
private static final String DOMAINS_TO_PROJECT_QUERY =
"FROM Domain d WHERE (d.transferData.transferStatus = 'PENDING' AND"
+ " d.transferData.pendingTransferExpirationTime < current_timestamp()) OR"
+ " (d.registrationExpirationTime < current_timestamp() AND d.deletionTime ="
+ " (:END_OF_TIME)) OR (EXISTS (SELECT 1 FROM GracePeriod gp WHERE gp.domainRepoId ="
+ " d.repoId AND gp.expirationTime < current_timestamp()))";
private final ResaveAllEppResourcesPipelineOptions options;
ResaveAllEppResourcesPipeline(ResaveAllEppResourcesPipelineOptions options) {
this.options = options;
}
PipelineResult run() {
Pipeline pipeline = Pipeline.create(options);
setupPipeline(pipeline);
return pipeline.run();
}
void setupPipeline(Pipeline pipeline) {
options.setIsolationOverride(TransactionIsolationLevel.TRANSACTION_READ_COMMITTED);
if (options.getFast()) {
fastResaveContacts(pipeline);
fastResaveDomains(pipeline);
} else {
EPP_RESOURCE_CLASSES.forEach(clazz -> forceResaveAllResources(pipeline, clazz));
}
}
/** Projects to the current time and saves any contacts with expired transfers. */
private void fastResaveContacts(Pipeline pipeline) {
Read<ContactResource, ContactResource> read =
RegistryJpaIO.read(
"FROM Contact WHERE transferData.transferStatus = 'PENDING' AND"
+ " transferData.pendingTransferExpirationTime < current_timestamp()",
ContactResource.class,
c -> c);
projectAndResaveResources(pipeline, ContactResource.class, read);
}
/**
* Projects to the current time and saves any domains with expired pending actions (e.g.
* transfers, grace periods).
*
* <p>The logic of what might have changed is paraphrased from {@link
* google.registry.model.domain.DomainContent#cloneProjectedAtTime(DateTime)}.
*/
private void fastResaveDomains(Pipeline pipeline) {
Read<DomainBase, DomainBase> read =
RegistryJpaIO.read(
DOMAINS_TO_PROJECT_QUERY,
ImmutableMap.of("END_OF_TIME", DateTimeUtils.END_OF_TIME),
DomainBase.class,
d -> d);
projectAndResaveResources(pipeline, DomainBase.class, read);
}
/** Projects all resources to the current time and saves them. */
private <T extends EppResource> void forceResaveAllResources(Pipeline pipeline, Class<T> clazz) {
Read<T, T> read = RegistryJpaIO.read(() -> CriteriaQueryBuilder.create(clazz).build());
projectAndResaveResources(pipeline, clazz, read);
}
/** Projects and re-saves the result of the provided {@link Read}. */
private <T extends EppResource> void projectAndResaveResources(
Pipeline pipeline, Class<T> clazz, Read<?, T> read) {
int numShards = options.getSqlWriteShards();
int batchSize = options.getSqlWriteBatchSize();
String className = clazz.getSimpleName();
pipeline
.apply("Read " + className, read)
.apply(
"Shard data for class" + className,
WithKeys.<Integer, T>of(e -> ThreadLocalRandom.current().nextInt(numShards))
.withKeyType(integers()))
.apply(
"Group into batches for class" + className,
GroupIntoBatches.<Integer, T>ofSize(batchSize).withShardedKey())
.apply("Map " + className + " to now", ParDo.of(new BatchedProjectionFunction<>()))
.apply(
"Write transformed " + className,
RegistryJpaIO.<EppResource>write()
.withName("Write transformed " + className)
.withBatchSize(batchSize)
.withShards(numShards));
}
private static class BatchedProjectionFunction<T extends EppResource>
extends DoFn<KV<ShardedKey<Integer>, Iterable<T>>, EppResource> {
@ProcessElement
public void processElement(
@Element KV<ShardedKey<Integer>, Iterable<T>> element,
OutputReceiver<EppResource> outputReceiver) {
jpaTm()
.transact(
() ->
element
.getValue()
.forEach(
resource ->
outputReceiver.output(
resource.cloneProjectedAtTime(jpaTm().getTransactionTime()))));
}
}
}

View File

@@ -1,4 +1,4 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -12,16 +12,15 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.beam.initsql;
package google.registry.beam.resave;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.junit.jupiter.api.Test;
import google.registry.beam.common.RegistryPipelineOptions;
import org.apache.beam.sdk.options.Description;
/** Unit tests for {@link google.registry.beam.initsql.InitSqlPipelineOptions}. * */
public class InitSqlPipelineOptionsTest {
public interface ResaveAllEppResourcesPipelineOptions extends RegistryPipelineOptions {
@Test
void registerToValidate() {
PipelineOptionsFactory.register(InitSqlPipelineOptions.class);
}
@Description("True if we should attempt to run only over potentially out-of-date EPP resources")
boolean getFast();
void setFast(boolean fast);
}

View File

@@ -14,30 +14,19 @@
package google.registry.beam.spec11;
import static google.registry.beam.BeamUtils.checkFieldsNotNull;
import static google.registry.beam.BeamUtils.extractField;
import com.google.auto.value.AutoValue;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableList;
import java.io.Serializable;
import org.apache.avro.generic.GenericRecord;
import org.apache.beam.sdk.io.gcp.bigquery.SchemaAndRecord;
/**
* A POJO representing a domain name and associated info, parsed from a {@code SchemaAndRecord}.
*
* <p>This is a trivially serializable class that allows Beam to transform the results of a Bigquery
* query into a standard Java representation, giving us the type guarantees and ease of manipulation
* Bigquery lacks, while localizing any Bigquery-side failures to the {@link #parseFromRecord}
* function.
* <p>This is a trivially serializable class that allows Beam to transform the results of a SQL
* query into a standard Java representation.
*/
@AutoValue
public abstract class DomainNameInfo implements Serializable {
private static final ImmutableList<String> FIELD_NAMES =
ImmutableList.of("domainName", "domainRepoId", "registrarId", "registrarEmailAddress");
/** Returns the fully qualified domain name. */
abstract String domainName();
@@ -50,28 +39,8 @@ public abstract class DomainNameInfo implements Serializable {
/** Returns the email address of the registrar associated with this domain. */
abstract String registrarEmailAddress();
/**
* Constructs a {@link DomainNameInfo} from an Apache Avro {@code SchemaAndRecord}.
*
* @see <a
* href=http://avro.apache.org/docs/1.7.7/api/java/org/apache/avro/generic/GenericData.Record.html>
* Apache AVRO GenericRecord</a>
*/
static DomainNameInfo parseFromRecord(SchemaAndRecord schemaAndRecord) {
checkFieldsNotNull(FIELD_NAMES, schemaAndRecord);
GenericRecord record = schemaAndRecord.getRecord();
return create(
extractField(record, "domainName"),
extractField(record, "domainRepoId"),
extractField(record, "registrarId"),
extractField(record, "registrarEmailAddress"));
}
/**
* Creates a concrete {@link DomainNameInfo}.
*
* <p>This should only be used outside this class for testing- instances of {@link DomainNameInfo}
* should otherwise come from {@link #parseFromRecord}.
*/
@VisibleForTesting
static DomainNameInfo create(

Some files were not shown because too many files have changed in this diff Show More