1
0
mirror of https://github.com/google/nomulus synced 2026-06-09 16:33:02 +00:00

Compare commits

...

50 Commits

Author SHA1 Message Date
Lai Jiang bf52a78e89 Update Gradle wrapper to 5.6 (#232)
* Update Gradle wrapper to 5.6

* Upgrade shadowJar to 5.1 and increase JVM heap size

* Use Gradle binary on GCS
2019-08-23 22:10:55 -07:00
gbrodman 8ec16dca8d Add a registry lock password to contacts (#226)
* Add a registry lock password to contacts

* enabled -> allowed

* Simple CR responses, still need to add tests

* Add a very simple hashing test file

* Allow setting of RL password rather than directly setting it

* Round out pw tests

* Include 'allowedToSet...' in registrar contact JSON

* Responses to CR

* fix the hardcoded tests

* Use null or empty rather than just null
2019-08-23 22:34:43 -04:00
Michael Muller 69cb852a9c Add a "showErrorOutput" property (#237)
* Add a "showErrorOutput" property

Add a property to let us dump test output and final status in real-time to the
console.
2019-08-22 11:10:56 -04:00
Michael Muller 6dee3d526e Add a generate_sql_schema command (#230)
* Add a generate_schema  command

Add a generate_schema command to nomulus tool and add the necessary
instrumentation to EppResource and DomainBase to allow us to generate a
proof-of-concept schema for DomainBase.

* Added forgotten command description

* Revert "Added forgotten command description"

This reverts commit 09326cb8ac.
(checked in the wrong file)

* Added fixes requested during review

* Add a todo to start postgresql container

Add a todo to start a postgresql container from generate_sql_command.
2019-08-20 12:29:36 -04:00
Shicong Huang 3ba6e7bd06 Remove hardcoded Java home path (#235) 2019-08-20 10:54:45 -04:00
Lai Jiang 2e84cdfc4b Rename packages in the prober (#233)
Package names should not be plural.
2019-08-19 12:11:19 -04:00
Lai Jiang 77f998ee4b Disable Travis (#231) 2019-08-16 17:13:43 -04:00
Lai Jiang c93248ec10 Update build badges to use Kokoro results for both FOSS and internal (#229) 2019-08-16 15:06:02 -04:00
Aman Sanger 3ac179aead Prober metrics collection (#222)
* Added MetricsHandler and Clock to ProbingSequences

* Minor fixes after rebase onto master

* Added metrics gathering to ProbingSequences

* Added testing of MetricsCollector method calls in ProbingSequenceTest

* Added tests on latency recording to ProbingSequenceTest

* Added response name as label to metrics
2019-08-16 13:49:50 -04:00
gbrodman 92f2f3274e Add a boolean for when a registrar has enabled registry lock (#228)
* Add a boolean for when a registrar has enabled registry lock

* enabled -> allowed

* get -> is
2019-08-16 10:46:07 -04:00
Aman Sanger 57975898d5 Prober EPP components added (#202)
* Updated issues in rebasing

* Minor style change on prober/build.gradle

* Fixed warnings for java compilation

* Fixed files to pass all style tests

* Initial Commit.

* Deleted unfinished features. Added ActionHandler and its Unit Tests.

* Included prober subproject in settings.gradle

* Added Protocol Class and its Basic Unit Tests

* Added Changes Suggested by jianglai

* Fixed Gitignore to take out AutoValue generated code

* Removed AutoValue java files

* Added gitignore within prober

* Removed all generated java

* Final Changes in .gitignore

* Added Ssl and WebWhois Action Handlers and their unit tests in addition to the ProbingAction class

* Fixed build.gradle changes requested

* Removed Files irrelevant to current pull request

* Minor fixes to ActionHandler, as responded in comments, removed package-info, and updated settings.gradle

* Fully Updated ActionHandler (missing updated JavaDoc)

* Added changed Protocol and both Inbound and Outbound Markers

* Removed AutoVaue ignore clause from .gitignore

* removed unneccessary dependencies in build.gradle

* Fixed Javadoc and comments for ActionHandler

* Fixed comments and JavaDoc on other files

* EOL added

* Removed Unnecessary Files

* fixed .gradle files styles

* Removed outbound message from ActionHandler's fields and renamed Marker Interfaces

* Fixed javadoc for Marker Interfaced

* Modified Comments on ActionHandler

* Removed LocalAddress from Protocol

* Fixed Travis Build Issues

* Rebased to Master and added in modified Handlers and ProbingAction

* Fixed changes suggested by CydeWeys

* Added missing license headers and JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor Style Fix

* Full WebWhoIs Sequence Added

* fixed build issues

* Refactored by responses suggested by jianglai.

* Updated build.gradle file

* Modified license header dates

* Updated WebWhois tests.

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* SpotlessApply run to fix style issues

* Added license header and newline where appropriate.

* Javadoc style fix in tests and removed unused methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Modified tests for WebWhois according to changes suggested by laijiang.

* Removed TestProvider from TestUtils.

* Rebased to Master and added in modified Handlers and ProbingAction

* Fixed changes suggested by CydeWeys

* Added missing license headers and JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor Style Fix

* Full WebWhoIs Sequence Added

* fixed build issues

* Refactored by responses suggested by jianglai.

* Updated build.gradle file

* Modified license header dates

* Updated WebWhois tests.

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* SpotlessApply run to fix style issues

* Added license header and newline where appropriate.

* Javadoc style fix in tests and removed unused methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Modified tests for WebWhois according to changes suggested by laijiang.

* Removed TestProvider from TestUtils.

* Rebased to master

* Updated issues in rebasing

* Minor style change on prober/build.gradle

* Fixed warnings for java compilation

* Fixed files to pass all style tests

* Minor syle fixes after succesful rebase onto master

* Initial Commit.

* Added Protocol Class and its Basic Unit Tests

* Fixed Gitignore to take out AutoValue generated code

* Final Changes in .gitignore

* Minor fixes to ActionHandler, as responded in comments, removed package-info, and updated settings.gradle

* Removed AutoVaue ignore clause from .gitignore

* Rebased to Master and added in modified Handlers and ProbingAction

* Full WebWhoIs Sequence Added

* fixed build issues

* Refactored by responses suggested by jianglai.

* Added missing license headers and JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* Full WebWhoIs Sequence Added

* fixed build issues

* Refactored by responses suggested by jianglai.

* Modified license header dates

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* SpotlessApply run to fix style issues

* Added license header and newline where appropriate.

* Javadoc style fix in tests and removed unused methods

* Fixed files to pass all style tests

* Fixed changes suggested by CydeWeys

* Rebased to Master and added in modified Handlers and ProbingAction

* Added missing license headers and JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor Style Fix

* Full WebWhoIs Sequence Added

* Refactored by responses suggested by jianglai.

* Updated build.gradle file

* Modified license header dates

* Updated WebWhois tests.

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* SpotlessApply run to fix style issues

* Added license header and newline where appropriate.

* Javadoc style fix in tests and removed unused methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Modified tests for WebWhois according to changes suggested by laijiang.

* Removed TestProvider from TestUtils.

* Rebased to Master and added in modified Handlers and ProbingAction

* Fixed changes suggested by CydeWeys

* Added missing license headers and JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor Style Fix

* Full WebWhoIs Sequence Added

* fixed build issues

* Refactored by responses suggested by jianglai.

* Updated build.gradle file

* Modified license header dates

* Updated WebWhois tests.

* Added license header and newline where appropriate.

* Javadoc style fix in tests and removed unused methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Modified tests for WebWhois according to changes suggested by laijiang.

* Removed TestProvider from TestUtils.

* Rebased to master

* Updated issues in rebasing

* Added circular linked list to utils

* License Header added

* Refactored probing sequence to be circular linked list iterator

* Modified ProbingStep tests to reflect new ProbingStep structure.

* Added circular linked list to utils

* Added circular linked list to utils

* License Header added

* License Header added

* Refactored probing sequence to be circular linked list iterator

* Modified ProbingStep tests to reflect new ProbingStep structure.

* Added missing license header to DefaultCircularLinkedListIterator

* Fixed changes suggested by CydeWeys

* Rebased to Master and added in modified Handlers and ProbingAction

* Added missing license headers and JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor Style Fix

* Full WebWhoIs Sequence Added

* fixed build issues

* Refactored by responses suggested by jianglai.

* Updated build.gradle file

* Fixed max column length to be 100

* Rebased to Master and added in modified Handlers and ProbingAction

* Modified license header dates

* Updated WebWhois tests.

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* Javadoc style fix in tests and removed unused methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Modified tests for WebWhois according to changes suggested by laijiang.

* Fixed changes suggested by CydeWeys

* Added missing license headers and JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor Style Fix

* Full WebWhoIs Sequence Added

* fixed build issues

* Refactored by responses suggested by jianglai.

* Updated build.gradle file

* Modified license header dates

* Updated WebWhois tests.

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* SpotlessApply run to fix style issues

* Added license header and newline where appropriate.

* Javadoc style fix in tests and removed unused methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Added circular linked list to utils

* Added circular linked list to utils

* License Header added

* License Header added

* Refactored probing sequence to be circular linked list iterator

* Refactored probing sequence to be circular linked list iterator

* Modified tests for WebWhois according to changes suggested by laijiang.

* Removed TestProvider from TestUtils.

* ProbingStepTest modified to have fewer unnecessary helper methods

* Updated issues in rebasing

* Fixed max column length to be 100

* Minor changes to pass style tests

* Successful rebase onto finished web-whois branch

* Removed need for TestTokens with Mockito mocks of Tokens

* Fixed style issues in DefaultCircularLinkedListIterator and AbstractCircularLinkedListIterator

* Modified CircularList according to changes suggested by jianglai.

* Added Protocol Class and its Basic Unit Tests

* Added Ssl and WebWhois Action Handlers and their unit tests in addition to the ProbingAction class

* Fixed changes suggested by CydeWeys

* Fixed changes suggested by CydeWeys

* Rebased to Master and added in modified Handlers and ProbingAction

* Rebased to Master and added in modified Handlers and ProbingAction

* Rebased to Master and added in modified Handlers and ProbingAction

* Rebased to Master and added in modified Handlers and ProbingAction

* Rebased to Master and added in modified Handlers and ProbingAction

* Added missing license headers and JavaDoc

* Minor Style Fix

* Minor Style Fix

* Full WebWhoIs Sequence Added

* Full WebWhoIs Sequence Added

* Full WebWhoIs Sequence Added

* Full WebWhoIs Sequence Added

* fixed build issues

* Refactored by responses suggested by jianglai.

* Updated build.gradle file

* Updated WebWhois tests.

* Added Basic EPP structure

* Added Basic EPP structure

* Prober Updated tests

* Prober Updated tests

* Fully functioning EPP sequences with modified WebWhois base.

* Fully functioning EPP sequences with modified WebWhois base.

* Added Modified test server infrastructure.

* Added Modified test server infrastructure.

* Allowed ActionHandler to pass status to next hanlder in pipeline (to be MetricsHandler).

* Allowed ActionHandler to pass status to next hanlder in pipeline (to be MetricsHandler).

* Javadoc on EppRequestMessage added

* Javadoc on EppRequestMessage added

* Updated EppServer to properly send successful Check responses.

* Updated EppServer to properly send successful Check responses.

* Allowed for expected failures in EPP actions.

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* Fully rebased branch to prober-web-whois after refactoring

* Added license header and newline where appropriate.

* Javadoc style fix in tests and removed unused methods

* Javadoc style fix in tests and removed unused methods

* Modified tests for WebWhois according to changes suggested by laijiang.

* Modified tests for WebWhois according to changes suggested by laijiang.

* Removed TestProvider from TestUtils.

* Rebased to master

* Fixed max column length to be 100

* Fixed files to pass all style tests

* Minor changes to pass style tests

* Successful rebase onto circular-list

* Epp Refactored to accomodate circular linked list PR

* Modified construction of Epp Probing Sequences to reflect CircularList change

* Renamed ProberModule provided Duration

* Removed unnecessary ServerSideException file

* Google-Java-Format run on all prober files

* Style fix on ProbingSequence and its unit tests

* Removed subclasses of EppRequestMessage and EppResponseMessage and fixed style and other minor issues

* Style changes implemented as suggested by jianglai

* Added style fixes suggested by mindhog
2019-08-15 16:03:36 -04:00
Weimin Yu 86fefa9a03 Make postgres dependency runtime (#225)
* Make postgres dependency runtime

Labeling it 'compile' unnecessarily makes psql-dependency a
public contract.
2019-08-14 12:32:07 -04:00
Shicong Huang d143cc83a0 Add Hibernate and PostgreSQL packages (#224) 2019-08-14 11:22:01 -04:00
Aman Sanger df5f450435 Prober circular list (#218)
* Fixed changes suggested by CydeWeys

* Fixed Javadoc and comments for ActionHandler

* Fixed comments and JavaDoc on other files

* EOL added

* Removed Unnecessary Files

* fixed .gradle files styles

* Removed outbound message from ActionHandler's fields and renamed Marker Interfaces

* Fixed javadoc for Marker Interfaced

* Modified Comments on ActionHandler

* Removed LocalAddress from Protocol

* Fixed Travis Build Issues

* Rebased to Master and added in modified Handlers and ProbingAction

* Added missing license headers and JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor Style Fix

* Full WebWhoIs Sequence Added

* fixed build issues

* Refactored by responses suggested by jianglai.

* Minor Style Fixes

* Updated build.gradle file

* Modified license header dates

* Updated WebWhois tests.

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* SpotlessApply run to fix style issues

* Added license header and newline where appropriate.

* Javadoc style fix in tests and removed unused methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Initial Commit.

* Initial Commit.

* Deleted unfinished features. Added ActionHandler and its Unit Tests.

* Included prober subproject in settings.gradle

* Added Protocol Class and its Basic Unit Tests

* Added Protocol Class and its Basic Unit Tests

* Added Changes Suggested by jianglai

* Fixed Gitignore to take out AutoValue generated code

* Fixed Gitignore to take out AutoValue generated code

* Removed AutoValue java files

* Added gitignore within prober

* Removed all generated java

* Final Changes in .gitignore

* Final Changes in .gitignore

* Added Ssl and WebWhois Action Handlers and their unit tests in addition to the ProbingAction class

* Fixed build.gradle changes requested

* Removed Files irrelevant to current pull request

* Fixed changes suggested by CydeWeys

* Fixed changes suggested by CydeWeys

* Fixed changes suggested by CydeWeys

* Fixed changes suggested by CydeWeys

* Minor fixes to ActionHandler, as responded in comments, removed package-info, and updated settings.gradle

* Minor fixes to ActionHandler, as responded in comments, removed package-info, and updated settings.gradle

* Fully Updated ActionHandler (missing updated JavaDoc)

* Added changed Protocol and both Inbound and Outbound Markers

* Removed AutoVaue ignore clause from .gitignore

* Removed AutoVaue ignore clause from .gitignore

* removed unneccessary dependencies in build.gradle

* Fixed Javadoc and comments for ActionHandler

* Fixed comments and JavaDoc on other files

* EOL added

* Removed Unnecessary Files

* fixed .gradle files styles

* Removed outbound message from ActionHandler's fields and renamed Marker Interfaces

* Fixed javadoc for Marker Interfaced

* Modified Comments on ActionHandler

* Removed LocalAddress from Protocol

* Fixed Travis Build Issues

* Rebased to Master and added in modified Handlers and ProbingAction

* Rebased to Master and added in modified Handlers and ProbingAction

* Rebased to Master and added in modified Handlers and ProbingAction

* Rebased to Master and added in modified Handlers and ProbingAction

* Rebased to Master and added in modified Handlers and ProbingAction

* Added missing license headers and JavaDoc

* Added missing license headers and JavaDoc

* Added missing license headers and JavaDoc

* Added missing license headers and JavaDoc

* Added missing license headers and JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor Style Fix

* Minor Style Fix

* Minor Style Fix

* Minor Style Fix

* Full WebWhoIs Sequence Added

* Full WebWhoIs Sequence Added

* Full WebWhoIs Sequence Added

* Full WebWhoIs Sequence Added

* Full WebWhoIs Sequence Added

* Full WebWhoIs Sequence Added

* fixed build issues

* fixed build issues

* fixed build issues

* fixed build issues

* fixed build issues

* Refactored by responses suggested by jianglai.

* Refactored by responses suggested by jianglai.

* Refactored by responses suggested by jianglai.

* Refactored by responses suggested by jianglai.

* Refactored by responses suggested by jianglai.

* Refactored by responses suggested by jianglai.

* Minor Style Fixes

* Minor Style Fixes

* Minor Style Fixes

* Minor Style Fixes

* Minor Style Fixes

* Updated build.gradle file

* Updated build.gradle file

* Updated build.gradle file

* Updated build.gradle file

* Modified license header dates

* Modified license header dates

* Modified license header dates

* Modified license header dates

* Modified license header dates

* Updated WebWhois tests.

* Updated WebWhois tests.

* Updated WebWhois tests.

* Updated WebWhois tests.

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* SpotlessApply run to fix style issues

* SpotlessApply run to fix style issues

* SpotlessApply run to fix style issues

* SpotlessApply run to fix style issues

* SpotlessApply run to fix style issues

* Added license header and newline where appropriate.

* Added license header and newline where appropriate.

* Added license header and newline where appropriate.

* Added license header and newline where appropriate.

* Added license header and newline where appropriate.

* Javadoc style fix in tests and removed unused methods

* Javadoc style fix in tests and removed unused methods

* Javadoc style fix in tests and removed unused methods

* Javadoc style fix in tests and removed unused methods

* Javadoc style fix in tests and removed unused methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Modified tests for WebWhois according to changes suggested by laijiang.

* Modified tests for WebWhois according to changes suggested by laijiang.

* Modified tests for WebWhois according to changes suggested by laijiang.

* Modified tests for WebWhois according to changes suggested by laijiang.

* Modified tests for WebWhois according to changes suggested by laijiang.

* Removed TestProvider from TestUtils.

* Removed TestProvider from TestUtils.

* Removed TestProvider from TestUtils.

* Removed TestProvider from TestUtils.

* Removed TestProvider from TestUtils.

* Rebased to master

* Rebased to master

* Updated issues in rebasing

* Updated issues in rebasing

* Minor style change on prober/build.gradle

* Minor style change on prober/build.gradle

* Fixed warnings for java compilation

* Fixed warnings for java compilation

* Fixed files to pass all style tests

* Fixed files to pass all style tests

* Fixed files to pass all style tests

* Minor syle fixes after succesful rebase onto master

* Fixed changes suggested by CydeWeys

* Fixed changes suggested by CydeWeys

* Fixed changes suggested by CydeWeys

* Rebased to Master and added in modified Handlers and ProbingAction

* Rebased to Master and added in modified Handlers and ProbingAction

* Rebased to Master and added in modified Handlers and ProbingAction

* Added missing license headers and JavaDoc

* Added missing license headers and JavaDoc

* Added missing license headers and JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor Style Fix

* Minor Style Fix

* Minor Style Fix

* Full WebWhoIs Sequence Added

* Full WebWhoIs Sequence Added

* Full WebWhoIs Sequence Added

* fixed build issues

* fixed build issues

* fixed build issues

* Refactored by responses suggested by jianglai.

* Refactored by responses suggested by jianglai.

* Refactored by responses suggested by jianglai.

* Minor Style Fixes

* Minor Style Fixes

* Updated build.gradle file

* Updated build.gradle file

* Updated build.gradle file

* Modified license header dates

* Modified license header dates

* Modified license header dates

* Updated WebWhois tests.

* Updated WebWhois tests.

* Updated WebWhois tests.

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* SpotlessApply run to fix style issues

* Added circular linked list to utils

* Added circular linked list to utils

* Added circular linked list to utils

* Added circular linked list to utils

* Added circular linked list to utils

* License Header added

* License Header added

* License Header added

* License Header added

* License Header added

* Added license header and newline where appropriate.

* Added license header and newline where appropriate.

* Refactored probing sequence to be circular linked list iterator

* Refactored probing sequence to be circular linked list iterator

* Refactored probing sequence to be circular linked list iterator

* Refactored probing sequence to be circular linked list iterator

* Javadoc style fix in tests and removed unused methods

* Javadoc style fix in tests and removed unused methods

* Javadoc style fix in tests and removed unused methods

* Modified ProbingStep tests to reflect new ProbingStep structure.

* Modified ProbingStep tests to reflect new ProbingStep structure.

* Refactored ProbingAction to minimize number of unnecessary methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Modified tests for WebWhois according to changes suggested by laijiang.

* Modified tests for WebWhois according to changes suggested by laijiang.

* Modified tests for WebWhois according to changes suggested by laijiang.

* Removed TestProvider from TestUtils.

* Removed TestProvider from TestUtils.

* Rebased to master

* ProbingStepTest modified to have fewer unnecessary helper methods

* ProbingStepTest modified to have fewer unnecessary helper methods

* Updated issues in rebasing

* Updated issues in rebasing

* Added missing license header to DefaultCircularLinkedListIterator

* Fixed max column length to be 100

* Fixed max column length to be 100

* Minor changes to pass style tests

* Successful rebase onto finished web-whois branch

* Removed need for TestTokens with Mockito mocks of Tokens

* Fixed style issues in DefaultCircularLinkedListIterator and AbstractCircularLinkedListIterator

* Modified CircularList according to changes suggested by jianglai.

* Merge branch 'master' into prober-circular-list

* Modified ProbingSequenceTest to not expect unnecessary NullPointerException

* ProbingSequence and tests modified to reflect addition of UnrecoverableStateException and restarts on failures

* Modified ProbingSequence and its tests to reflect action generation and calling being put in the same try catch block
2019-08-12 18:41:50 -04:00
gbrodman 89a44f176c Clean up token generation (#205)
* Clean up token generation

- Allow tokenLength of 0
- If specifying a token length of 0, throw an error if numTokens > 1

* Allow generation of 0-length strings

* Allow for --tokens option to generate specific tokens

* Revert String generators and disallow 0 'length' param

* Add verifyInput method and batch the listed tokens

* Check the number of tokens created
2019-08-12 17:41:29 -04:00
gbrodman d2319b13fa Bucket RDAP metrics by type (#220)
* Bucket RDAP metrics by type

* Rename method

* Use Guava Splitter and a joining Collector

* Use Iterables.limit

* Whoops, extra paren

* Checkstyle
2019-08-12 11:11:48 -04:00
gbrodman 77259f368d Fix checkstyle in Prober (#221) 2019-08-09 14:10:36 -04:00
Aman Sanger 27b81ba898 Add full prober WebWHOIS sequence functionality (#180)
* Initial Commit.

* Deleted unfinished features. Added ActionHandler and its Unit Tests.

* Included prober subproject in settings.gradle

* Added Protocol Class and its Basic Unit Tests

* Added Changes Suggested by jianglai

* Fixed Gitignore to take out AutoValue generated code

* Removed AutoValue java files

* Added gitignore within prober

* Removed all generated java

* Added Ssl and WebWhois Action Handlers and their unit tests in addition to the ProbingAction class

* Fixed build.gradle changes requested

* Removed Files irrelevant to current pull request

* Minor fixes to ActionHandler, as responded in comments, removed package-info, and updated settings.gradle

* Fully Updated ActionHandler (missing updated JavaDoc)

* Added changed Protocol and both Inbound and Outbound Markers

* Removed AutoVaue ignore clause from .gitignore

* removed unneccessary dependencies in build.gradle

* Fixed Javadoc and comments for ActionHandler

* Fixed comments and JavaDoc on other files

* EOL added

* Removed Unnecessary Files

* fixed .gradle files styles

* Removed outbound message from ActionHandler's fields and renamed Marker Interfaces

* Fixed javadoc for Marker Interfaced

* Modified Comments on ActionHandler

* Removed LocalAddress from Protocol

* Fixed Travis Build Issues

* Rebased to Master and added in modified Handlers and ProbingAction

* Fixed changes suggested by CydeWeys

* Added missing license headers and JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor Style Fix

* Full WebWhoIs Sequence Added

* fixed build issues

* Refactored by responses suggested by jianglai.

* Initial Commit.

* Deleted unfinished features. Added ActionHandler and its Unit Tests.

* Included prober subproject in settings.gradle

* Added Protocol Class and its Basic Unit Tests

* Added Changes Suggested by jianglai

* Fixed Gitignore to take out AutoValue generated code

* Removed AutoValue java files

* Added gitignore within prober

* Removed all generated java

* Final Changes in .gitignore

* Added Ssl and WebWhois Action Handlers and their unit tests in addition to the ProbingAction class

* Fixed build.gradle changes requested

* Removed Files irrelevant to current pull request

* Fixed changes suggested by CydeWeys

* Minor fixes to ActionHandler, as responded in comments, removed package-info, and updated settings.gradle

* Fully Updated ActionHandler (missing updated JavaDoc)

* Added changed Protocol and both Inbound and Outbound Markers

* Removed AutoVaue ignore clause from .gitignore

* removed unneccessary dependencies in build.gradle

* Fixed Javadoc and comments for ActionHandler

* Fixed comments and JavaDoc on other files

* EOL added

* Removed Unnecessary Files

* fixed .gradle files styles

* Rebased to Master and added in modified Handlers and ProbingAction

* Added missing license headers and JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor Style Fix

* Full WebWhoIs Sequence Added

* fixed build issues

* Refactored by responses suggested by jianglai.

* Minor Style Fixes

* Minor Style Fixes

* Updated build.gradle file

* Updated build.gradle file

* Modified license header dates

* Modified license header dates

* Updated WebWhois tests.

* Updated WebWhois tests.

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* SpotlessApply run to fix style issues

* SpotlessApply run to fix style issues

* Added license header and newline where appropriate.

* Added license header and newline where appropriate.

* Javadoc style fix in tests and removed unused methods

* Javadoc style fix in tests and removed unused methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Modified tests for WebWhois according to changes suggested by laijiang.

* Modified tests for WebWhois according to changes suggested by laijiang.

* Removed TestProvider from TestUtils.

* Removed TestProvider from TestUtils.

* Rebased to master

* Updated issues in rebasing

* Minor style change on prober/build.gradle

* Fixed warnings for java compilation

* Fixed files to pass all style tests

* Removed outbound message from ActionHandler's fields and renamed Marker Interfaces

* Initial Commit.

* Deleted unfinished features. Added ActionHandler and its Unit Tests.

* Included prober subproject in settings.gradle

* Added Protocol Class and its Basic Unit Tests

* Added Changes Suggested by jianglai

* Fixed Gitignore to take out AutoValue generated code

* Removed AutoValue java files

* Added gitignore within prober

* Removed all generated java

* Final Changes in .gitignore

* Added Ssl and WebWhois Action Handlers and their unit tests in addition to the ProbingAction class

* Fixed build.gradle changes requested

* Removed Files irrelevant to current pull request

* Fixed changes suggested by CydeWeys

* Fixed changes suggested by CydeWeys

* Fixed changes suggested by CydeWeys

* Minor fixes to ActionHandler, as responded in comments, removed package-info, and updated settings.gradle

* Fully Updated ActionHandler (missing updated JavaDoc)

* Added changed Protocol and both Inbound and Outbound Markers

* Removed AutoVaue ignore clause from .gitignore

* removed unneccessary dependencies in build.gradle

* Fixed Javadoc and comments for ActionHandler

* Fixed comments and JavaDoc on other files

* EOL added

* Removed Unnecessary Files

* fixed .gradle files styles

* Removed outbound message from ActionHandler's fields and renamed Marker Interfaces

* Fixed javadoc for Marker Interfaced

* Fixed javadoc for Marker Interfaced

* Modified Comments on ActionHandler

* Modified Comments on ActionHandler

* Removed LocalAddress from Protocol

* Removed LocalAddress from Protocol

* Fixed Travis Build Issues

* Fixed Travis Build Issues

* Rebased to Master and added in modified Handlers and ProbingAction

* Rebased to Master and added in modified Handlers and ProbingAction

* Rebased to Master and added in modified Handlers and ProbingAction

* Added missing license headers and JavaDoc

* Added missing license headers and JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor Style Fix

* Minor Style Fix

* Full WebWhoIs Sequence Added

* Full WebWhoIs Sequence Added

* fixed build issues

* fixed build issues

* Refactored by responses suggested by jianglai.

* Refactored by responses suggested by jianglai.

* Minor Style Fixes

* Minor Style Fixes

* Updated build.gradle file

* Updated build.gradle file

* Modified license header dates

* Modified license header dates

* Updated WebWhois tests.

* Updated WebWhois tests.

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* SpotlessApply run to fix style issues

* SpotlessApply run to fix style issues

* Added license header and newline where appropriate.

* Added license header and newline where appropriate.

* Javadoc style fix in tests and removed unused methods

* Javadoc style fix in tests and removed unused methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Modified tests for WebWhois according to changes suggested by laijiang.

* Modified tests for WebWhois according to changes suggested by laijiang.

* Removed TestProvider from TestUtils.

* Removed TestProvider from TestUtils.

* Rebased to master

* Updated issues in rebasing

* Minor style change on prober/build.gradle

* Fixed warnings for java compilation

* Fixed files to pass all style tests

* Fixed changes suggested by CydeWeys

* Rebased to Master and added in modified Handlers and ProbingAction

* Added missing license headers and JavaDoc

* Added missing license headers and JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor fix in NewChannelAction JavaDoc

* Minor Style Fix

* Minor Style Fix

* Full WebWhoIs Sequence Added

* Full WebWhoIs Sequence Added

* fixed build issues

* fixed build issues

* Refactored by responses suggested by jianglai.

* Refactored by responses suggested by jianglai.

* Minor Style Fixes

* Minor Style Fixes

* Updated build.gradle file

* Updated build.gradle file

* Modified license header dates

* Modified license header dates

* Updated WebWhois tests.

* Updated WebWhois tests.

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring

* SpotlessApply run to fix style issues

* SpotlessApply run to fix style issues

* Added license header and newline where appropriate.

* Added license header and newline where appropriate.

* Javadoc style fix in tests and removed unused methods

* Javadoc style fix in tests and removed unused methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Refactored ProbingAction to minimize number of unnecessary methods

* Modified tests for WebWhois according to changes suggested by laijiang.

* Modified tests for WebWhois according to changes suggested by laijiang.

* Removed TestProvider from TestUtils.

* Removed TestProvider from TestUtils.

* Rebased to master

* Updated issues in rebasing

* Minor style change on prober/build.gradle

* Fixed warnings for java compilation

* Fixed files to pass all style tests

* Minor syle fixes after succesful rebase onto master
2019-08-08 15:17:41 -04:00
Lai Jiang c7478fc52b Fix expired access token bug in the proxy (#217)
https://github.com/google/nomulus/pull/129 migrated `GoogleCredential`
to `GoogleCredentialsBundle` and introduced a subtle bug. I don't fully
understand why but there are times when the access token is null but
`credentials.refresh()` is not called, resulting in NullPointerException
when `credentials.getAccessToken().getTokenValue()` is called.

Since the new GoogleCredentials class supports `shouldRefresh()`, we now
just rely on it to make sure that we always get a value access token.
2019-08-08 14:44:33 -04:00
Shicong Huang a68b1a12fd Implement TransactionManager for datastore (#207)
This PR created the new interface named TransactionManager which defines
methods to manage transaction. Also, the access to all transaction related
methods of Ofy.java are restricted to package private, and they will be exposed
by DatastoreTransactionManager which is the datastore implementation of
TransactionManager.
2019-08-07 12:59:08 -04:00
Weimin Yu ce791e8b97 Generate code coverage report (#216)
* Generate code coverage report

Enable jacoco, the official Gradle code coverage plugin.

The 'build' task will write a code coverage report to
build/reports/jacoco for each subproject that has tests.
We should consider publish periodical reports to a well known
location.

This change also defines a minimum coverage verification task.
The task is for experiment only, and is not added to the build
process yet.
2019-08-07 11:21:04 -04:00
gbrodman cf3f9603f7 Clean up RDAP action logs and request plain text (#213)
* Remove unused log argument

* Use the right accept-encoding

By default we request gzip and theoretically we'd decode it
automatically on our end but for some reason that's not working. I
tested this on Alpha and it worked
2019-08-06 14:00:09 -04:00
Lai Jiang 4fd32c8b5c Add a Gradle task to deploy the proxy (#214)
Also refactored some common code regarding the mapping between
environments and projects.

The script to update proxy deployments and kill all pods are lifted from
https://cs.corp.google.com/piper///depot/google3/domain/registry/tools/bashrc?l=163
2019-08-06 11:45:34 -04:00
gbrodman 3488f95990 Clean up Gradle stylings and fix issues IDed in Prober (#212) 2019-08-05 15:54:20 -04:00
Shicong Huang 8e00f85f60 Use OpenJDK8 for Gradle build (#210) 2019-08-02 17:00:35 -04:00
Shicong Huang 3cc8d83396 Hard code webdriver docker version (#208) 2019-08-01 14:02:57 -04:00
gbrodman 0a779612f6 Remove old EPP processing time metrics (#206) 2019-07-31 19:18:40 -04:00
gbrodman 545a03618b Add an underline when hovering/focusing on <a> tags (#193) 2019-07-31 17:36:53 -04:00
Ben McIlwain bcdacc88d3 Remove vestigial path for streaming EPP metrics (#184)
* Remove vestigial path for streaming EPP metrics

The relevant action was deleted last year here: google@218c451

This removes the final hanging piece.
2019-07-29 14:37:04 -04:00
gbrodman 56b10ea136 Create a Gradle task to run the test server (#192)
* Create a Gradle task to run the test server

As an artifact of the old build system, the test server relies on having
the built registrar_(bin|dbg)*(\.css)?.js in place (see ConsoleUiAction
among others). As a result, we create a Gradle task that puts those
files into the correct, readable, location before running the test
server.

* Depend on assemble rather than build

* refactor gitignores
2019-07-29 11:03:29 -04:00
Ben McIlwain 9479e1e8b9 Rename Spec11 reply-to email and also use it as sending address (#201)
* Rename Spec11 reply-to email and also use it as sending address
2019-07-26 15:30:46 -04:00
Shicong Huang 110bd9c057 Fix deploying to alpha from workstation (#198)
appengineDeployAll requires appengine.deploy.version to be set
otherwiese the deployment would fail.
2019-07-26 14:37:49 -04:00
Lai Jiang 2ae5aba7ff Rename dependency_license.gradle file (#196)
The file name confused the licensee app
(https://github.com/licensee/licensee) that GitHub uses to detect
license files. It thinks this file is also a license file and was not
able to determine is type.
2019-07-26 14:19:08 -04:00
Lai Jiang b878e5acc1 Remove port to protocol map (#200)
There's no need for it. We can get the port from the protocol.
2019-07-25 22:05:12 -04:00
gbrodman 4d0409c924 Add HTML lang tags where possible (#199) 2019-07-25 21:09:32 -04:00
gbrodman 0292887cb9 Store only interesting failures, not login failures (#188)
Login failures will happen any time that we aren't coming from a
whitelisted IP for that particular TLD. Since whitelists are out of date
(and we don't whitelist IPs for every TLD anyway) those failures aren't
interesting. Store and fully-log the interesting failures if one
happened.
2019-07-24 10:55:55 -04:00
Lai Jiang 464f6ba90a Add a missing space (#197) 2019-07-24 10:50:27 -04:00
Lai Jiang 0ab0a8c2e6 Use positive booleans to improve readability (#190)
This allows us to use lgtm.com to scan for vulnerabilities.
2019-07-23 22:30:46 -04:00
Lai Jiang 2cc4e5fa94 Add a badge for LGTM analysis results (#194) 2019-07-23 21:42:19 -04:00
gbrodman 47a890253e Revert "Use parallel Gradle builds by default (#189)" (#195)
This reverts commit 5dfd96d26d.
2019-07-23 18:12:04 -04:00
Lai Jiang b452b0628d Add customized .lgtm.yml (#191)
* Add customized .lgtm.yml
2019-07-23 17:09:05 -04:00
gbrodman 5dfd96d26d Use parallel Gradle builds by default (#189) 2019-07-23 15:27:30 -04:00
Shicong Huang e2a673d914 Bring back the old GoogleCredential for Drive API (#187)
Using the new GoogleCredentials to access Drive API caused 403 forbidden
exception. So, this PR brought back the old GoogleCredential to
temporarily resolve the production issue while we are figuring out the
long term fix.

TESTED=Deployed to alpha and verified exportPremiumTerms succeeded, see
https://paste.googleplex.com/6153215760400384.
2019-07-23 11:31:35 -04:00
gbrodman bf29d159f9 Fix a few deprecations (#186) 2019-07-22 14:12:55 -04:00
Lai Jiang e17cb52bf7 Fail gracefully when copying detailed reports (#181)
* Fail gracefully when copying detailed reports

When the detailed reports are copied from GCS to registrars' Drive
folders, do not fail the entire copy operation when a single registrar
fails. Instead, send an alert email about the failure, and continue to copy the
rest of the reports.

Also, instead of creating duplicates, overwrite the existing files on
Drive.

BUG=127690361
2019-07-22 14:09:49 -04:00
Lai Jiang 7352f9b4a6 Remove unused local variable (#185) 2019-07-22 10:04:16 -04:00
Lai Jiang 5da48184f9 Merge beam and GAE configs deployment to one GCB job (#182)
* Merge beam and GAE configs deployment to one GCB job

Deployment of GAE configs requires that the credential used by gcloud to
have GAE admin role of the project to be managed. We do not want to
grant the GCB service account that role, because it would all *any* GCB
job to deploy anything to GAE. Instead we use a dedicated credential
originally created to deploy beam pipelines. This credential is
encrypted by KMS and stored on GCS. Since the beam pipeline deployment
GCB job already does the decryption, it make sense to add the config
deployment step there as well. The beam deployment steps are tweaked to
use the nomulus tool docker image instead of the jar file.

Also moved the content of deploy_configs_to_env.sh to the GCB yaml file
itself because the shell script is not uploaded to GC Bat the same time
as the yaml file when the job is triggered by Spinnaker.

Lastly, due to b/137891685, using GCB to deploy cron jobs does not work
as we cannot use service account credential to deploy to projects under
google.com.
2019-07-19 16:54:56 -04:00
gbrodman 5bd2ccd210 Add a Cloud Build task to update YAML configs (#177)
* Add a Cloud Build task to update YAML configs

* CR responses

* Move config deployment to a script

* Pin builder version

* Create different beam and deploy-config files per environment

* Update comments and make a for loop
2019-07-18 12:15:15 -04:00
Lai Jiang 8fd5ab2bec Build proxy image in Gradle (#179) 2019-07-17 20:38:03 -04:00
Lai Jiang 30f6113b05 Upgrade to Gradle 5.5.1 (#178)
Also make the default wrapper type "all" instead of "binary". This is
helpful for IDEs to understand the gradle script.
2019-07-17 17:37:44 -04:00
251 changed files with 11383 additions and 884 deletions
+6
View File
@@ -100,3 +100,9 @@ nomulus.iws
node_modules/**
!node_modules/soyutils_usegoog.js
/repos/
# Compiled JS/CSS code
core/**/registrar_bin*.js
core/**/registrar_dbg*.js
core/**/registrar_bin*.css
core/**/registrar_dbg*.css
+4
View File
@@ -0,0 +1,4 @@
extraction:
java:
prepare:
packages: "npm"
-78
View File
@@ -1,78 +0,0 @@
# Copyright 2018 The Nomulus Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Collecting report files from a travis build to GCS requires setting the
# "CREDZ" and "REPORT_GCS_BUCKET" environment variables in your travis
# repository.
#
# The report destination can be any GCS path (e.g. "gcs://bucket-name/path").
# You'll want to select "Display value in build log", otherwise travis will
# hide the bucket name in the URL that is displayed.
#
# The CREDZ variable should be the contents of a json credentials file for
# a service account with write access to the bucket, escaped for bash shell
# usage (usually just wrapping it in single quotes should suffice).
language: java
jdk:
# Our builds fail against Oracle Java for reasons yet unknown.
- openjdk8
# Caching options suggested by a random article.
before_cache:
- rm -f $HOME/.gradle/caches/modules-2/modules-2.lock
- rm -f $HOME/.gradle/caches/*/plugin-resolution/
cache:
directories:
- $HOME/.gradle/caches/
- $HOME/.gradle/wrapper/
- $HOME/google-cloud-sdk/
# WebDriver tests need Chrome and ChromeDriver provisioned by the docker image
services:
- docker
env:
global:
# Disable fancy status information (looks bad on travis and exceeds logfile
# quota)
- TERM=dumb
# Do not prompt for user input when using any SDK methods.
- CLOUDSDK_CORE_DISABLE_PROMPTS=1
install:
- |
if [ ! -d $HOME/google-cloud-sdk/bin ]
then
# The install script errors if this directory already exists,
# but Travis already creates it when we mark it as cached.
rm -rf $HOME/google-cloud-sdk
# The install script is overly verbose, which sometimes causes
# problems on Travis, so ignore stdout.
curl https://sdk.cloud.google.com | bash
fi
# This line is critical. We setup the SDK to take precedence in our
# environment over the old SDK that is already on the machine.
- source $HOME/google-cloud-sdk/path.bash.inc
- gcloud components install app-engine-java
- gcloud version
# Specialize gradle build to use an up-to-date gradle and the /gradle
# directory.
# The "travis_wait 45" lets our build spend up to 45 minutes without writing
# output, instead of the default 10.
# See notes on the CREDZ and REPORT_GCS_BUCKET environment variable in the
# comments at the top of the file.
script: echo "$CREDZ" >credz.json && chmod 755 ./gradlew && travis_wait 45 ./gradlew build --continue -P uploaderDestination="$REPORT_GCS_DESTINATION" -P uploaderCredentialsFile=credz.json -P uploaderMultithreadedUpload=yes -P mavenUrl=https://storage.googleapis.com/domain-registry-maven-repository/maven -P pluginsUrl=https://storage.googleapis.com/domain-registry-maven-repository/plugins
+3 -3
View File
@@ -1,8 +1,8 @@
# Nomulus
| Internal Build | FOSS Build | License | Code Search |
|----------------|------------|---------|-------------|
|[![Build Status for Google Registry internal build](https://storage.googleapis.com/domain-registry-kokoro/build.svg)](https://storage.googleapis.com/domain-registry-kokoro/index.html)|[![Build Status for the open source build](https://travis-ci.org/google/nomulus.svg?branch=master)](https://travis-ci.org/google/nomulus)|[![License for this repo](https://img.shields.io/github/license/google/nomulus.svg)](https://github.com/google/nomulus/blob/master/LICENSE)|[![Link to Source Graph](https://github.com/sourcegraph/sourcegraph/blob/master/ui/assets/img/sourcegraph-logo.svg)](https://sourcegraph.com/github.com/google/nomulus)|
| Internal Build | FOSS Build | LGTM | License | Code Search |
|----------------|------------|------|---------|-------------|
|[![Build Status for Google Registry internal build](https://storage.googleapis.com/domain-registry-kokoro/internal/build.svg)](https://storage.googleapis.com/domain-registry-kokoro/internal/index.html)|[![Build Status for the open source build](https://storage.googleapis.com/domain-registry-kokoro/foss/build.svg)](https://storage.googleapis.com/domain-registry-kokoro/foss/index.html)|[![Total alerts](https://img.shields.io/lgtm/alerts/g/google/nomulus.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/google/nomulus/alerts/)|[![License for this repo](https://img.shields.io/github/license/google/nomulus.svg)](https://github.com/google/nomulus/blob/master/LICENSE)|[![Link to Source Graph](https://github.com/sourcegraph/sourcegraph/blob/master/ui/assets/img/sourcegraph-logo.svg)](https://sourcegraph.com/github.com/google/nomulus)|
![Nomulus logo](./nomulus-logo.png)
+11 -18
View File
@@ -12,25 +12,12 @@
// See the License for the specific language governing permissions and
// limitations under the License.
def environments = ['production', 'sandbox', 'alpha', 'crash']
def projects = ['production': 'domain-registry',
'sandbox' : 'domain-registry-sandbox',
'alpha' : 'domain-registry-alpha',
'crash' : 'domain-registry-crash']
def environment = rootProject.findProperty("environment")
if (environment == null) {
environment = 'production'
}
def gcpProject = projects[environment]
if (gcpProject == null) {
throw new GradleException("-Penvironment must be one of ${environments}.")
}
apply from: "${rootDir.path}/projects.gradle"
apply plugin: 'war'
def environment = rootProject.environment
def gcpProject = rootProject.gcpProject
// Set this directory before applying the appengine plugin so that the
// plugin will recognize this as an app-engine standard app (and also
// obtains the appengine-web.xml from the correct location)
@@ -75,6 +62,12 @@ if (project.path == ":services:default") {
appengine {
deploy {
// appengineDeployAll task requires the version to be set. So,
// this config lets gcloud select a version name when deploying
// to alpha or sandbox from our workstation.
if (!rootProject.prodOrSandboxEnv) {
version = 'GCLOUD_CONFIG'
}
projectId = gcpProject
}
}
@@ -85,4 +78,4 @@ dependencies {
rootProject.deploy.dependsOn appengineDeployAll
rootProject.stage.dependsOn appengineStage
appengineDeploy.dependsOn rootProject.verifyDeployment
appengineDeployAll.dependsOn rootProject.verifyDeployment
+49 -13
View File
@@ -12,8 +12,11 @@
// See the License for the specific language governing permissions and
// limitations under the License.
import org.gradle.api.tasks.testing.logging.TestExceptionFormat
import org.gradle.api.tasks.testing.logging.TestLogEvent
buildscript {
if (project.disableDependencyLocking.toBoolean() == false) {
if (rootProject.enableDependencyLocking.toBoolean()) {
// Lock buildscript dependencies.
configurations.classpath {
resolutionStrategy.activateDependencyLocking()
@@ -37,13 +40,19 @@ plugins {
id 'net.ltgt.apt' version '0.19' apply false
id 'net.ltgt.errorprone' version '0.6.1'
id 'checkstyle'
id 'com.github.johnrengelman.shadow' version '5.0.0'
id 'com.github.johnrengelman.shadow' version '5.1.0'
// NodeJs plugin
id "com.moowork.node" version "1.2.0"
id 'idea'
id 'com.diffplug.gradle.spotless' version '3.18.0'
id 'jacoco'
}
wrapper {
distributionType = Wrapper.DistributionType.ALL
}
apply plugin: google.registry.gradle.plugin.ReportUploaderPlugin
@@ -71,7 +80,7 @@ reportUploader {
apply from: 'dependencies.gradle'
apply from: 'dependency_license.gradle'
apply from: 'dependency_lic.gradle'
// Custom task to run checkLicense in buildSrc, which is not triggered
// by root project tasks. A shell task is used because buildSrc tasks
@@ -105,10 +114,10 @@ task deploy {
task verifyDeployment {
group = 'deployment'
description = 'Ensure that one can only deploy to alpha.'
description = 'Ensure that one cannot deploy to production or sandbox.'
doFirst {
if (rootProject.findProperty("environment") != 'alpha') {
throw new GradleException("Can only deploy to alpha.");
if (rootProject.prodOrSandboxEnv) {
throw new GradleException("Cannot deploy to production or sandbox.");
}
}
}
@@ -153,8 +162,8 @@ subprojects {
attributes 'Main-Class': mainClass
}
zip64 = true
classifier = null
version = null
classifier = ''
archiveVersion = ''
configurations = [project.configurations.runtimeClasspath]
from project.sourceSets.main.output
// Excludes signature files that accompany some dependency jars, like
@@ -165,7 +174,7 @@ subprojects {
}
}
if (rootProject.disableDependencyLocking.toBoolean() == false) {
if (rootProject.enableDependencyLocking.toBoolean()) {
buildscript {
// Lock buildscript dependencies.
configurations.classpath {
@@ -173,7 +182,7 @@ subprojects {
}
}
// Lock application dependencies except for the gradle-license-report
// plugin. See dependency_license.gradle for more information.
// plugin. See dependency_lic.gradle for more information.
configurations.findAll { it.name != 'dependencyLicenseReport' }.each {
it.resolutionStrategy.activateDependencyLocking()
}
@@ -232,9 +241,7 @@ subprojects {
}
}
if (project.name == 'util') return
if (project.name == 'proxy') return
if (project.name == 'core') return
if (['util', 'proxy', 'core', 'prober'].contains(project.name)) return
test {
testLogging.showStandardStreams = Boolean.parseBoolean(showAllOutput)
@@ -252,6 +259,35 @@ subprojects {
}
}
// If "-P verboseTestOutput=true" is passed in, configure all subprojects to dump all of their
// output and final test status (pass/fail, errors) for each test class.
//
// Note that we can't do this in the main subprojects section above because that's evaluated before
// the subproject build files and the test tasks haven't been defined yet. We have to do it from
// the projectsEvaluted hook, which gets called after the subprojects are configured.
if (verboseTestOutput.toBoolean()) {
gradle.projectsEvaluated({
subprojects {
tasks.withType(Test) {
testLogging {
events TestLogEvent.FAILED, TestLogEvent.PASSED, TestLogEvent.SKIPPED,
TestLogEvent.STANDARD_OUT, TestLogEvent.STANDARD_ERROR
exceptionFormat TestExceptionFormat.FULL
showExceptions true
showCauses true
showStackTraces true
afterSuite { desc, result ->
println "Results: ${result.resultType}, " +
"${result.successfulTestCount}/${result.testCount} tests " +
"passed, ${result.failedTestCount} failures.";
}
}
}
}
})
}
task checkDependenciesDotGradle {
def buildSrcDepsFile = File.createTempFile('buildSrc', 'deps')
buildSrcDepsFile.deleteOnExit()
+3 -3
View File
@@ -13,7 +13,7 @@
// limitations under the License.
buildscript {
if (rootProject.disableDependencyLocking.toBoolean() == false) {
if (project.enableDependencyLocking.toBoolean()) {
// Lock buildscript dependencies.
configurations.classpath {
resolutionStrategy.activateDependencyLocking()
@@ -32,7 +32,7 @@ plugins {
id 'com.diffplug.gradle.spotless' version '3.18.0'
}
if (rootProject.disableDependencyLocking.toBoolean() == false) {
if (rootProject.enableDependencyLocking.toBoolean()) {
// Lock application dependencies.
dependencyLocking {
lockAllConfigurations()
@@ -52,7 +52,7 @@ repositories {
}
apply from: '../dependencies.gradle'
apply from: '../dependency_license.gradle'
apply from: '../dependency_lic.gradle'
apply from: '../java_common.gradle'
sourceSets {
+1 -1
View File
@@ -1 +1 @@
disableDependencyLocking=false
enableDependencyLocking=false
@@ -39,6 +39,9 @@
{
"moduleLicense": "BSD"
},
{
"moduleLicense": "BSD-2-Clause"
},
{
"moduleLicense": "BSD 2-Clause License"
},
@@ -54,6 +57,9 @@
{
"moduleLicense": "BSD 3-Clause License"
},
{
"moduleLicense": "BSD 3-clause New License"
},
{
"moduleLicense": "The 3-Clause BSD License"
},
@@ -69,6 +75,9 @@
{
"moduleLicense": "BSD style"
},
{
"moduleLicense": "BSD-2-Clause"
},
{
"moduleLicense": "New BSD License"
},
@@ -78,6 +87,9 @@
{
"moduleLicense": "The BSD License"
},
{
"moduleLicense": "The PostgreSQL License"
},
{
"moduleLicense": "CC0 1.0 Universal License"
},
@@ -99,6 +111,9 @@
{
"moduleLicense": "\\n Dual license consisting of the CDDL v1.1 and GPL v2\\n "
},
{
"moduleLicense": "Eclipse Distribution License v. 1.0"
},
{
"moduleLicense": "Eclipse Public License - Version 1.0"
},
@@ -108,6 +123,9 @@
{
"moduleLicense": "Eclipse Public License 1.0"
},
{
"moduleLicense": "Eclipse Public License v1.0"
},
{
"moduleLicense": "Google App Engine Terms of Service"
},
@@ -117,12 +135,18 @@
{
"moduleLicense": "GNU GENERAL PUBLIC LICENSE, Version 2 + Classpath Exception"
},
{
"moduleLicense": "GNU Library General Public License v2.1 or later"
},
{
"moduleLicense": "GPL2 w/ CPE"
},
{
"moduleLicense": "GNU Lesser Public License"
},
{
"moduleLicense": "GNU Library General Public License v2.1 or later"
},
{
"moduleLicense": "The Go license"
},
@@ -156,6 +180,9 @@
{
"moduleLicense": "The MIT license"
},
{
"moduleLicense": "The PostgreSQL License"
},
{
"moduleLicense": "Mozilla Public License Version 2.0"
},
+3 -4
View File
@@ -78,7 +78,7 @@ PRESUBMITS = {
r".*Copyright 20\d{2} The Nomulus Authors\. All Rights Reserved\.",
("java", "js", "soy", "sql", "py", "sh", "gradle"), {
".git", "/build/", "/generated/", "node_modules/",
"JUnitBackports.java"
"JUnitBackports.java", "registrar_bin.", "registrar_dbg."
}, REQUIRED):
"File did not include the license header.",
@@ -148,7 +148,7 @@ PRESUBMITS = {
PresubmitCheck(
r".*(innerHTML|outerHTML)\s*(=|[+]=)([^=]|$)",
"js",
{"/node_modules/"},
{"/node_modules/", "registrar_bin."},
):
"Do not assign directly to the dom. Use goog.dom.setTextContent to set"
" to plain text, goog.dom.removeChildren to clear, or "
@@ -156,14 +156,13 @@ PRESUBMITS = {
PresubmitCheck(
r".*console\.(log|info|warn|error)",
"js",
{"/node_modules/", "google/registry/ui/js/util.js"},
{"/node_modules/", "google/registry/ui/js/util.js", "registrar_bin."},
):
"JavaScript files should not include console logging."
}
def get_files():
result = []
for root, dirnames, filenames in os.walk("."):
for filename in filenames:
yield os.path.join(root, filename)
+25 -2
View File
@@ -53,15 +53,19 @@ def fragileTestPatterns = [
// Test Datastore inexplicably aborts transaction.
"google/registry/model/tmch/ClaimsListShardTest.*",
// Creates large object (64MBytes), occasionally throws OOM error.
"google/registry/model/server/KmsSecretRevisionTest.*"
"google/registry/model/server/KmsSecretRevisionTest.*",
"google/registry/tools/GenerateSqlSchemaCommandTest.*",
"google/registry/webdriver/*",
]
// Tests that fail when running Gradle in a docker container, e. g. when
// building the release artifacts in Google Cloud Build.
def dockerIncompatibleTestPatterns = [
// The webdriver tests start headless Chrome in a Docker container,
// resulting in Docker-in-Docker complications.
// resulting in Docker-in-Docker complications. Likewise,
// GenerateSqlSchemaCommandTest launches postgresql in a docker container.
"google/registry/webdriver/*",
"google/registry/tools/GenerateSqlSchemaCommandTest.*",
// PathParameterTest includes tests which validate that file permissions are
// respected. However when running in Docker the user is root by default, so
// every file is read/write-able. There is no way to exclude specific test
@@ -214,13 +218,16 @@ dependencies {
compile deps['org.bouncycastle:bcpg-jdk15on']
testCompile deps['org.bouncycastle:bcpkix-jdk15on']
compile deps['org.bouncycastle:bcprov-jdk15on']
compile deps['org.hibernate:hibernate-core']
compile deps['org.joda:joda-money']
compile deps['org.json:json']
testCompile deps['org.mortbay.jetty:jetty']
runtimeOnly deps['org.postgresql:postgresql']
testCompile deps['org.seleniumhq.selenium:selenium-api']
testCompile deps['org.seleniumhq.selenium:selenium-chrome-driver']
testCompile deps['org.seleniumhq.selenium:selenium-java']
testCompile deps['org.seleniumhq.selenium:selenium-remote-driver']
testCompile deps['org.testcontainers:postgresql']
testCompile deps['org.testcontainers:selenium']
compile deps['xerces:xmlParserAPIs']
compile deps['xpp3:xpp3']
@@ -247,8 +254,10 @@ dependencies {
testCompile deps['org.hamcrest:hamcrest-all']
testCompile deps['org.hamcrest:hamcrest-core']
testCompile deps['org.hamcrest:hamcrest-library']
compile deps['org.hibernate:hibernate-core']
testCompile deps['junit:junit']
testCompile deps['org.mockito:mockito-core']
runtime deps['org.postgresql:postgresql']
// Indirect dependency found by undeclared-dependency check. Such
// dependencies should go after all other compile and testCompile
@@ -651,5 +660,19 @@ task buildToolImage(dependsOn: nomulus, type: Exec) {
commandLine 'docker', 'build', '-t', 'nomulus-tool', '.'
}
task copyJsFilesForTestServer(dependsOn: assemble, type: Copy) {
// Unfortunately the test server relies on having some compiled JS/CSS
// in place, so copy it over here
from "${resourcesDir}/google/registry/ui/"
include '**/*.js'
include '**/*.css'
into "${project.projectDir}/src/main/resources/google/registry/ui/"
}
task runTestServer(dependsOn: copyJsFilesForTestServer, type: JavaExec) {
main = 'google.registry.server.RegistryTestServerMain'
classpath = sourceSets.test.runtimeClasspath
}
project.build.dependsOn buildToolImage
project.build.dependsOn ':stage'
@@ -19,6 +19,7 @@ import static com.google.appengine.api.taskqueue.TaskOptions.Builder.withUrl;
import static google.registry.backup.ExportCommitLogDiffAction.LOWER_CHECKPOINT_TIME_PARAM;
import static google.registry.backup.ExportCommitLogDiffAction.UPPER_CHECKPOINT_TIME_PARAM;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import com.google.common.flogger.FluentLogger;
@@ -63,7 +64,7 @@ public final class CommitLogCheckpointAction implements Runnable {
final CommitLogCheckpoint checkpoint = strategy.computeCheckpoint();
logger.atInfo().log(
"Generated candidate checkpoint for time: %s", checkpoint.getCheckpointTime());
ofy()
tm()
.transact(
() -> {
DateTime lastWrittenTime = CommitLogCheckpointRoot.loadRoot().getLastWrittenTime();
@@ -18,6 +18,7 @@ import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.mapreduce.MapreduceRunner.PARAM_DRY_RUN;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static java.lang.Boolean.FALSE;
import static java.lang.Boolean.TRUE;
@@ -276,7 +277,7 @@ public final class DeleteOldCommitLogsAction implements Runnable {
return;
}
DeletionResult deletionResult = ofy().transactNew(() -> {
DeletionResult deletionResult = tm().transactNew(() -> {
CommitLogManifest manifest = ofy().load().key(manifestKey).now();
// It is possible that the same manifestKey was run twice, if a shard had to be restarted
// or some weird failure. If this happens, we want to exit immediately.
@@ -39,6 +39,7 @@ import static google.registry.model.reporting.HistoryEntry.Type.CONTACT_DELETE;
import static google.registry.model.reporting.HistoryEntry.Type.CONTACT_DELETE_FAILURE;
import static google.registry.model.reporting.HistoryEntry.Type.HOST_DELETE;
import static google.registry.model.reporting.HistoryEntry.Type.HOST_DELETE_FAILURE;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.model.transfer.TransferStatus.SERVER_CANCELLED;
import static java.math.RoundingMode.CEILING;
import static java.util.concurrent.TimeUnit.DAYS;
@@ -308,7 +309,7 @@ public class DeleteContactsAndHostsAction implements Runnable {
final boolean hasNoActiveReferences = !Iterators.contains(values, true);
logger.atInfo().log("Processing async deletion request for %s", deletionRequest.key());
DeletionResult result =
ofy()
tm()
.transactNew(
() -> {
DeletionResult deletionResult =
@@ -329,7 +330,7 @@ public class DeleteContactsAndHostsAction implements Runnable {
private DeletionResult attemptToDeleteResource(
DeletionRequest deletionRequest, boolean hasNoActiveReferences) {
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
EppResource resource =
ofy().load().key(deletionRequest.key()).now().cloneProjectedAtTime(now);
// Double-check transactionally that the resource is still active and in PENDING_DELETE.
@@ -19,6 +19,7 @@ import static google.registry.config.RegistryEnvironment.PRODUCTION;
import static google.registry.mapreduce.MapreduceRunner.PARAM_DRY_RUN;
import static google.registry.mapreduce.inputs.EppResourceInputs.createEntityInput;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import com.google.appengine.tools.mapreduce.Mapper;
@@ -124,7 +125,7 @@ public class DeleteLoadTestDataAction implements Runnable {
Key.create(EppResourceIndex.create(Key.create(resource)));
final Key<? extends ForeignKeyIndex<?>> fki = ForeignKeyIndex.createKey(resource);
int numEntitiesDeleted =
ofy()
tm()
.transact(
() -> {
// This ancestor query selects all descendant entities.
@@ -23,6 +23,7 @@ import static google.registry.model.ResourceTransferUtils.updateForeignKeyIndexD
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.registry.Registries.getTldsOfType;
import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_DELETE;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static google.registry.request.RequestParameters.PARAM_TLDS;
import static org.joda.time.DateTimeZone.UTC;
@@ -219,7 +220,7 @@ public class DeleteProberDataAction implements Runnable {
final Key<? extends ForeignKeyIndex<?>> fki = ForeignKeyIndex.createKey(domain);
int entitiesDeleted =
ofy()
tm()
.transact(
() -> {
// This ancestor query selects all descendant HistoryEntries, BillingEvents,
@@ -245,16 +246,16 @@ public class DeleteProberDataAction implements Runnable {
}
private void softDeleteDomain(final DomainBase domain) {
ofy().transactNew(() -> {
tm().transactNew(() -> {
DomainBase deletedDomain = domain
.asBuilder()
.setDeletionTime(ofy().getTransactionTime())
.setDeletionTime(tm().getTransactionTime())
.setStatusValues(null)
.build();
HistoryEntry historyEntry = new HistoryEntry.Builder()
.setParent(domain)
.setType(DOMAIN_DELETE)
.setModificationTime(ofy().getTransactionTime())
.setModificationTime(tm().getTransactionTime())
.setBySuperuser(true)
.setReason("Deletion of prober data")
.setClientId(registryAdminClientId)
@@ -23,6 +23,7 @@ import static google.registry.model.common.Cursor.CursorType.RECURRING_BILLING;
import static google.registry.model.domain.Period.Unit.YEARS;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_AUTORENEW;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.pricing.PricingEngineProxy.getDomainRenewCost;
import static google.registry.util.CollectionUtils.union;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
@@ -150,7 +151,7 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
}
int numBillingEventsSaved = 0;
try {
numBillingEventsSaved = ofy().transactNew(() -> {
numBillingEventsSaved = tm().transactNew(() -> {
ImmutableSet.Builder<OneTime> syntheticOneTimesBuilder =
new ImmutableSet.Builder<>();
final Registry tld = Registry.get(getTldFromDomainName(recurring.getTargetId()));
@@ -183,7 +184,7 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
HistoryEntry historyEntry = new HistoryEntry.Builder()
.setBySuperuser(false)
.setClientId(recurring.getClientId())
.setModificationTime(ofy().getTransactionTime())
.setModificationTime(tm().getTransactionTime())
.setParent(domainKey)
.setPeriod(Period.create(1, YEARS))
.setReason("Domain autorenewal by ExpandRecurringBillingEventsAction")
@@ -308,7 +309,7 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
logger.atInfo().log(
"Recurring event expansion %s complete for billing event range [%s, %s).",
isDryRun ? "(dry run) " : "", cursorTime, executionTime);
ofy()
tm()
.transact(
() -> {
Cursor cursor = ofy().load().key(Cursor.createGlobalKey(RECURRING_BILLING)).now();
@@ -15,6 +15,7 @@
package google.registry.batch;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.google.appengine.tools.mapreduce.Mapper;
import com.google.common.collect.ImmutableList;
@@ -69,7 +70,7 @@ public class ResaveAllEppResourcesAction implements Runnable {
@Override
public final void map(final Key<EppResource> resourceKey) {
ofy()
tm()
.transact(
() -> {
EppResource projectedResource =
@@ -77,7 +78,7 @@ public class ResaveAllEppResourcesAction implements Runnable {
.load()
.key(resourceKey)
.now()
.cloneProjectedAtTime(ofy().getTransactionTime());
.cloneProjectedAtTime(tm().getTransactionTime());
ofy().save().entity(projectedResource).now();
});
getContext().incrementCounter(String.format("%s entities re-saved", resourceKey.getKind()));
@@ -18,6 +18,7 @@ import static google.registry.batch.AsyncTaskEnqueuer.PARAM_REQUESTED_TIME;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_RESAVE_TIMES;
import static google.registry.batch.AsyncTaskEnqueuer.PARAM_RESOURCE_KEY;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedSet;
@@ -73,11 +74,11 @@ public class ResaveEntityAction implements Runnable {
public void run() {
logger.atInfo().log(
"Re-saving entity %s which was enqueued at %s.", resourceKey, requestedTime);
ofy().transact(() -> {
tm().transact(() -> {
ImmutableObject entity = ofy().load().key(resourceKey).now();
ofy().save().entity(
(entity instanceof EppResource)
? ((EppResource) entity).cloneProjectedAtTime(ofy().getTransactionTime()) : entity
? ((EppResource) entity).cloneProjectedAtTime(tm().getTransactionTime()) : entity
);
if (!resaveTimes.isEmpty()) {
asyncTaskEnqueuer.enqueueAsyncResave(entity, requestedTime, resaveTimes);
@@ -16,6 +16,7 @@ package google.registry.config;
import static java.nio.charset.StandardCharsets.UTF_8;
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.common.collect.ImmutableList;
import dagger.Module;
@@ -68,6 +69,29 @@ public abstract class CredentialModule {
return GoogleCredentialsBundle.create(credential);
}
/**
* Provides the default {@link GoogleCredential} from the Google Cloud runtime for G Suite
* Drive API.
* TODO(b/138195359): Deprecate this credential once we figure out how to use
* {@link GoogleCredentials} for G Suite Drive API.
*/
@GSuiteDriveCredential
@Provides
@Singleton
public static GoogleCredential provideGSuiteDriveCredential(
@Config("defaultCredentialOauthScopes") ImmutableList<String> requiredScopes) {
GoogleCredential credential;
try {
credential = GoogleCredential.getApplicationDefault();
} catch (IOException e) {
throw new RuntimeException(e);
}
if (credential.createScopedRequired()) {
credential = credential.createScoped(requiredScopes);
}
return credential;
}
/**
* Provides a {@link GoogleCredentialsBundle} from the service account's JSON key file.
*
@@ -118,6 +142,13 @@ public abstract class CredentialModule {
@Retention(RetentionPolicy.RUNTIME)
public @interface DefaultCredential {}
/** Dagger qualifier for the credential for G Suite Drive API. */
@Qualifier
@Documented
@Retention(RetentionPolicy.RUNTIME)
public @interface GSuiteDriveCredential {}
/**
* Dagger qualifier for a credential from a service account's JSON key, to be used in non-request
* threads.
@@ -894,9 +894,9 @@ public final class RegistryConfig {
* @see google.registry.reporting.spec11.Spec11EmailUtils
*/
@Provides
@Config("spec11ReplyToEmailAddress")
public static InternetAddress provideSpec11ReplyToEmailAddress(RegistryConfigSettings config) {
return parseEmailAddress(config.misc.spec11ReplyToEmailAddress);
@Config("spec11OutgoingEmailAddress")
public static InternetAddress provideSpec11OutgoingEmailAddress(RegistryConfigSettings config) {
return parseEmailAddress(config.misc.spec11OutgoingEmailAddress);
}
/**
@@ -173,7 +173,7 @@ public class RegistryConfigSettings {
public static class Misc {
public String sheetExportId;
public String alertRecipientEmailAddress;
public String spec11ReplyToEmailAddress;
public String spec11OutgoingEmailAddress;
public int asyncDeleteDelaySeconds;
public int transientFailureRetries;
}
@@ -357,9 +357,9 @@ misc:
# Address we send alert summary emails to.
alertRecipientEmailAddress: email@example.com
# Address to which the Spec 11 emails to registrars should be replied. This needs
# to be a deliverable email address in case the registrars want to contact us.
spec11ReplyToEmailAddress: reply-to@example.com
# Address from which Spec 11 emails to registrars are sent. This needs
# to be a deliverable email address to handle replies from registrars as well.
spec11OutgoingEmailAddress: abuse@example.com
# How long to delay processing of asynchronous deletions. This should always
# be longer than eppResourceCachingSeconds, to prevent deleted contacts or
@@ -13,11 +13,6 @@
<load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>backend-servlet</servlet-name>
<url-pattern>/_dr/task/metrics</url-pattern>
</servlet-mapping>
<!-- RDE -->
<!--
@@ -14,16 +14,16 @@
package google.registry.export;
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.services.drive.Drive;
import dagger.Component;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule;
import google.registry.config.CredentialModule.DefaultCredential;
import google.registry.config.CredentialModule.GSuiteDriveCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.storage.drive.DriveConnection;
import google.registry.util.GoogleCredentialsBundle;
import javax.inject.Singleton;
/** Dagger module for Google {@link Drive} service connection objects. */
@@ -32,13 +32,13 @@ public final class DriveModule {
@Provides
static Drive provideDrive(
@DefaultCredential GoogleCredentialsBundle credentialsBundle,
@GSuiteDriveCredential GoogleCredential googleCredential,
@Config("projectId") String projectId) {
return new Drive.Builder(
credentialsBundle.getHttpTransport(),
credentialsBundle.getJsonFactory(),
credentialsBundle.getHttpRequestInitializer())
googleCredential.getTransport(),
googleCredential.getJsonFactory(),
googleCredential)
.setApplicationName(projectId)
.build();
}
@@ -17,6 +17,7 @@ package google.registry.export;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static google.registry.util.CollectionUtils.nullToEmpty;
import static google.registry.util.RegistrarUtils.normalizeClientId;
@@ -163,7 +164,7 @@ public final class SyncGroupMembersAction implements Runnable {
registrarsToSave.add(result.getKey().asBuilder().setContactsRequireSyncing(false).build());
}
}
ofy().transactNew(() -> ofy().save().entities(registrarsToSave.build()));
tm().transactNew(() -> ofy().save().entities(registrarsToSave.build()));
return errors;
}
@@ -25,6 +25,7 @@ import static google.registry.model.registrar.RegistrarContact.Type.LEGAL;
import static google.registry.model.registrar.RegistrarContact.Type.MARKETING;
import static google.registry.model.registrar.RegistrarContact.Type.TECH;
import static google.registry.model.registrar.RegistrarContact.Type.WHOIS;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import com.google.common.base.Joiner;
@@ -152,7 +153,7 @@ class SyncRegistrarsSheet {
return builder.build();
})
.collect(toImmutableList()));
ofy()
tm()
.transact(
() -> ofy().save().entity(Cursor.createGlobal(SYNC_REGISTRAR_SHEET, executionTime)));
}
@@ -62,24 +62,6 @@ public class EppMetrics {
"count",
LABEL_DESCRIPTORS_BY_TLD);
private static final EventMetric processingTimeByRegistrar =
MetricRegistryImpl.getDefault()
.newEventMetric(
"/epp/processing_time",
"EPP Processing Time By Registrar",
"milliseconds",
LABEL_DESCRIPTORS_BY_REGISTRAR,
DEFAULT_FITTER);
private static final EventMetric processingTimeByTld =
MetricRegistryImpl.getDefault()
.newEventMetric(
"/epp/processing_time_by_tld",
"EPP Processing Time By TLD",
"milliseconds",
LABEL_DESCRIPTORS_BY_TLD,
DEFAULT_FITTER);
private static final EventMetric requestTime =
MetricRegistryImpl.getDefault()
.newEventMetric(
@@ -118,10 +100,7 @@ public class EppMetrics {
long processingTime =
metric.getEndTimestamp().getMillis() - metric.getStartTimestamp().getMillis();
String commandName = metric.getCommandName().orElse("");
processingTimeByRegistrar
.record(processingTime, commandName, metric.getClientId().orElse(""), eppStatusCode);
String tld = metric.getTld().orElse("");
processingTimeByTld.record(processingTime, commandName, tld, eppStatusCode);
requestTime.record(processingTime, commandName, getTrafficType(tld).toString(), eppStatusCode);
}
@@ -14,7 +14,7 @@
package google.registry.flows;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.xml.XmlTransformer.prettyPrint;
import com.google.common.base.Strings;
@@ -80,7 +80,7 @@ public class FlowRunner {
return eppOutput;
}
try {
return ofy()
return tm()
.transact(
() -> {
try {
@@ -23,7 +23,7 @@ import google.registry.model.registrar.Registrar;
public class PasswordOnlyTransportCredentials implements TransportCredentials {
@Override
public void validate(Registrar r, String password) throws AuthenticationErrorException {
if (!r.testPassword(password)) {
if (!r.verifyPassword(password)) {
throw new BadRegistrarPasswordException();
}
}
@@ -19,6 +19,7 @@ import static google.registry.model.EppResourceUtils.loadByForeignKey;
import static google.registry.model.EppResourceUtils.queryForLinkedDomains;
import static google.registry.model.index.ForeignKeyIndex.loadAndGetKey;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Sets;
@@ -77,7 +78,7 @@ public final class ResourceFlowUtils {
final Function<DomainBase, ImmutableSet<?>> getPotentialReferences) throws EppException {
// Enter a transactionless context briefly.
EppException failfastException =
ofy()
tm()
.doTransactionless(
() -> {
final ForeignKeyIndex<R> fki =
@@ -145,7 +145,7 @@ public class TlsCredentials implements TransportCredentials {
private void validatePassword(Registrar registrar, String password)
throws BadRegistrarPasswordException {
if (!registrar.testPassword(password)) {
if (!registrar.verifyPassword(password)) {
throw new BadRegistrarPasswordException();
}
}
@@ -20,6 +20,7 @@ import static google.registry.flows.contact.ContactFlowUtils.validateAsciiPostal
import static google.registry.flows.contact.ContactFlowUtils.validateContactAgainstPolicy;
import static google.registry.model.EppResourceUtils.createRepoId;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.googlecode.objectify.Key;
import google.registry.config.RegistryConfig.Config;
@@ -71,7 +72,7 @@ public final class ContactCreateFlow implements TransactionalFlow {
extensionManager.validate();
validateClientIsLoggedIn(clientId);
Create command = (Create) resourceCommand;
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
verifyResourceDoesNotExist(ContactResource.class, targetId, now, clientId);
ContactResource newContact =
new ContactResource.Builder()
@@ -22,6 +22,7 @@ import static google.registry.flows.ResourceFlowUtils.verifyOptionalAuthInfo;
import static google.registry.flows.ResourceFlowUtils.verifyResourceOwnership;
import static google.registry.model.eppoutput.Result.Code.SUCCESS_WITH_ACTION_PENDING;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
@@ -84,7 +85,7 @@ public final class ContactDeleteFlow implements TransactionalFlow {
extensionManager.register(MetadataExtension.class);
extensionManager.validate();
validateClientIsLoggedIn(clientId);
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
failfastForAsyncDelete(targetId, now, ContactResource.class, DomainBase::getReferencedContacts);
ContactResource existingContact = loadAndVerifyExistence(ContactResource.class, targetId, now);
verifyNoDisallowedStatuses(existingContact, DISALLOWED_STATUSES);
@@ -93,7 +94,7 @@ public final class ContactDeleteFlow implements TransactionalFlow {
verifyResourceOwnership(clientId, existingContact);
}
asyncTaskEnqueuer.enqueueAsyncDelete(
existingContact, ofy().getTransactionTime(), clientId, trid, isSuperuser);
existingContact, tm().getTransactionTime(), clientId, trid, isSuperuser);
ContactResource newContact =
existingContact.asBuilder().addStatusValue(StatusValue.PENDING_DELETE).build();
historyBuilder
@@ -23,6 +23,7 @@ import static google.registry.flows.contact.ContactFlowUtils.createGainingTransf
import static google.registry.flows.contact.ContactFlowUtils.createTransferResponse;
import static google.registry.model.ResourceTransferUtils.approvePendingTransfer;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
@@ -78,7 +79,7 @@ public final class ContactTransferApproveFlow implements TransactionalFlow {
extensionManager.register(MetadataExtension.class);
extensionManager.validate();
validateClientIsLoggedIn(clientId);
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
ContactResource existingContact = loadAndVerifyExistence(ContactResource.class, targetId, now);
verifyOptionalAuthInfo(authInfo, existingContact);
verifyHasPendingTransfer(existingContact);
@@ -23,6 +23,7 @@ import static google.registry.flows.contact.ContactFlowUtils.createLosingTransfe
import static google.registry.flows.contact.ContactFlowUtils.createTransferResponse;
import static google.registry.model.ResourceTransferUtils.denyPendingTransfer;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
@@ -74,7 +75,7 @@ public final class ContactTransferCancelFlow implements TransactionalFlow {
extensionManager.register(MetadataExtension.class);
extensionManager.validate();
validateClientIsLoggedIn(clientId);
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
ContactResource existingContact = loadAndVerifyExistence(ContactResource.class, targetId, now);
verifyOptionalAuthInfo(authInfo, existingContact);
verifyHasPendingTransfer(existingContact);
@@ -23,6 +23,7 @@ import static google.registry.flows.contact.ContactFlowUtils.createGainingTransf
import static google.registry.flows.contact.ContactFlowUtils.createTransferResponse;
import static google.registry.model.ResourceTransferUtils.denyPendingTransfer;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.googlecode.objectify.Key;
import google.registry.flows.EppException;
@@ -72,7 +73,7 @@ public final class ContactTransferRejectFlow implements TransactionalFlow {
extensionManager.register(MetadataExtension.class);
extensionManager.validate();
validateClientIsLoggedIn(clientId);
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
ContactResource existingContact = loadAndVerifyExistence(ContactResource.class, targetId, now);
verifyOptionalAuthInfo(authInfo, existingContact);
verifyHasPendingTransfer(existingContact);
@@ -24,6 +24,7 @@ import static google.registry.flows.contact.ContactFlowUtils.createLosingTransfe
import static google.registry.flows.contact.ContactFlowUtils.createTransferResponse;
import static google.registry.model.eppoutput.Result.Code.SUCCESS_WITH_ACTION_PENDING;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
@@ -91,7 +92,7 @@ public final class ContactTransferRequestFlow implements TransactionalFlow {
extensionManager.register(MetadataExtension.class);
extensionManager.validate();
validateClientIsLoggedIn(gainingClientId);
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
ContactResource existingContact = loadAndVerifyExistence(ContactResource.class, targetId, now);
verifyAuthInfoPresentForResourceTransfer(authInfo);
verifyAuthInfo(authInfo.get(), existingContact);
@@ -25,6 +25,7 @@ import static google.registry.flows.ResourceFlowUtils.verifyResourceOwnership;
import static google.registry.flows.contact.ContactFlowUtils.validateAsciiPostalInfo;
import static google.registry.flows.contact.ContactFlowUtils.validateContactAgainstPolicy;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
@@ -92,7 +93,7 @@ public final class ContactUpdateFlow implements TransactionalFlow {
extensionManager.validate();
validateClientIsLoggedIn(clientId);
Update command = (Update) resourceCommand;
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
ContactResource existingContact = loadAndVerifyExistence(ContactResource.class, targetId, now);
verifyOptionalAuthInfo(authInfo, existingContact);
ImmutableSet<StatusValue> statusToRemove = command.getInnerRemove().getStatusValues();
@@ -42,11 +42,11 @@ import static google.registry.flows.domain.DomainFlowUtils.verifyRegistrarIsActi
import static google.registry.flows.domain.DomainFlowUtils.verifyUnitIsYears;
import static google.registry.model.EppResourceUtils.createDomainRepoId;
import static google.registry.model.eppcommon.StatusValue.SERVER_HOLD;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.registry.Registry.TldState.GENERAL_AVAILABILITY;
import static google.registry.model.registry.Registry.TldState.QUIET_PERIOD;
import static google.registry.model.registry.Registry.TldState.START_DATE_SUNRISE;
import static google.registry.model.registry.label.ReservationType.NAME_COLLISION;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static google.registry.util.DateTimeUtils.leapSafeAddYears;
@@ -221,7 +221,7 @@ public class DomainCreateFlow implements TransactionalFlow {
extensionManager.validate();
validateClientIsLoggedIn(clientId);
verifyRegistrarIsActive(clientId);
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
DomainCommand.Create command = cloneAndLinkReferences((Create) resourceCommand, now);
Period period = command.getPeriod();
verifyUnitIsYears(period);
@@ -34,6 +34,7 @@ import static google.registry.model.eppoutput.Result.Code.SUCCESS_WITH_ACTION_PE
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.reporting.DomainTransactionRecord.TransactionReportField.ADD_FIELDS;
import static google.registry.model.reporting.DomainTransactionRecord.TransactionReportField.RENEW_FIELDS;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.pricing.PricingEngineProxy.getDomainRenewCost;
import static google.registry.util.CollectionUtils.nullToEmpty;
import static google.registry.util.CollectionUtils.union;
@@ -140,7 +141,7 @@ public final class DomainDeleteFlow implements TransactionalFlow {
flowCustomLogic.beforeValidation();
extensionManager.validate();
validateClientIsLoggedIn(clientId);
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
// Loads the target resource if it exists
DomainBase existingDomain = loadAndVerifyExistence(DomainBase.class, targetId, now);
Registry registry = Registry.get(existingDomain.getTld());
@@ -28,7 +28,7 @@ import static google.registry.flows.domain.DomainFlowUtils.validateFeeChallenge;
import static google.registry.flows.domain.DomainFlowUtils.validateRegistrationPeriod;
import static google.registry.flows.domain.DomainFlowUtils.verifyRegistrarIsActive;
import static google.registry.flows.domain.DomainFlowUtils.verifyUnitIsYears;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.leapSafeAddYears;
import com.google.common.collect.ImmutableList;
@@ -137,7 +137,7 @@ public final class DomainRenewFlow implements TransactionalFlow {
extensionManager.validate();
validateClientIsLoggedIn(clientId);
verifyRegistrarIsActive(clientId);
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
Renew command = (Renew) resourceCommand;
// Loads the target resource if it exists
DomainBase existingDomain = loadAndVerifyExistence(DomainBase.class, targetId, now);
@@ -27,6 +27,7 @@ import static google.registry.flows.domain.DomainFlowUtils.verifyPremiumNameIsNo
import static google.registry.flows.domain.DomainFlowUtils.verifyRegistrarIsActive;
import static google.registry.model.ResourceTransferUtils.updateForeignKeyIndexDeletionTime;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableList;
@@ -133,7 +134,7 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow {
validateClientIsLoggedIn(clientId);
verifyRegistrarIsActive(clientId);
Update command = (Update) resourceCommand;
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
DomainBase existingDomain = loadAndVerifyExistence(DomainBase.class, targetId, now);
FeesAndCredits feesAndCredits =
pricingLogic.getRestorePrice(Registry.get(existingDomain.getTld()), targetId, now);
@@ -29,6 +29,7 @@ import static google.registry.model.ResourceTransferUtils.approvePendingTransfer
import static google.registry.model.domain.DomainBase.extendRegistrationWithCap;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.reporting.DomainTransactionRecord.TransactionReportField.TRANSFER_SUCCESSFUL;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.pricing.PricingEngineProxy.getDomainRenewCost;
import static google.registry.util.CollectionUtils.union;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
@@ -102,7 +103,7 @@ public final class DomainTransferApproveFlow implements TransactionalFlow {
extensionManager.register(MetadataExtension.class);
extensionManager.validate();
validateClientIsLoggedIn(clientId);
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
DomainBase existingDomain = loadAndVerifyExistence(DomainBase.class, targetId, now);
verifyOptionalAuthInfo(authInfo, existingDomain);
verifyHasPendingTransfer(existingDomain);
@@ -27,6 +27,7 @@ import static google.registry.flows.domain.DomainTransferUtils.createTransferRes
import static google.registry.model.ResourceTransferUtils.denyPendingTransfer;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.reporting.DomainTransactionRecord.TransactionReportField.TRANSFER_SUCCESSFUL;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableSet;
@@ -87,7 +88,7 @@ public final class DomainTransferCancelFlow implements TransactionalFlow {
extensionManager.register(MetadataExtension.class);
extensionManager.validate();
validateClientIsLoggedIn(clientId);
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
DomainBase existingDomain = loadAndVerifyExistence(DomainBase.class, targetId, now);
verifyOptionalAuthInfo(authInfo, existingDomain);
verifyHasPendingTransfer(existingDomain);
@@ -28,6 +28,7 @@ import static google.registry.model.ResourceTransferUtils.denyPendingTransfer;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.reporting.DomainTransactionRecord.TransactionReportField.TRANSFER_NACKED;
import static google.registry.model.reporting.DomainTransactionRecord.TransactionReportField.TRANSFER_SUCCESSFUL;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.union;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
@@ -89,7 +90,7 @@ public final class DomainTransferRejectFlow implements TransactionalFlow {
extensionManager.register(MetadataExtension.class);
extensionManager.validate();
validateClientIsLoggedIn(clientId);
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
DomainBase existingDomain = loadAndVerifyExistence(DomainBase.class, targetId, now);
Registry registry = Registry.get(existingDomain.getTld());
HistoryEntry historyEntry = buildHistoryEntry(existingDomain, registry, now);
@@ -32,6 +32,7 @@ import static google.registry.flows.domain.DomainTransferUtils.createTransferSer
import static google.registry.model.domain.DomainBase.extendRegistrationWithCap;
import static google.registry.model.eppoutput.Result.Code.SUCCESS_WITH_ACTION_PENDING;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
@@ -143,7 +144,7 @@ public final class DomainTransferRequestFlow implements TransactionalFlow {
extensionManager.validate();
validateClientIsLoggedIn(gainingClientId);
verifyRegistrarIsActive(gainingClientId);
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
DomainBase existingDomain = loadAndVerifyExistence(DomainBase.class, targetId, now);
Optional<DomainTransferRequestSuperuserExtension> superuserExtension =
eppInput.getSingleExtension(DomainTransferRequestSuperuserExtension.class);
@@ -38,7 +38,7 @@ import static google.registry.flows.domain.DomainFlowUtils.validateRegistrantAll
import static google.registry.flows.domain.DomainFlowUtils.validateRequiredContactsPresent;
import static google.registry.flows.domain.DomainFlowUtils.verifyClientUpdateNotProhibited;
import static google.registry.flows.domain.DomainFlowUtils.verifyNotInPendingDelete;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.google.common.net.InternetDomainName;
@@ -154,7 +154,7 @@ public final class DomainUpdateFlow implements TransactionalFlow {
flowCustomLogic.beforeValidation();
extensionManager.validate();
validateClientIsLoggedIn(clientId);
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
Update command = cloneAndLinkReferences((Update) resourceCommand, now);
DomainBase existingDomain = loadAndVerifyExistence(DomainBase.class, targetId, now);
verifyUpdateAllowed(command, existingDomain, now);
@@ -22,6 +22,7 @@ import static google.registry.flows.host.HostFlowUtils.verifySuperordinateDomain
import static google.registry.flows.host.HostFlowUtils.verifySuperordinateDomainOwnership;
import static google.registry.model.EppResourceUtils.createRepoId;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.isNullOrEmpty;
import static google.registry.util.CollectionUtils.union;
@@ -103,7 +104,7 @@ public final class HostCreateFlow implements TransactionalFlow {
extensionManager.validate();
validateClientIsLoggedIn(clientId);
Create command = (Create) resourceCommand;
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
verifyResourceDoesNotExist(HostResource.class, targetId, now, clientId);
// The superordinate domain of the host object if creating an in-bailiwick host, or null if
// creating an external host. This is looked up before we actually create the Host object so
@@ -22,6 +22,7 @@ import static google.registry.flows.ResourceFlowUtils.verifyResourceOwnership;
import static google.registry.flows.host.HostFlowUtils.validateHostName;
import static google.registry.model.eppoutput.Result.Code.SUCCESS_WITH_ACTION_PENDING;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
@@ -85,7 +86,7 @@ public final class HostDeleteFlow implements TransactionalFlow {
extensionManager.register(MetadataExtension.class);
extensionManager.validate();
validateClientIsLoggedIn(clientId);
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
validateHostName(targetId);
failfastForAsyncDelete(targetId, now, HostResource.class, DomainBase::getNameservers);
HostResource existingHost = loadAndVerifyExistence(HostResource.class, targetId, now);
@@ -101,7 +102,7 @@ public final class HostDeleteFlow implements TransactionalFlow {
verifyResourceOwnership(clientId, owningResource);
}
asyncTaskEnqueuer.enqueueAsyncDelete(
existingHost, ofy().getTransactionTime(), clientId, trid, isSuperuser);
existingHost, tm().getTransactionTime(), clientId, trid, isSuperuser);
HostResource newHost =
existingHost.asBuilder().addStatusValue(StatusValue.PENDING_DELETE).build();
historyBuilder
@@ -28,6 +28,7 @@ import static google.registry.flows.host.HostFlowUtils.verifySuperordinateDomain
import static google.registry.flows.host.HostFlowUtils.verifySuperordinateDomainOwnership;
import static google.registry.model.index.ForeignKeyIndex.loadAndGetKey;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.isNullOrEmpty;
import com.google.common.collect.ImmutableSet;
@@ -129,7 +130,7 @@ public final class HostUpdateFlow implements TransactionalFlow {
Update command = (Update) resourceCommand;
Change change = command.getInnerChange();
String suppliedNewHostName = change.getFullyQualifiedHostName();
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
validateHostName(targetId);
HostResource existingHost = loadAndVerifyExistence(HostResource.class, targetId, now);
boolean isHostRename = suppliedNewHostName != null;
@@ -271,7 +272,7 @@ public final class HostUpdateFlow implements TransactionalFlow {
}
// We must also enqueue updates for all domains that use this host as their nameserver so
// that their NS records can be updated to point at the new name.
asyncTaskEnqueuer.enqueueAsyncDnsRefresh(existingHost, ofy().getTransactionTime());
asyncTaskEnqueuer.enqueueAsyncDnsRefresh(existingHost, tm().getTransactionTime());
}
}
@@ -21,6 +21,7 @@ import static google.registry.model.eppoutput.Result.Code.SUCCESS_WITH_NO_MESSAG
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.poll.PollMessageExternalKeyConverter.makePollMessageExternalId;
import static google.registry.model.poll.PollMessageExternalKeyConverter.parsePollMessageExternalId;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import com.googlecode.objectify.Key;
@@ -78,7 +79,7 @@ public class PollAckFlow implements TransactionalFlow {
throw new InvalidMessageIdException(messageId);
}
final DateTime now = ofy().getTransactionTime();
final DateTime now = tm().getTransactionTime();
// Load the message to be acked. If a message is queued to be delivered in the future, we treat
// it as if it doesn't exist yet. Same for if the message ID year isn't the same as the actual
@@ -124,7 +125,7 @@ public class PollAckFlow implements TransactionalFlow {
// acked, then we return a special status code indicating that. Note that the query will
// include the message being acked.
int messageCount = ofy().doTransactionless(() -> getPollMessagesQuery(clientId, now).count());
int messageCount = tm().doTransactionless(() -> getPollMessagesQuery(clientId, now).count());
if (!includeAckedMessageInCount) {
messageCount--;
}
@@ -33,6 +33,7 @@ import static google.registry.keyring.kms.KmsKeyring.StringKeyLabel.RDE_SSH_CLIE
import static google.registry.keyring.kms.KmsKeyring.StringKeyLabel.RDE_SSH_CLIENT_PUBLIC_STRING;
import static google.registry.keyring.kms.KmsKeyring.StringKeyLabel.SAFE_BROWSING_API_KEY;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.google.common.collect.ImmutableMap;
@@ -182,7 +183,7 @@ public final class KmsUpdater {
*/
private static void persistEncryptedValues(
final ImmutableMap<String, EncryptResponse> encryptedValues) {
ofy()
tm()
.transact(
() -> {
for (Map.Entry<String, EncryptResponse> entry : encryptedValues.entrySet()) {
@@ -22,6 +22,7 @@ import static com.google.common.collect.Sets.union;
import static google.registry.config.RegistryConfig.getEppResourceCachingDuration;
import static google.registry.config.RegistryConfig.getEppResourceMaxCachedEntries;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.nullToEmpty;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableCopy;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
@@ -49,10 +50,13 @@ import java.util.Map.Entry;
import java.util.Optional;
import java.util.Set;
import java.util.concurrent.ExecutionException;
import javax.persistence.MappedSuperclass;
import javax.persistence.Transient;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/** An EPP entity object (i.e. a domain, contact, or host). */
@MappedSuperclass
public abstract class EppResource extends BackupGroupRoot implements Buildable {
/**
@@ -61,8 +65,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
* <p>This is in the (\w|_){1,80}-\w{1,8} format specified by RFC 5730 for roidType.
* @see <a href="https://tools.ietf.org/html/rfc5730">RFC 5730</a>
*/
@Id
String repoId;
@Id @javax.persistence.Id String repoId;
/** The ID of the registrar that is currently sponsoring this resource. */
@Index
@@ -84,8 +87,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
// Map the method to XML, not the field, because if we map the field (with an adaptor class) it
// will never be omitted from the xml even if the timestamp inside creationTime is null and we
// return null from the adaptor. (Instead it gets written as an empty tag.)
@Index
CreateAutoTimestamp creationTime = CreateAutoTimestamp.create(null);
@Index @Transient CreateAutoTimestamp creationTime = CreateAutoTimestamp.create(null);
/**
* The time when this resource was or will be deleted.
@@ -114,7 +116,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
DateTime lastEppUpdateTime;
/** Status values associated with this resource. */
Set<StatusValue> status;
@Transient Set<StatusValue> status;
/**
* Sorted map of {@link DateTime} keys (modified time) to {@link CommitLogManifest} entries.
@@ -325,13 +327,13 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
@Override
public EppResource load(Key<? extends EppResource> key) {
return ofy().doTransactionless(() -> ofy().load().key(key).now());
return tm().doTransactionless(() -> ofy().load().key(key).now());
}
@Override
public Map<Key<? extends EppResource>, EppResource> loadAll(
Iterable<? extends Key<? extends EppResource>> keys) {
return ofy().doTransactionless(() -> loadMultiple(keys));
return tm().doTransactionless(() -> loadMultiple(keys));
}
};
@@ -21,6 +21,7 @@ import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.registry.Registry.TldState.GENERAL_AVAILABILITY;
import static google.registry.model.registry.Registry.TldState.START_DATE_SUNRISE;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import com.google.common.collect.ImmutableList;
@@ -236,7 +237,7 @@ public final class OteAccountBuilder {
*/
public ImmutableMap<String, String> buildAndPersist() {
// save all the entitiesl in a single transaction
ofy().transact(this::saveAllEntities);
tm().transact(this::saveAllEntities);
return clientIdToTld;
}
@@ -249,7 +250,7 @@ public final class OteAccountBuilder {
/** Saves all the OT&amp;E entities we created. */
private void saveAllEntities() {
ofy().assertInTransaction();
tm().assertInTransaction();
ImmutableList<Registry> registries = ImmutableList.of(sunriseTld, gaTld, eapTld);
ImmutableList<RegistrarContact> contacts = contactsBuilder.build();
@@ -17,6 +17,7 @@ package google.registry.model.common;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.model.ofy.ObjectifyService.allocateId;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.google.appengine.api.users.User;
import com.google.common.base.Splitter;
@@ -55,13 +56,13 @@ public class GaeUserIdConverter extends ImmutableObject {
try {
// Perform these operations in a transactionless context to avoid enlisting in some outer
// transaction (if any).
ofy().doTransactionless(() -> ofy().saveWithoutBackup().entity(gaeUserIdConverter).now());
tm().doTransactionless(() -> ofy().saveWithoutBackup().entity(gaeUserIdConverter).now());
// The read must be done in its own transaction to avoid reading from the session cache.
return ofy()
return tm()
.transactNew(() -> ofy().load().entity(gaeUserIdConverter).safe().user.getUserId());
} finally {
ofy().doTransactionless(() -> ofy().deleteWithoutBackup().entity(gaeUserIdConverter).now());
tm().doTransactionless(() -> ofy().deleteWithoutBackup().entity(gaeUserIdConverter).now());
}
}
}
@@ -69,6 +69,7 @@ import java.util.Optional;
import java.util.Set;
import java.util.function.Predicate;
import javax.annotation.Nullable;
import javax.persistence.Transient;
import org.joda.time.DateTime;
import org.joda.time.Interval;
@@ -83,6 +84,8 @@ import org.joda.time.Interval;
*/
@ReportedOn
@Entity
@javax.persistence.Entity
@javax.persistence.Table(name = "domain")
@ExternalMessagingName("domain")
public class DomainBase extends EppResource
implements ForeignKeyedEppResource, ResourceWithTransferData {
@@ -115,18 +118,17 @@ public class DomainBase extends EppResource
String tld;
/** References to hosts that are the nameservers for the domain. */
@Index
Set<Key<HostResource>> nsHosts;
@Index @Transient Set<Key<HostResource>> nsHosts;
/**
* The union of the contacts visible via {@link #getContacts} and {@link #getRegistrant}.
*
* <p>These are stored in one field so that we can query across all contacts at once.
*/
Set<DesignatedContact> allContacts;
@Transient Set<DesignatedContact> allContacts;
/** Authorization info (aka transfer secret) of the domain. */
DomainAuthInfo authInfo;
@Transient DomainAuthInfo authInfo;
/**
* Data used to construct DS records for this domain.
@@ -134,14 +136,13 @@ public class DomainBase extends EppResource
* <p>This is {@literal @}XmlTransient because it needs to be returned under the "extension" tag
* of an info response rather than inside the "infData" tag.
*/
Set<DelegationSignerData> dsData;
@Transient Set<DelegationSignerData> dsData;
/**
* The claims notice supplied when this application or domain was created, if there was one. It's
* {@literal @}XmlTransient because it's not returned in an info response.
*/
@IgnoreSave(IfNull.class)
LaunchNotice launchNotice;
@IgnoreSave(IfNull.class) @Transient LaunchNotice launchNotice;
/**
* Name of first IDN table associated with TLD that matched the characters in this domain label.
@@ -152,7 +153,7 @@ public class DomainBase extends EppResource
String idnTableName;
/** Fully qualified host names of this domain's active subordinate hosts. */
Set<String> subordinateHosts;
@Transient Set<String> subordinateHosts;
/** When this domain's registration will expire. */
DateTime registrationExpirationTime;
@@ -187,7 +188,7 @@ public class DomainBase extends EppResource
Key<PollMessage.Autorenew> autorenewPollMessage;
/** The unexpired grace periods for this domain (some of which may not be active yet). */
Set<GracePeriod> gracePeriods;
@Transient Set<GracePeriod> gracePeriods;
/**
* The id of the signed mark that was used to create this domain in sunrise.
@@ -198,7 +199,7 @@ public class DomainBase extends EppResource
String smdId;
/** Data about any pending or past transfers on this domain. */
TransferData transferData;
@Transient TransferData transferData;
/**
* The time that this resource was last transferred.
@@ -15,9 +15,9 @@
package google.registry.model.domain.fee;
import com.google.common.base.Ascii;
import com.google.common.base.CharMatcher;
import google.registry.model.ImmutableObject;
import google.registry.model.domain.fee.FeeQueryCommandExtensionItem.CommandName;
import java.util.Locale;
import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.bind.annotation.XmlValue;
@@ -47,7 +47,7 @@ public class FeeExtensionCommandDescriptor extends ImmutableObject {
public CommandName getCommand() {
// Require the xml string to be lowercase.
if (command != null && CharMatcher.javaLowerCase().matchesAllOf(command)) {
if (command != null && command.toLowerCase(Locale.ENGLISH).equals(command)) {
try {
return CommandName.valueOf(Ascii.toUpperCase(command));
} catch (IllegalArgumentException e) {
@@ -15,9 +15,9 @@
package google.registry.model.domain.fee12;
import com.google.common.base.Ascii;
import com.google.common.base.CharMatcher;
import google.registry.model.domain.Period;
import google.registry.model.domain.fee.FeeCheckCommandExtensionItem;
import java.util.Locale;
import java.util.Optional;
import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.bind.annotation.XmlElement;
@@ -83,7 +83,7 @@ public class FeeCheckCommandExtensionItemV12 extends FeeCheckCommandExtensionIte
@Override
public CommandName getCommandName() {
// Require the xml string to be lowercase.
if (commandName != null && CharMatcher.javaLowerCase().matchesAllOf(commandName)) {
if (commandName != null && commandName.toLowerCase(Locale.ENGLISH).equals(commandName)) {
try {
return CommandName.valueOf(Ascii.toUpperCase(commandName));
} catch (IllegalArgumentException e) {
@@ -19,6 +19,7 @@ import static com.google.common.collect.Maps.filterValues;
import static google.registry.config.RegistryConfig.getEppResourceCachingDuration;
import static google.registry.config.RegistryConfig.getEppResourceMaxCachedEntries;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.TypeUtils.instantiate;
import static java.util.concurrent.TimeUnit.MILLISECONDS;
@@ -188,7 +189,7 @@ public abstract class ForeignKeyIndex<E extends EppResource> extends BackupGroup
@Override
public Optional<ForeignKeyIndex<?>> load(Key<ForeignKeyIndex<?>> key) {
return Optional.ofNullable(ofy().doTransactionless(() -> ofy().load().key(key).now()));
return Optional.ofNullable(tm().doTransactionless(() -> ofy().load().key(key).now()));
}
@Override
@@ -196,7 +197,7 @@ public abstract class ForeignKeyIndex<E extends EppResource> extends BackupGroup
Iterable<? extends Key<ForeignKeyIndex<?>>> keys) {
ImmutableSet<Key<ForeignKeyIndex<?>>> typedKeys = ImmutableSet.copyOf(keys);
Map<Key<ForeignKeyIndex<?>>, ForeignKeyIndex<?>> existingFkis =
ofy().doTransactionless(() -> ofy().load().keys(typedKeys));
tm().doTransactionless(() -> ofy().load().keys(typedKeys));
// ofy() omits keys that don't have values in Datastore, so re-add them in
// here with Optional.empty() values.
return Maps.asMap(
@@ -250,7 +251,7 @@ public abstract class ForeignKeyIndex<E extends EppResource> extends BackupGroup
public static <E extends EppResource> Map<String, ForeignKeyIndex<E>> loadCached(
Class<E> clazz, Iterable<String> foreignKeys, final DateTime now) {
if (!RegistryConfig.isEppResourceCachingEnabled()) {
return ofy().doTransactionless(() -> load(clazz, foreignKeys, now));
return tm().doTransactionless(() -> load(clazz, foreignKeys, now));
}
ImmutableList<Key<ForeignKeyIndex<?>>> fkiKeys =
Streams.stream(foreignKeys)
@@ -28,10 +28,9 @@ import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.VoidWork;
import com.googlecode.objectify.Work;
import google.registry.model.BackupGroupRoot;
import google.registry.model.ImmutableObject;
import google.registry.model.transaction.TransactionManager.Work;
import google.registry.util.Clock;
import java.util.HashSet;
import java.util.Map;
@@ -40,7 +39,7 @@ import java.util.Set;
import org.joda.time.DateTime;
/** Wrapper for {@link Work} that associates a time with each attempt. */
class CommitLoggedWork<R> extends VoidWork {
class CommitLoggedWork<R> implements Runnable {
private final Work<R> work;
private final Clock clock;
@@ -74,8 +73,8 @@ class CommitLoggedWork<R> extends VoidWork {
*/
protected ImmutableSet<ImmutableObject> mutations = ImmutableSet.of();
/** Lifecycle marker to track whether {@link #vrun} has been called. */
private boolean vrunCalled;
/** Lifecycle marker to track whether {@link #run} has been called. */
private boolean runCalled;
CommitLoggedWork(Work<R> work, Clock clock) {
this.work = work;
@@ -87,26 +86,26 @@ class CommitLoggedWork<R> extends VoidWork {
}
boolean hasRun() {
return vrunCalled;
return runCalled;
}
R getResult() {
checkState(vrunCalled, "Cannot call getResult() before vrun()");
checkState(runCalled, "Cannot call getResult() before run()");
return result;
}
CommitLogManifest getManifest() {
checkState(vrunCalled, "Cannot call getManifest() before vrun()");
checkState(runCalled, "Cannot call getManifest() before run()");
return manifest;
}
ImmutableSet<ImmutableObject> getMutations() {
checkState(vrunCalled, "Cannot call getMutations() before vrun()");
checkState(runCalled, "Cannot call getMutations() before run()");
return mutations;
}
@Override
public void vrun() {
public void run() {
// The previous time will generally be null, except when using transactNew.
TransactionInfo previous = Ofy.TRANSACTION_INFO.get();
// Set the time to be used for "now" within the transaction.
@@ -117,7 +116,7 @@ class CommitLoggedWork<R> extends VoidWork {
} finally {
Ofy.TRANSACTION_INFO.set(previous);
}
vrunCalled = true;
runCalled = true;
}
/** Records all mutations enrolled by this transaction to a {@link CommitLogManifest} entry. */
@@ -0,0 +1,86 @@
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.ofy;
import static google.registry.model.ofy.ObjectifyService.ofy;
import google.registry.model.transaction.TransactionManager;
import org.joda.time.DateTime;
/** Datastore implementation of {@link TransactionManager}. */
public class DatastoreTransactionManager implements TransactionManager {
private Ofy injectedOfy;
/** Constructs an instance. */
public DatastoreTransactionManager(Ofy injectedOfy) {
this.injectedOfy = injectedOfy;
}
private Ofy getOfy() {
return injectedOfy == null ? ofy() : injectedOfy;
}
@Override
public boolean inTransaction() {
return getOfy().inTransaction();
}
@Override
public void assertInTransaction() {
getOfy().assertInTransaction();
}
@Override
public <T> T transact(Work<T> work) {
return getOfy().transact(work);
}
@Override
public void transact(Runnable work) {
getOfy().transact(work);
}
@Override
public <T> T transactNew(Work<T> work) {
return getOfy().transactNew(work);
}
@Override
public void transactNew(Runnable work) {
getOfy().transactNew(work);
}
@Override
public <R> R transactNewReadOnly(Work<R> work) {
return getOfy().transactNewReadOnly(work);
}
@Override
public void transactNewReadOnly(Runnable work) {
getOfy().transactNewReadOnly(work);
}
@Override
public <R> R doTransactionless(Work<R> work) {
return getOfy().doTransactionless(work);
}
@Override
public DateTime getTransactionTime() {
return getOfy().getTransactionTime();
}
}
@@ -32,13 +32,13 @@ import com.google.common.flogger.FluentLogger;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.Objectify;
import com.googlecode.objectify.ObjectifyFactory;
import com.googlecode.objectify.Work;
import com.googlecode.objectify.cmd.Deleter;
import com.googlecode.objectify.cmd.Loader;
import com.googlecode.objectify.cmd.Saver;
import google.registry.model.annotations.NotBackedUp;
import google.registry.model.annotations.VirtualEntity;
import google.registry.model.ofy.ReadOnlyWork.KillTransactionException;
import google.registry.model.transaction.TransactionManager.Work;
import google.registry.util.Clock;
import google.registry.util.NonFinalForTesting;
import google.registry.util.Sleeper;
@@ -111,11 +111,11 @@ public class Ofy {
ofy().clear();
}
public boolean inTransaction() {
boolean inTransaction() {
return ofy().getTransaction() != null;
}
public void assertInTransaction() {
void assertInTransaction() {
checkState(inTransaction(), "Must be called in a transaction");
}
@@ -194,7 +194,7 @@ public class Ofy {
}
/** Execute a transaction. */
public <R> R transact(Work<R> work) {
<R> R transact(Work<R> work) {
// If we are already in a transaction, don't wrap in a CommitLoggedWork.
return inTransaction() ? work.run() : transactNew(work);
}
@@ -205,7 +205,7 @@ public class Ofy {
* <p>This overload is used for transactions that don't return a value, formerly implemented using
* VoidWork.
*/
public void transact(Runnable work) {
void transact(Runnable work) {
transact(
() -> {
work.run();
@@ -214,7 +214,7 @@ public class Ofy {
}
/** Pause the current transaction (if any) and complete this one before returning to it. */
public <R> R transactNew(Work<R> work) {
<R> R transactNew(Work<R> work) {
// Wrap the Work in a CommitLoggedWork so that we can give transactions a frozen view of time
// and maintain commit logs for them.
return transactCommitLoggedWork(new CommitLoggedWork<>(work, getClock()));
@@ -226,7 +226,7 @@ public class Ofy {
* <p>This overload is used for transactions that don't return a value, formerly implemented using
* VoidWork.
*/
public void transactNew(Runnable work) {
void transactNew(Runnable work) {
transactNew(
() -> {
work.run();
@@ -246,7 +246,10 @@ public class Ofy {
true;
attempt++, sleepMillis *= 2) {
try {
ofy().transactNew(work);
ofy().transactNew(() -> {
work.run();
return null;
});
return work.getResult();
} catch (TransientFailureException
| TimestampInversionException
@@ -295,10 +298,13 @@ public class Ofy {
}
/** A read-only transaction is useful to get strongly consistent reads at a shared timestamp. */
public <R> R transactNewReadOnly(Work<R> work) {
<R> R transactNewReadOnly(Work<R> work) {
ReadOnlyWork<R> readOnlyWork = new ReadOnlyWork<>(work, getClock());
try {
ofy().transactNew(readOnlyWork);
ofy().transactNew(() -> {
readOnlyWork.run();
return null;
});
} catch (TransientFailureException | DatastoreTimeoutException | DatastoreFailureException e) {
// These are always retryable for a read-only operation.
return transactNewReadOnly(work);
@@ -309,7 +315,7 @@ public class Ofy {
throw new AssertionError(); // How on earth did we get here?
}
public void transactNewReadOnly(Runnable work) {
void transactNewReadOnly(Runnable work) {
transactNewReadOnly(
() -> {
work.run();
@@ -318,7 +324,7 @@ public class Ofy {
}
/** Execute some work in a transactionless context. */
public <R> R doTransactionless(Work<R> work) {
<R> R doTransactionless(Work<R> work) {
try {
com.googlecode.objectify.ObjectifyService.push(
com.googlecode.objectify.ObjectifyService.ofy().transactionless());
@@ -347,7 +353,7 @@ public class Ofy {
}
/** Get the time associated with the start of this particular transaction attempt. */
public DateTime getTransactionTime() {
DateTime getTransactionTime() {
assertInTransaction();
return TRANSACTION_INFO.get().transactionTime;
}
@@ -14,7 +14,7 @@
package google.registry.model.ofy;
import com.googlecode.objectify.Work;
import google.registry.model.transaction.TransactionManager.Work;
import google.registry.util.Clock;
/** Wrapper for {@link Work} that disallows mutations and fails the transaction at the end. */
@@ -30,8 +30,8 @@ class ReadOnlyWork<R> extends CommitLoggedWork<R> {
}
@Override
public void vrun() {
super.vrun();
public void run() {
super.run();
throw new KillTransactionException();
}
@@ -19,6 +19,7 @@ import static com.google.common.base.Verify.verify;
import static com.google.common.base.Verify.verifyNotNull;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.rde.RdeNamingUtils.makePartialName;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.google.common.base.VerifyException;
import com.googlecode.objectify.annotation.Entity;
@@ -70,7 +71,7 @@ public final class RdeRevision extends ImmutableObject {
public static void saveRevision(String tld, DateTime date, RdeMode mode, int revision) {
checkArgument(revision >= 0, "Negative revision: %s", revision);
String triplet = makePartialName(tld, date, mode);
ofy().assertInTransaction();
tm().assertInTransaction();
RdeRevision object = ofy().load().type(RdeRevision.class).id(triplet).now();
if (revision == 0) {
verify(object == null, "RdeRevision object already created: %s", object);
@@ -31,12 +31,14 @@ import static google.registry.model.CacheUtils.memoizeWithShortExpiration;
import static google.registry.model.common.EntityGroupRoot.getCrossTldKey;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.registry.Registries.assertTldsExist;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableCopy;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableSortedCopy;
import static google.registry.util.PasswordUtils.SALT_SUPPLIER;
import static google.registry.util.PasswordUtils.hashPassword;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import static google.registry.util.X509Utils.getCertificateHash;
import static google.registry.util.X509Utils.loadCertificate;
import static java.nio.charset.StandardCharsets.UTF_8;
import static java.util.Comparator.comparing;
import static java.util.function.Predicate.isEqual;
@@ -72,10 +74,6 @@ import google.registry.model.common.EntityGroupRoot;
import google.registry.model.registrar.Registrar.BillingAccountEntry.CurrencyMapper;
import google.registry.model.registry.Registry;
import google.registry.util.CidrAddressBlock;
import google.registry.util.NonFinalForTesting;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.cert.CertificateParsingException;
import java.util.Comparator;
import java.util.List;
@@ -206,7 +204,7 @@ public class Registrar extends ImmutableObject implements Buildable, Jsonifiable
private static final Supplier<ImmutableMap<String, Registrar>> CACHE_BY_CLIENT_ID =
memoizeWithShortExpiration(
() ->
ofy()
tm()
.doTransactionless(
() -> Maps.uniqueIndex(loadAll(), Registrar::getClientId)));
@@ -408,14 +406,8 @@ public class Registrar extends ImmutableObject implements Buildable, Jsonifiable
*/
boolean contactsRequireSyncing = true;
@NonFinalForTesting
private static Supplier<byte[]> saltSupplier =
() -> {
// There are 32 bytes in a sha-256 hash, and the salt should generally be the same size.
byte[] salt = new byte[32];
new SecureRandom().nextBytes(salt);
return salt;
};
/** Whether or not registry lock is allowed for this registrar. */
boolean registryLockAllowed = false;
public String getClientId() {
return clientIdentifier;
@@ -544,6 +536,10 @@ public class Registrar extends ImmutableObject implements Buildable, Jsonifiable
return contactsRequireSyncing;
}
public boolean isRegistryLockAllowed() {
return registryLockAllowed;
}
public String getUrl() {
return url;
}
@@ -626,16 +622,6 @@ public class Registrar extends ImmutableObject implements Buildable, Jsonifiable
.build();
}
private String hashPassword(String password) {
try {
return base64()
.encode(MessageDigest.getInstance("SHA-256").digest((password + salt).getBytes(UTF_8)));
} catch (NoSuchAlgorithmException e) {
// All implementations of MessageDigest are required to support SHA-256.
throw new RuntimeException(e);
}
}
private static String checkValidPhoneNumber(String phoneNumber) {
checkArgument(
E164_PATTERN.matcher(phoneNumber).matches(),
@@ -644,8 +630,8 @@ public class Registrar extends ImmutableObject implements Buildable, Jsonifiable
return phoneNumber;
}
public boolean testPassword(String password) {
return hashPassword(password).equals(passwordHash);
public boolean verifyPassword(String password) {
return hashPassword(password, salt).equals(passwordHash);
}
public String getPhonePasscode() {
@@ -880,8 +866,8 @@ public class Registrar extends ImmutableObject implements Buildable, Jsonifiable
checkArgument(
Range.closed(6, 16).contains(nullToEmpty(password).length()),
"Password must be 6-16 characters long.");
getInstance().salt = base64().encode(saltSupplier.get());
getInstance().passwordHash = getInstance().hashPassword(password);
getInstance().salt = base64().encode(SALT_SUPPLIER.get());
getInstance().passwordHash = hashPassword(password, getInstance().salt);
return this;
}
@@ -894,6 +880,11 @@ public class Registrar extends ImmutableObject implements Buildable, Jsonifiable
return this;
}
public Builder setRegistryLockAllowed(boolean registryLockAllowed) {
getInstance().registryLockAllowed = registryLockAllowed;
return this;
}
/** Build the registrar, nullifying empty fields. */
@Override
public Registrar build() {
@@ -14,12 +14,18 @@
package google.registry.model.registrar;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Strings.isNullOrEmpty;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.collect.Sets.difference;
import static com.google.common.io.BaseEncoding.base64;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.registrar.Registrar.checkValidEmail;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableSortedCopy;
import static google.registry.util.PasswordUtils.SALT_SUPPLIER;
import static google.registry.util.PasswordUtils.hashPassword;
import static java.util.stream.Collectors.joining;
import com.google.common.base.Enums;
@@ -134,6 +140,21 @@ public class RegistrarContact extends ImmutableObject implements Jsonifiable {
*/
boolean visibleInDomainWhoisAsAbuse = false;
/**
* Whether or not the contact is allowed to set their registry lock password through the registrar
* console. This will be set to false on contact creation and when the user sets a password.
*/
boolean allowedToSetRegistryLockPassword = false;
/**
* A hashed password that exists iff this contact is registry-lock-enabled. The hash is a base64
* encoded SHA256 string.
*/
String registryLockPasswordHash;
/** Randomly generated hash salt. */
String registryLockPasswordSalt;
public static ImmutableSet<Type> typesFromCSV(String csv) {
return typesFromStrings(Arrays.asList(csv.split(",")));
}
@@ -154,7 +175,7 @@ public class RegistrarContact extends ImmutableObject implements Jsonifiable {
*/
public static void updateContacts(
final Registrar registrar, final Set<RegistrarContact> contacts) {
ofy()
tm()
.transact(
() -> {
ofy()
@@ -212,19 +233,39 @@ public class RegistrarContact extends ImmutableObject implements Jsonifiable {
return new Builder(clone(this));
}
public boolean isAllowedToSetRegistryLockPassword() {
return allowedToSetRegistryLockPassword;
}
public boolean isRegistryLockAllowed() {
return !isNullOrEmpty(registryLockPasswordHash) && !isNullOrEmpty(registryLockPasswordSalt);
}
public boolean verifyRegistryLockPassword(String registryLockPassword) {
if (isNullOrEmpty(registryLockPassword)
|| isNullOrEmpty(registryLockPasswordSalt)
|| isNullOrEmpty(registryLockPasswordHash)) {
return false;
}
return hashPassword(registryLockPassword, registryLockPasswordSalt)
.equals(registryLockPasswordHash);
}
/**
* Returns a string representation that's human friendly.
*
* <p>The output will look something like this:<pre> {@code
* <p>The output will look something like this:
*
* Some Person
* person@example.com
* Tel: +1.2125650666
* Types: [ADMIN, WHOIS]
* Visible in WHOIS as Admin contact: Yes
* Visible in WHOIS as Technical contact: No
* GAE-UserID: 1234567890
* Registrar-Console access: Yes}</pre>
* <pre>{@code
* Some Person
* person@example.com
* Tel: +1.2125650666
* Types: [ADMIN, WHOIS]
* Visible in WHOIS as Admin contact: Yes
* Visible in WHOIS as Technical contact: No
* GAE-UserID: 1234567890
* Registrar-Console access: Yes
* }</pre>
*/
public String toStringMultilinePlainText() {
StringBuilder result = new StringBuilder(256);
@@ -272,6 +313,7 @@ public class RegistrarContact extends ImmutableObject implements Jsonifiable {
.put("visibleInWhoisAsAdmin", visibleInWhoisAsAdmin)
.put("visibleInWhoisAsTech", visibleInWhoisAsTech)
.put("visibleInDomainWhoisAsAbuse", visibleInDomainWhoisAsAbuse)
.put("allowedToSetRegistryLockPassword", allowedToSetRegistryLockPassword)
.put("gaeUserId", gaeUserId)
.build();
}
@@ -345,5 +387,27 @@ public class RegistrarContact extends ImmutableObject implements Jsonifiable {
getInstance().gaeUserId = gaeUserId;
return this;
}
public Builder setAllowedToSetRegistryLockPassword(boolean allowedToSetRegistryLockPassword) {
if (allowedToSetRegistryLockPassword) {
getInstance().registryLockPasswordSalt = null;
getInstance().registryLockPasswordHash = null;
}
getInstance().allowedToSetRegistryLockPassword = allowedToSetRegistryLockPassword;
return this;
}
public Builder setRegistryLockPassword(String registryLockPassword) {
checkArgument(
getInstance().allowedToSetRegistryLockPassword,
"Not allowed to set registry lock password for this contact");
checkArgument(
!isNullOrEmpty(registryLockPassword), "Registry lock password was null or empty");
getInstance().registryLockPasswordSalt = base64().encode(SALT_SUPPLIER.get());
getInstance().registryLockPasswordHash =
hashPassword(registryLockPassword, getInstance().registryLockPasswordSalt);
getInstance().allowedToSetRegistryLockPassword = false;
return this;
}
}
}
@@ -24,6 +24,7 @@ import static com.google.common.collect.Maps.filterValues;
import static google.registry.model.CacheUtils.memoizeWithShortExpiration;
import static google.registry.model.common.EntityGroupRoot.getCrossTldKey;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.google.common.base.Joiner;
@@ -52,7 +53,7 @@ public final class Registries {
private static Supplier<ImmutableMap<String, TldType>> createFreshCache() {
return memoizeWithShortExpiration(
() ->
ofy()
tm()
.doTransactionless(
() -> {
ImmutableMap.Builder<String, TldType> builder =
@@ -21,6 +21,7 @@ import static com.google.common.base.Predicates.not;
import static google.registry.config.RegistryConfig.getSingletonCacheRefreshDuration;
import static google.registry.model.common.EntityGroupRoot.getCrossTldKey;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableCopy;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
@@ -222,7 +223,7 @@ public class Registry extends ImmutableObject implements Buildable {
// Enter a transactionless context briefly; we don't want to enroll every TLD in a
// transaction that might be wrapping this call.
return Optional.ofNullable(
ofy()
tm()
.doTransactionless(
() -> ofy()
.load()
@@ -22,6 +22,7 @@ import static google.registry.config.RegistryConfig.getStaticPremiumListMaxCache
import static google.registry.model.common.EntityGroupRoot.getCrossTldKey;
import static google.registry.model.ofy.ObjectifyService.allocateId;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static java.util.concurrent.TimeUnit.MILLISECONDS;
import com.google.common.annotations.VisibleForTesting;
@@ -150,7 +151,7 @@ public final class PremiumList extends BaseDomainLabelList<Money, PremiumList.Pr
new CacheLoader<String, PremiumList>() {
@Override
public PremiumList load(final String name) {
return ofy().doTransactionless(() -> loadPremiumList(name));
return tm().doTransactionless(() -> loadPremiumList(name));
}
});
}
@@ -174,7 +175,7 @@ public final class PremiumList extends BaseDomainLabelList<Money, PremiumList.Pr
new CacheLoader<Key<PremiumListRevision>, PremiumListRevision>() {
@Override
public PremiumListRevision load(final Key<PremiumListRevision> revisionKey) {
return ofy().doTransactionless(() -> ofy().load().key(revisionKey).now());
return tm().doTransactionless(() -> ofy().load().key(revisionKey).now());
}
});
@@ -214,7 +215,7 @@ public final class PremiumList extends BaseDomainLabelList<Money, PremiumList.Pr
new CacheLoader<Key<PremiumListEntry>, Optional<PremiumListEntry>>() {
@Override
public Optional<PremiumListEntry> load(final Key<PremiumListEntry> entryKey) {
return ofy()
return tm()
.doTransactionless(() -> Optional.ofNullable(ofy().load().key(entryKey).now()));
}
});
@@ -27,6 +27,7 @@ import static google.registry.model.registry.label.DomainLabelMetrics.PremiumLis
import static google.registry.model.registry.label.PremiumList.cachePremiumListEntries;
import static google.registry.model.registry.label.PremiumList.cachePremiumListRevisions;
import static google.registry.model.registry.label.PremiumList.cachePremiumLists;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static org.joda.time.DateTimeZone.UTC;
import com.google.auto.value.AutoValue;
@@ -151,12 +152,12 @@ public final class PremiumListUtils {
// Save the new child entities in a series of transactions.
for (final List<PremiumListEntry> batch : partition(parentedEntries, TRANSACTION_BATCH_SIZE)) {
ofy().transactNew(() -> ofy().save().entities(batch));
tm().transactNew(() -> ofy().save().entities(batch));
}
// Save the new PremiumList and revision itself.
PremiumList updated = ofy().transactNew(() -> {
DateTime now = ofy().getTransactionTime();
PremiumList updated = tm().transactNew(() -> {
DateTime now = tm().getTransactionTime();
// Assert that the premium list hasn't been changed since we started this process.
PremiumList existing = ofy().load()
.type(PremiumList.class)
@@ -201,7 +202,7 @@ public final class PremiumListUtils {
/** Deletes the PremiumList and all of its child entities. */
public static void deletePremiumList(final PremiumList premiumList) {
ofy().transactNew(() -> ofy().delete().entity(premiumList));
tm().transactNew(() -> ofy().delete().entity(premiumList));
deleteRevisionAndEntriesOfPremiumList(premiumList);
cachePremiumLists.invalidate(premiumList.getName());
}
@@ -214,9 +215,9 @@ public final class PremiumListUtils {
partition(
ofy().load().type(PremiumListEntry.class).ancestor(premiumList.revisionKey).keys(),
TRANSACTION_BATCH_SIZE)) {
ofy().transactNew(() -> ofy().delete().keys(batch));
tm().transactNew(() -> ofy().delete().keys(batch));
}
ofy().transactNew(() -> ofy().delete().key(premiumList.getRevisionKey()));
tm().transactNew(() -> ofy().delete().key(premiumList.getRevisionKey()));
}
/**
@@ -16,6 +16,7 @@ package google.registry.model.server;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.isAtOrAfter;
import com.google.auto.value.AutoValue;
@@ -176,10 +177,10 @@ public class Lock extends ImmutableObject implements Serializable {
// access to resources like GCS that can't be transactionally rolled back. Therefore, the lock
// must be definitively acquired before it is used, even when called inside another transaction.
AcquireResult acquireResult =
ofy()
tm()
.transactNew(
() -> {
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
// Checking if an unexpired lock still exists - if so, the lock can't be acquired.
Lock lock = ofy().load().type(Lock.class).id(lockId).now();
@@ -217,7 +218,7 @@ public class Lock extends ImmutableObject implements Serializable {
/** Release the lock. */
public void release() {
// Just use the default clock because we aren't actually doing anything that will use the clock.
ofy()
tm()
.transact(
() -> {
// To release a lock, check that no one else has already obtained it and if not
@@ -231,7 +232,7 @@ public class Lock extends ImmutableObject implements Serializable {
logger.atInfo().log("Deleting lock: %s", lockId);
ofy().deleteWithoutBackup().entity(Lock.this);
lockMetrics.recordRelease(
resourceName, tld, new Duration(acquiredTime, ofy().getTransactionTime()));
resourceName, tld, new Duration(acquiredTime, tm().getTransactionTime()));
} else {
logger.atSevere().log(
"The lock we acquired was transferred to someone else before we"
@@ -15,6 +15,7 @@
package google.registry.model.server;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.cache.CacheBuilder;
@@ -54,7 +55,7 @@ public class ServerSecret extends CrossTldSingleton {
return secret;
}
// Slow path - transactionally create a new ServerSecret (once per app setup).
return ofy().transact(() -> {
return tm().transact(() -> {
// Check again for an existing secret within the transaction to avoid races.
ServerSecret secret1 = ofy().load().entity(new ServerSecret()).now();
if (secret1 == null) {
@@ -22,6 +22,7 @@ import static google.registry.model.CacheUtils.memoizeWithShortExpiration;
import static google.registry.model.common.EntityGroupRoot.getCrossTldKey;
import static google.registry.model.ofy.ObjectifyService.allocateId;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
@@ -92,7 +93,7 @@ public class SignedMarkRevocationList extends ImmutableObject {
private static final Supplier<SignedMarkRevocationList> CACHE =
memoizeWithShortExpiration(
() ->
ofy()
tm()
.transactNewReadOnly(
() -> {
Iterable<SignedMarkRevocationList> shards =
@@ -150,7 +151,7 @@ public class SignedMarkRevocationList extends ImmutableObject {
/** Save this list to Datastore in sharded form. Returns {@code this}. */
public SignedMarkRevocationList save() {
ofy()
tm()
.transact(
() -> {
ofy()
@@ -21,6 +21,7 @@ import static com.google.common.base.Verify.verify;
import static google.registry.model.CacheUtils.memoizeWithShortExpiration;
import static google.registry.model.ofy.ObjectifyService.allocateId;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import com.google.common.annotations.VisibleForTesting;
@@ -111,7 +112,7 @@ public class ClaimsListShard extends ImmutableObject {
Concurrent.transform(
shardKeys,
key ->
ofy()
tm()
.transactNewReadOnly(
() -> {
ClaimsListShard claimsListShard = ofy().load().key(key).now();
@@ -188,7 +189,7 @@ public class ClaimsListShard extends ImmutableObject {
Concurrent.transform(
CollectionUtils.partitionMap(labelsToKeys, shardSize),
(final ImmutableMap<String, String> labelsToKeysShard) ->
ofy()
tm()
.transactNew(
() -> {
ClaimsListShard shard = create(creationTime, labelsToKeysShard);
@@ -199,7 +200,7 @@ public class ClaimsListShard extends ImmutableObject {
}));
// Persist the new revision, thus causing the newly created shards to go live.
ofy()
tm()
.transactNew(
() -> {
verify(
@@ -16,6 +16,7 @@ package google.registry.model.tmch;
import static com.google.common.base.Preconditions.checkNotNull;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.googlecode.objectify.annotation.Entity;
import google.registry.model.annotations.NotBackedUp;
@@ -48,11 +49,11 @@ public final class TmchCrl extends CrossTldSingleton {
* and actually newer than the one currently in Datastore.
*/
public static void set(final String crl, final String url) {
ofy()
tm()
.transactNew(
() -> {
TmchCrl tmchCrl = new TmchCrl();
tmchCrl.updated = ofy().getTransactionTime();
tmchCrl.updated = tm().getTransactionTime();
tmchCrl.crl = checkNotNull(crl, "crl");
tmchCrl.url = checkNotNull(url, "url");
ofy().saveWithoutBackup().entity(tmchCrl);
@@ -0,0 +1,84 @@
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.transaction;
import org.joda.time.DateTime;
/**
* This interface defines the methods to execute database operations with or without a transaction.
*/
public interface TransactionManager {
/** This functional interface defines a method to execute a work and return the result. */
@FunctionalInterface
interface Work<R> {
R run();
}
/** Returns {@code true} if the caller is in a transaction.
*
* <p>Note that this function is kept for backward compatibility. We will review the use case
* later when adding the cloud sql implementation.
*/
boolean inTransaction();
/** Throws {@link IllegalStateException} if the caller is not in a transaction.
*
* <p>Note that this function is kept for backward compatibility. We will review the use case
* later when adding the cloud sql implementation.
*/
void assertInTransaction();
/** Executes the work in a transaction and returns the result. */
<T> T transact(Work<T> work);
/** Executes the work in a transaction. */
void transact(Runnable work);
/** Pauses the current transaction (if any), executes the work in a new transaction
* and returns the result.
*
* <p>Note that this function is kept for backward compatibility. We will review the use case
* later when adding the cloud sql implementation.
*/
<T> T transactNew(Work<T> work);
/** Pauses the current transaction (if any) and executes the work in a new transaction.
*
* <p>Note that this function is kept for backward compatibility. We will review the use case
* later when adding the cloud sql implementation.
*/
void transactNew(Runnable work);
/** Executes the work in a read-only transaction and returns the result.
*
* <p>Note that this function is kept for backward compatibility. We will review the use case
* later when adding the cloud sql implementation.
*/
<R> R transactNewReadOnly(Work<R> work);
/** Executes the work in a read-only transaction.
*
* <p>Note that this function is kept for backward compatibility. We will review the use case
* later when adding the cloud sql implementation.
*/
void transactNewReadOnly(Runnable work);
/** Executes the work in a transactionless context. */
<R> R doTransactionless(Work<R> work);
/** Returns the time associated with the start of this particular transaction attempt. */
DateTime getTransactionTime();
}
@@ -0,0 +1,37 @@
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.transaction;
import google.registry.model.ofy.DatastoreTransactionManager;
/** Factory class to create {@link TransactionManager} instance. */
public class TransactionManagerFactory {
private static final TransactionManager TM = createTransactionManager();
private TransactionManagerFactory() {}
private static TransactionManager createTransactionManager() {
// TODO: Conditionally returns the corresponding implementation once we have
// CloudSqlTransactionManager
return new DatastoreTransactionManager(null);
}
/** Returns {@link TransactionManager} instance. */
public static TransactionManager tm() {
return TM;
}
}
@@ -17,6 +17,7 @@ package google.registry.model.translators;
import static com.google.common.base.MoreObjects.firstNonNull;
import static google.registry.config.RegistryConfig.getCommitLogDatastoreRetention;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import com.google.common.collect.ImmutableSortedMap;
@@ -62,7 +63,7 @@ public final class CommitLogRevisionsTranslatorFactory
@Override
ImmutableSortedMap<DateTime, Key<CommitLogManifest>> transformBeforeSave(
ImmutableSortedMap<DateTime, Key<CommitLogManifest>> revisions) {
DateTime now = ofy().getTransactionTime();
DateTime now = tm().getTransactionTime();
DateTime threshold = now.minus(getCommitLogDatastoreRetention());
DateTime preThresholdTime = firstNonNull(revisions.floorKey(threshold), START_OF_TIME);
return new ImmutableSortedMap.Builder<DateTime, Key<CommitLogManifest>>(Ordering.natural())
@@ -15,7 +15,7 @@
package google.registry.model.translators;
import static com.google.common.base.MoreObjects.firstNonNull;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static org.joda.time.DateTimeZone.UTC;
import google.registry.model.CreateAutoTimestamp;
@@ -46,7 +46,7 @@ public class CreateAutoTimestampTranslatorFactory
/** Save a timestamp, setting it to the current time if it did not have a previous value. */
@Override
public Date saveValue(CreateAutoTimestamp pojoValue) {
return firstNonNull(pojoValue.getTimestamp(), ofy().getTransactionTime()).toDate();
return firstNonNull(pojoValue.getTimestamp(), tm().getTransactionTime()).toDate();
}};
}
}
@@ -14,7 +14,7 @@
package google.registry.model.translators;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static org.joda.time.DateTimeZone.UTC;
import google.registry.model.UpdateAutoTimestamp;
@@ -46,7 +46,7 @@ public class UpdateAutoTimestampTranslatorFactory
/** Save a timestamp, setting it to the current time. */
@Override
public Date saveValue(UpdateAutoTimestamp pojoValue) {
return ofy().getTransactionTime().toDate();
return tm().getTransactionTime().toDate();
}};
}
}
@@ -17,6 +17,7 @@ package google.registry.rdap;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static java.nio.charset.StandardCharsets.UTF_8;
import com.google.api.client.http.GenericUrl;
@@ -92,27 +93,23 @@ public final class UpdateRegistrarRdapBaseUrlsAction implements Runnable {
@Inject
UpdateRegistrarRdapBaseUrlsAction() {}
private String loginAndGetId(HttpRequestFactory requestFactory, String tld) {
try {
logger.atInfo().log("Logging in to MoSAPI");
HttpRequest request =
requestFactory.buildGetRequest(new GenericUrl(String.format(LOGIN_URL, tld)));
request.getHeaders().setBasicAuthentication(String.format("%s_ry", tld), password);
HttpResponse response = request.execute();
private String loginAndGetId(HttpRequestFactory requestFactory, String tld) throws IOException {
logger.atInfo().log("Logging in to MoSAPI");
HttpRequest request =
requestFactory.buildGetRequest(new GenericUrl(String.format(LOGIN_URL, tld)));
request.getHeaders().setBasicAuthentication(String.format("%s_ry", tld), password);
HttpResponse response = request.execute();
Optional<HttpCookie> idCookie =
HttpCookie.parse(response.getHeaders().getFirstHeaderStringValue("Set-Cookie")).stream()
.filter(cookie -> cookie.getName().equals(COOKIE_ID))
.findAny();
checkState(
idCookie.isPresent(),
"Didn't get the ID cookie from the login response. Code: %s, headers: %s",
response.getStatusCode(),
response.getHeaders());
return idCookie.get().getValue();
} catch (IOException e) {
throw new UncheckedIOException("Error logging in to MoSAPI server: " + e.getMessage(), e);
}
Optional<HttpCookie> idCookie =
HttpCookie.parse(response.getHeaders().getFirstHeaderStringValue("Set-Cookie")).stream()
.filter(cookie -> cookie.getName().equals(COOKIE_ID))
.findAny();
checkState(
idCookie.isPresent(),
"Didn't get the ID cookie from the login response. Code: %s, headers: %s",
response.getStatusCode(),
response.getHeaders());
return idCookie.get().getValue();
}
private void logout(HttpRequestFactory requestFactory, String id, String tld) {
@@ -128,13 +125,13 @@ public final class UpdateRegistrarRdapBaseUrlsAction implements Runnable {
}
}
private ImmutableSetMultimap<String, String> getRdapBaseUrlsPerIanaIdWithTld(String tld) {
HttpRequestFactory requestFactory = httpTransport.createRequestFactory();
String id = loginAndGetId(requestFactory, tld);
private ImmutableSetMultimap<String, String> getRdapBaseUrlsPerIanaIdWithTld(
String tld, String id, HttpRequestFactory requestFactory) {
String content;
try {
HttpRequest request =
requestFactory.buildGetRequest(new GenericUrl(String.format(LIST_URL, tld)));
request.getHeaders().setAcceptEncoding("identity");
request.getHeaders().setCookie(String.format("%s=%s", COOKIE_ID, id));
HttpResponse response = request.execute();
@@ -173,11 +170,22 @@ public final class UpdateRegistrarRdapBaseUrlsAction implements Runnable {
checkArgument(!tlds.isEmpty(), "There must exist at least one REAL TLD.");
Throwable finalThrowable = null;
for (String tld : tlds) {
HttpRequestFactory requestFactory = httpTransport.createRequestFactory();
String id;
try {
return getRdapBaseUrlsPerIanaIdWithTld(tld);
id = loginAndGetId(requestFactory, tld);
} catch (Throwable e) {
// Login failures are bad but not unexpected for certain TLDs. We shouldn't store those
// but rather should only store useful Throwables.
logger.atWarning().log("Error logging in to MoSAPI server: " + e.getMessage());
continue;
}
try {
return getRdapBaseUrlsPerIanaIdWithTld(tld, id, requestFactory);
} catch (Throwable throwable) {
logger.atWarning().log(String
.format("Error retrieving RDAP urls with TLD %s: %s", tld, throwable.getMessage()));
logger.atWarning().log(
String.format(
"Error retrieving RDAP urls with TLD %s: %s", tld, throwable.getMessage()));
finalThrowable = throwable;
}
}
@@ -190,7 +198,7 @@ public final class UpdateRegistrarRdapBaseUrlsAction implements Runnable {
ImmutableSetMultimap<String, String> ianaToBaseUrls = getRdapBaseUrlsPerIanaId();
for (Key<Registrar> registrarKey : ofy().load().type(Registrar.class).keys()) {
ofy()
tm()
.transact(
() -> {
Registrar registrar = ofy().load().key(registrarKey).now();
@@ -15,6 +15,7 @@
package google.registry.rde;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.google.common.flogger.FluentLogger;
import google.registry.model.common.Cursor;
@@ -98,7 +99,7 @@ class EscrowTaskRunner {
task.runWithLock(nextRequiredRun);
DateTime nextRun = nextRequiredRun.plus(interval);
logger.atInfo().log("Rolling cursor forward to %s.", nextRun);
ofy().transact(() -> ofy().save().entity(Cursor.create(cursorType, nextRun, registry)));
tm().transact(() -> ofy().save().entity(Cursor.create(cursorType, nextRun, registry)));
return null;
};
String lockName = String.format("EscrowTaskRunner %s", task.getClass().getSimpleName());
@@ -16,6 +16,7 @@ package google.registry.rde;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import com.google.common.collect.ImmutableSetMultimap;
@@ -106,7 +107,7 @@ public final class PendingDepositChecker {
final Registry registry,
final CursorType cursorType,
final DateTime initialValue) {
return ofy()
return tm()
.transact(
() -> {
Cursor cursor = ofy().load().key(Cursor.createKey(cursorType, registry)).now();
@@ -21,6 +21,7 @@ import static com.google.common.base.Preconditions.checkState;
import static com.google.common.base.Verify.verify;
import static google.registry.model.common.Cursor.getCursorTimeOrStartOfTime;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static java.nio.charset.StandardCharsets.UTF_8;
import com.google.appengine.tools.cloudstorage.GcsFilename;
@@ -202,7 +203,7 @@ public final class RdeStagingReducer extends Reducer<PendingDeposit, DepositFrag
logger.atInfo().log("Manual operation; not advancing cursor or enqueuing upload task");
return;
}
ofy()
tm()
.transact(
() -> {
Registry registry = Registry.get(tld);
@@ -22,6 +22,7 @@ import static google.registry.model.common.Cursor.CursorType.RDE_UPLOAD_SFTP;
import static google.registry.model.common.Cursor.getCursorTimeOrStartOfTime;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.rde.RdeMode.FULL;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static google.registry.util.DateTimeUtils.isBeforeOrAt;
import static java.util.Arrays.asList;
@@ -169,11 +170,11 @@ public final class RdeUploadAction implements Runnable, EscrowTask {
() -> upload(xmlFilename, xmlLength, watermark, name), JSchException.class);
logger.atInfo().log(
"Updating RDE cursor '%s' for TLD '%s' following successful upload.", RDE_UPLOAD_SFTP, tld);
ofy()
tm()
.transact(
() -> {
Cursor updatedSftpCursor =
Cursor.create(RDE_UPLOAD_SFTP, ofy().getTransactionTime(), Registry.get(tld));
Cursor.create(RDE_UPLOAD_SFTP, tm().getTransactionTime(), Registry.get(tld));
ofy().save().entity(updatedSftpCursor);
});
response.setContentType(PLAIN_TEXT_UTF_8);
@@ -22,6 +22,7 @@ import static javax.servlet.http.HttpServletResponse.SC_OK;
import com.google.appengine.tools.cloudstorage.GcsFilename;
import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.Iterables;
import com.google.common.flogger.FluentLogger;
import com.google.common.io.ByteStreams;
@@ -38,6 +39,7 @@ import google.registry.util.Retrier;
import java.io.IOException;
import java.io.InputStream;
import java.util.Optional;
import java.util.stream.Collectors;
import javax.inject.Inject;
/** Copy all registrar detail reports in a given bucket's subdirectory from GCS to Drive. */
@@ -95,6 +97,8 @@ public final class CopyDetailReportsAction implements Runnable {
response.setPayload(String.format("Failure, encountered %s", e.getMessage()));
return;
}
ImmutableMap.Builder<String, Throwable> copyErrorsBuilder =
new ImmutableMap.Builder<String, Throwable>();
for (String detailReportName : detailReportObjectNames) {
// The standard report format is "invoice_details_yyyy-MM_registrarId_tld.csv
// TODO(larryruili): Determine a safer way of enforcing this.
@@ -117,7 +121,7 @@ public final class CopyDetailReportsAction implements Runnable {
try (InputStream input =
gcsUtils.openInputStream(
new GcsFilename(billingBucket, invoiceDirectoryPrefix + detailReportName))) {
driveConnection.createFile(
driveConnection.createOrUpdateFile(
detailReportName,
MediaType.CSV_UTF_8,
driveFolderId,
@@ -129,15 +133,31 @@ public final class CopyDetailReportsAction implements Runnable {
},
IOException.class);
} catch (Throwable e) {
emailUtils.sendAlertEmail(
String alertMessage =
String.format(
"Warning: CopyDetailReportsAction failed.\nEncountered: %s on file: %s",
getRootCause(e).getMessage(), detailReportName));
throw e;
"Warning: CopyDetailReportsAction failed for registrar %s.\n"
+ "Encountered: %s on file: %s",
registrarId, getRootCause(e).getMessage(), detailReportName);
copyErrorsBuilder.put(registrarId, e);
logger.atSevere().withCause(e).log(alertMessage);
}
}
response.setStatus(SC_OK);
response.setContentType(MediaType.PLAIN_TEXT_UTF_8);
response.setPayload("Copied detail reports.");
StringBuilder payload = new StringBuilder().append("Copied detail reports.\n");
ImmutableMap<String, Throwable> copyErrors = copyErrorsBuilder.build();
if (!copyErrors.isEmpty()) {
payload.append("The following errors were encountered:\n");
payload.append(
copyErrors.entrySet().stream()
.map(
entrySet ->
String.format(
"Registrar: %s\nError: %s\n",
entrySet.getKey(), entrySet.getValue().getMessage()))
.collect(Collectors.joining()));
}
response.setPayload(payload.toString());
emailUtils.sendAlertEmail(payload.toString());
}
}
@@ -59,22 +59,19 @@ public class Spec11EmailUtils {
private final SendEmailService emailService;
private final InternetAddress outgoingEmailAddress;
private final InternetAddress alertRecipientAddress;
private final InternetAddress spec11ReplyToAddress;
private final ImmutableList<String> spec11WebResources;
private final String registryName;
@Inject
Spec11EmailUtils(
SendEmailService emailService,
@Config("gSuiteOutgoingEmailAddress") InternetAddress outgoingEmailAddress,
@Config("alertRecipientEmailAddress") InternetAddress alertRecipientAddress,
@Config("spec11ReplyToEmailAddress") InternetAddress spec11ReplyToAddress,
@Config("spec11OutgoingEmailAddress") InternetAddress spec11OutgoingEmailAddress,
@Config("spec11WebResources") ImmutableList<String> spec11WebResources,
@Config("registryName") String registryName) {
this.emailService = emailService;
this.outgoingEmailAddress = outgoingEmailAddress;
this.outgoingEmailAddress = spec11OutgoingEmailAddress;
this.alertRecipientAddress = alertRecipientAddress;
this.spec11ReplyToAddress = spec11ReplyToAddress;
this.spec11WebResources = spec11WebResources;
this.registryName = registryName;
}
@@ -149,7 +146,7 @@ public class Spec11EmailUtils {
.setContentType(MediaType.HTML_UTF_8)
.setFrom(outgoingEmailAddress)
.addRecipient(getEmailAddressForRegistrar(registrarThreatMatches.clientId()))
.setBcc(spec11ReplyToAddress)
.setBcc(outgoingEmailAddress)
.build());
}
@@ -172,7 +169,7 @@ public class Spec11EmailUtils {
ImmutableMap.of(
"date", date.toString(),
"registry", registryName,
"replyToEmail", spec11ReplyToAddress.getAddress(),
"replyToEmail", outgoingEmailAddress.getAddress(),
"threats", threatMatchMap,
"resources", spec11WebResources);
renderer.setData(data);
@@ -16,12 +16,17 @@ package google.registry.request;
import static com.google.monitoring.metrics.EventMetric.DEFAULT_FITTER;
import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Iterables;
import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import com.google.monitoring.metrics.EventMetric;
import com.google.monitoring.metrics.LabelDescriptor;
import com.google.monitoring.metrics.MetricRegistryImpl;
import google.registry.request.auth.AuthLevel;
import java.util.List;
import java.util.stream.Collectors;
import org.joda.time.Duration;
class RequestMetrics {
@@ -50,7 +55,7 @@ class RequestMetrics {
Duration duration, String path, Action.Method method, AuthLevel authLevel, boolean success) {
requestDurationMetric.record(
duration.getMillis(),
path,
truncatePath(path),
String.valueOf(method),
String.valueOf(authLevel),
String.valueOf(success));
@@ -58,4 +63,15 @@ class RequestMetrics {
"Action called for path=%s, method=%s, authLevel=%s, success=%s. Took: %.3fs",
path, method, authLevel, success, duration.getMillis() / 1000d);
}
private static String truncatePath(String path) {
// We want to bucket RDAP requests by type to use less metric space,
// e.g. "/rdap/domains" rather than "/rdap/domains/foo.tld"
if (path.startsWith("/rdap")) {
List<String> splitPath = Splitter.on("/").omitEmptyStrings().splitToList(path);
return Streams.stream(Iterables.limit(splitPath, 2))
.collect(Collectors.joining("/", "/", "/"));
}
return path;
}
}
@@ -54,6 +54,10 @@ public class DriveConnection {
/**
* Creates a file with the given parent.
*
* <p>If a file with the same path already exists, a duplicate is created. If overwriting the
* existing file is the desired behavior, use {@link #createOrUpdateFile(String, MediaType,
* String, byte[])} instead.
*
* @returns the file id.
*/
public String createFile(String title, MediaType mimeType, String parentFolderId, byte[] bytes)
@@ -16,7 +16,7 @@ package google.registry.tmch;
import static com.google.appengine.api.taskqueue.QueueFactory.getQueue;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.google.appengine.api.taskqueue.TaskOptions;
import com.google.appengine.api.taskqueue.TaskOptions.Method;
@@ -47,7 +47,7 @@ public final class LordnTaskUtils {
* Enqueues a task in the LORDN queue representing a line of CSV for LORDN export.
*/
public static void enqueueDomainBaseTask(DomainBase domain) {
ofy().assertInTransaction();
tm().assertInTransaction();
// This method needs to use ofy transactionTime as the DomainBase's creationTime because
// CreationTime isn't yet populated when this method is called during the resource flow.
String tld = domain.getTld();
@@ -55,12 +55,12 @@ public final class LordnTaskUtils {
getQueue(QUEUE_SUNRISE).add(TaskOptions.Builder
.withTag(tld)
.method(Method.PULL)
.payload(getCsvLineForSunriseDomain(domain, ofy().getTransactionTime())));
.payload(getCsvLineForSunriseDomain(domain, tm().getTransactionTime())));
} else {
getQueue(QUEUE_CLAIMS).add(TaskOptions.Builder
.withTag(tld)
.method(Method.PULL)
.payload(getCsvLineForClaimsDomain(domain, ofy().getTransactionTime())));
.payload(getCsvLineForClaimsDomain(domain, tm().getTransactionTime())));
}
}
@@ -19,6 +19,7 @@ import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.flows.poll.PollFlowUtils.getPollMessagesQuery;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.poll.PollMessageExternalKeyConverter.makePollMessageExternalId;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.beust.jcommander.Parameter;
import com.beust.jcommander.Parameters;
@@ -85,7 +86,7 @@ final class AckPollMessagesCommand implements CommandWithRemoteApi {
public void run() {
QueryKeys<PollMessage> query = getPollMessagesQuery(clientId, clock.nowUtc()).keys();
for (List<Key<PollMessage>> keys : Iterables.partition(query, BATCH_SIZE)) {
ofy()
tm()
.transact(
() -> {
// Load poll messages and filter to just those of interest.
@@ -52,7 +52,7 @@ final class CreateAnchorTenantCommand extends MutatingEppToolCommand {
@Parameter(
names = {"--contact"},
description = "Contact ID for the request. This will be used for registrant, admin contact,"
description = "Contact ID for the request. This will be used for registrant, admin contact, "
+ "and tech contact.",
required = true)
private String contact;
@@ -234,6 +234,13 @@ abstract class CreateOrUpdateRegistrarCommand extends MutatingCommand {
arity = 1)
private Boolean contactsRequireSyncing;
@Nullable
@Parameter(
names = "--registry_lock_allowed",
description = "Whether this registrar is allowed to use registry lock",
arity = 1)
private Boolean registryLockAllowed;
@Nullable
@Parameter(
names = "--drive_folder_id",
@@ -393,6 +400,7 @@ abstract class CreateOrUpdateRegistrarCommand extends MutatingCommand {
}
Optional.ofNullable(blockPremiumNames).ifPresent(builder::setBlockPremiumNames);
Optional.ofNullable(contactsRequireSyncing).ifPresent(builder::setContactsRequireSyncing);
Optional.ofNullable(registryLockAllowed).ifPresent(builder::setRegistryLockAllowed);
Optional.ofNullable(phonePasscode).ifPresent(builder::setPhonePasscode);
Optional.ofNullable(icannReferralEmail).ifPresent(builder::setIcannReferralEmail);
Optional.ofNullable(whoisServer).ifPresent(builder::setWhoisServer);
@@ -19,7 +19,6 @@ import static google.registry.util.CollectionUtils.findDuplicates;
import static google.registry.util.DomainNameUtils.canonicalizeDomainName;
import com.beust.jcommander.Parameter;
import com.google.common.base.CharMatcher;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
@@ -255,7 +254,7 @@ abstract class CreateOrUpdateTldCommand extends MutatingCommand {
tld,
canonicalizeDomainName(tld));
checkArgument(
!CharMatcher.javaDigit().matches(tld.charAt(0)),
!Character.isDigit(tld.charAt(0)),
"TLDs cannot begin with a number");
Registry oldRegistry = getOldRegistry(tld);
// TODO(b/26901539): Add a flag to set the pricing engine once we have more than one option.
@@ -19,6 +19,7 @@ import static com.google.common.collect.Iterables.partition;
import static com.google.common.collect.Streams.stream;
import static google.registry.model.domain.token.AllocationToken.TokenType.SINGLE_USE;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.beust.jcommander.Parameter;
import com.beust.jcommander.Parameters;
@@ -64,7 +65,7 @@ final class DeleteAllocationTokensCommand extends UpdateOrDeleteAllocationTokens
protected String execute() {
long numDeleted =
stream(partition(tokensToDelete, BATCH_SIZE))
.mapToLong(batch -> ofy().transact(() -> deleteBatch(batch)))
.mapToLong(batch -> tm().transact(() -> deleteBatch(batch)))
.sum();
return String.format("Deleted %d tokens in total.", numDeleted);
}
@@ -16,6 +16,7 @@ package google.registry.tools;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import com.beust.jcommander.Parameter;
import com.beust.jcommander.Parameters;
@@ -76,7 +77,7 @@ final class DeleteTldCommand extends ConfirmingCommand implements CommandWithRem
@Override
protected String execute() {
ofy().transactNew(() -> ofy().delete().entity(registry).now());
tm().transactNew(() -> ofy().delete().entity(registry).now());
registry.invalidateInCache();
return String.format("Deleted TLD '%s'.\n", tld);
}
@@ -21,6 +21,7 @@ import static com.google.common.collect.Sets.difference;
import static google.registry.model.domain.token.AllocationToken.TokenType.SINGLE_USE;
import static google.registry.model.domain.token.AllocationToken.TokenType.UNLIMITED_USE;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.nullToEmpty;
import static google.registry.util.StringGenerator.DEFAULT_PASSWORD_LENGTH;
import static java.nio.charset.StandardCharsets.UTF_8;
@@ -34,6 +35,8 @@ import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedMap;
import com.google.common.collect.Iterables;
import com.google.common.collect.Streams;
import com.google.common.io.Files;
import com.googlecode.objectify.Key;
import google.registry.model.domain.token.AllocationToken;
@@ -50,6 +53,7 @@ import java.util.Collection;
import java.util.Deque;
import java.util.List;
import java.util.Optional;
import java.util.stream.Stream;
import javax.inject.Inject;
import javax.inject.Named;
import org.joda.time.DateTime;
@@ -63,6 +67,13 @@ import org.joda.time.DateTime;
@NonFinalForTesting
class GenerateAllocationTokensCommand implements CommandWithRemoteApi {
@Parameter(
names = {"--tokens"},
description =
"Comma-separated list of exact tokens to generate, otherwise use --prefix or "
+ "--domain_names_file to generate tokens if the exact strings do not matter")
private List<String> tokenStrings;
@Parameter(
names = {"-p", "--prefix"},
description = "Allocation token prefix; defaults to blank")
@@ -131,24 +142,10 @@ class GenerateAllocationTokensCommand implements CommandWithRemoteApi {
@Override
public void run() throws IOException {
checkArgument(
(numTokens > 0) ^ (domainNamesFile != null),
"Must specify either --number or --domain_names_file, but not both");
checkArgument(
!(UNLIMITED_USE.equals(tokenType) && CollectionUtils.isNullOrEmpty(tokenStatusTransitions)),
"For UNLIMITED_USE tokens, must specify --token_status_transitions");
// A list consisting solely of the empty string means user error when formatting parameters
checkArgument(
!ImmutableList.of("").equals(allowedClientIds),
"Either omit --allowed_client_ids if all registrars are allowed, or include a"
+ " comma-separated list");
checkArgument(
!ImmutableList.of("").equals(allowedTlds),
"Either omit --allowed_tlds if all TLDs are allowed, or include a comma-separated list");
verifyInput();
if (tokenStrings != null) {
numTokens = tokenStrings.size();
}
Deque<String> domainNames;
if (domainNamesFile == null) {
domainNames = null;
@@ -165,8 +162,7 @@ class GenerateAllocationTokensCommand implements CommandWithRemoteApi {
int tokensSaved = 0;
do {
ImmutableSet<AllocationToken> tokens =
generateTokens(BATCH_SIZE).stream()
.limit(numTokens - tokensSaved)
getNextTokenBatch(tokensSaved)
.map(
t -> {
AllocationToken.Builder token =
@@ -188,10 +184,66 @@ class GenerateAllocationTokensCommand implements CommandWithRemoteApi {
} while (tokensSaved < numTokens);
}
private void verifyInput() {
int inputMethods = 0;
if (numTokens > 0) {
inputMethods++;
}
if (domainNamesFile != null) {
inputMethods++;
}
if (tokenStrings != null && !tokenStrings.isEmpty()) {
inputMethods++;
}
checkArgument(
inputMethods == 1,
"Must specify exactly one of '--number', '--domain_names_file', and '--tokens'");
checkArgument(
tokenLength > 0,
"Token length should not be 0. To generate exact tokens, use the --tokens parameter.");
checkArgument(
!(UNLIMITED_USE.equals(tokenType) && CollectionUtils.isNullOrEmpty(tokenStatusTransitions)),
"For UNLIMITED_USE tokens, must specify --token_status_transitions");
// A list consisting solely of the empty string means user error when formatting parameters
checkArgument(
!ImmutableList.of("").equals(allowedClientIds),
"Either omit --allowed_client_ids if all registrars are allowed, or include a"
+ " comma-separated list");
checkArgument(
!ImmutableList.of("").equals(allowedTlds),
"Either omit --allowed_tlds if all TLDs are allowed, or include a comma-separated list");
if (tokenStrings != null) {
verifyTokenStringsDoNotExist();
}
}
private void verifyTokenStringsDoNotExist() {
ImmutableSet<String> existingTokenStrings =
getExistingTokenStrings(ImmutableSet.copyOf(tokenStrings));
checkArgument(
existingTokenStrings.isEmpty(),
String.format(
"Cannot create specified tokens; the following tokens already exist: %s",
existingTokenStrings));
}
private Stream<String> getNextTokenBatch(int tokensSaved) {
if (tokenStrings != null) {
return Streams.stream(Iterables.limit(Iterables.skip(tokenStrings, tokensSaved), BATCH_SIZE));
} else {
return generateTokens(BATCH_SIZE).stream().limit(numTokens - tokensSaved);
}
}
@VisibleForTesting
int saveTokens(final ImmutableSet<AllocationToken> tokens) {
Collection<AllocationToken> savedTokens =
dryRun ? tokens : ofy().transact(() -> ofy().save().entities(tokens).now().values());
dryRun ? tokens : tm().transact(() -> ofy().save().entities(tokens).now().values());
savedTokens.forEach(
t -> System.out.println(SKIP_NULLS.join(t.getDomainName().orElse(null), t.getToken())));
return savedTokens.size();
@@ -209,14 +261,17 @@ class GenerateAllocationTokensCommand implements CommandWithRemoteApi {
stringGenerator.createStrings(tokenLength, count).stream()
.map(s -> prefix + s)
.collect(toImmutableSet());
ImmutableSet<String> existingTokenStrings = getExistingTokenStrings(candidates);
return ImmutableSet.copyOf(difference(candidates, existingTokenStrings));
}
private ImmutableSet<String> getExistingTokenStrings(ImmutableSet<String> candidates) {
ImmutableSet<Key<AllocationToken>> existingTokenKeys =
candidates.stream()
.map(input -> Key.create(AllocationToken.class, input))
.collect(toImmutableSet());
ImmutableSet<String> existingTokenStrings =
ofy().load().keys(existingTokenKeys).values().stream()
.map(AllocationToken::getToken)
.collect(toImmutableSet());
return ImmutableSet.copyOf(difference(candidates, existingTokenStrings));
return ofy().load().keys(existingTokenKeys).values().stream()
.map(AllocationToken::getToken)
.collect(toImmutableSet());
}
}
@@ -0,0 +1,79 @@
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.tools;
import com.beust.jcommander.Parameter;
import com.beust.jcommander.Parameters;
import google.registry.model.domain.DomainBase;
import java.util.EnumSet;
import java.util.HashMap;
import java.util.Map;
import org.hibernate.boot.MetadataSources;
import org.hibernate.boot.registry.StandardServiceRegistryBuilder;
import org.hibernate.tool.hbm2ddl.SchemaExport;
import org.hibernate.tool.schema.TargetType;
/**
* Generates a schema for JPA annotated classes using hibernate.
*
* <p>Note that this isn't complete yet, as all of the persistent classes have not yet been
* converted. After converting a class, a call to "addAnnotatedClass()" for the new class must be
* added to the code below.
*/
@Parameters(separators = " =", commandDescription = "Generate postgresql schema.")
public class GenerateSqlSchemaCommand implements Command {
public static final int POSTGRESQL_PORT = 5432;
@Parameter(
names = {"-o", "--out-file"},
description = "")
String outFile;
@Parameter(
names = {"-a", "--db-host"},
description = "Database host name.")
String databaseHost;
@Parameter(
names = {"-p", "--db-port"},
description = "Database port number. This defaults to the postgresql default port.")
int databasePort = POSTGRESQL_PORT;
@Override
public void run() {
// TODO(mmuller): Optionally (and perhaps by default) start a postgresql instance container
// rather than relying on the user to have one to connect to.
Map<String, String> settings = new HashMap<>();
settings.put("hibernate.dialect", "org.hibernate.dialect.PostgreSQL9Dialect");
settings.put(
"hibernate.connection.url",
"jdbc:postgresql://" + databaseHost + ":" + databasePort + "/postgres?useSSL=false");
settings.put("hibernate.connection.username", "postgres");
settings.put("hibernate.connection.password", "domain-registry");
settings.put("hibernate.hbm2ddl.auto", "none");
settings.put("show_sql", "true");
MetadataSources metadata =
new MetadataSources(new StandardServiceRegistryBuilder().applySettings(settings).build());
metadata.addAnnotatedClass(DomainBase.class);
SchemaExport schemaExport = new SchemaExport();
schemaExport.setHaltOnError(true);
schemaExport.setFormat(true);
schemaExport.setDelimiter(";");
schemaExport.setOutputFile(outFile);
schemaExport.createOnly(EnumSet.of(TargetType.SCRIPT), metadata.buildMetadata());
}
}
@@ -21,6 +21,7 @@ import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.base.Strings.emptyToNull;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DatastoreServiceUtils.getNameOrId;
import static google.registry.util.DiffUtils.prettyPrintEntityDeepDiff;
import static java.util.stream.Collectors.joining;
@@ -140,7 +141,7 @@ public abstract class MutatingCommand extends ConfirmingCommand implements Comma
@Override
protected String execute() throws Exception {
for (final List<EntityChange> batch : getCollatedEntityChangeBatches()) {
ofy().transact(() -> batch.forEach(this::executeChange));
tm().transact(() -> batch.forEach(this::executeChange));
}
return String.format("Updated %d entities.\n", changedEntitiesMap.size());
}

Some files were not shown because too many files have changed in this diff Show More