Commit Graph
67 Commits
Author SHA1 Message Date
Ben McIlwain 202ed9db6a Allow third_party/java_src/gtld/ to use :mockito2_for_third_party
In [] a change would have been made to your project that is incompatible with
your open source integration. To make sure the open source variant of your
project remains working, we have eagerly updated your open source copy to use
Mockito 2. This CL integrates that change into [].

Please read []and
understand the consequences of this change.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=238445356
2019-03-20 14:25:28 -04:00
mcilwain 8e427681f0 De-premiumize wing.dev for internal registration
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=236661149
2019-03-20 14:25:28 -04:00
jianglai 7fe16689d8 Change redacted output for domain whois query
The redacted text for the email field displays a longer prompt to
contact the registrar, per the request filed at b/123573370.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=232716133
2019-02-06 18:03:11 -05:00
gbrodmanandBen McIlwain 3cf26ff9b6 Fix various Error Prone errors that were found by the FOSS build
Most common:
- Unnecessary parentheses and operator precedence clarify (self-explanatory)
- Reference equality--there were a few instances of using == or != improperly
- Qualification of Builder (and similar) imports so that it's clear which type of Builder we're referring to
- Marking some immutable classes with @Immutable since EP desires that all enums be deeply immutable
- String.split() having "surprising behavior"

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230971531
2019-01-28 16:05:09 -05:00
mcilwain e2528875b2 Merge DomainResource into DomainBase
This eliminates the use of Objectify polymorphism for EPP resources entirely
(yay!), which makes the Registry 3.0 database migration easier.

It is unfortunate that the naming parallelism of EppResources is lost between
ContactResource, HostResource, and DomainResource, but the actual type as far as
Datastore was concerned was DomainBase all along, and it would be a much more
substantial data migration to allow us to continue using the class name
DomainResource now that we're no longer using Objectify polymorphism. This
simply isn't worth it.

This also removes the polymorphic Datastore indexes (which will no longer
function as of this change). The non-polymorphic replacement indexes were added
in []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230930546
2019-01-28 15:57:10 -05:00
mcilwainandjianglai d2ee63cf69 Consolidate Dagger modules for utils classes
There was no reason to have several different modules all providing a single
thing. This approach, which creates a single UtilsModule for everything in the
util package, is cleaner. This also removes provisioning of Random and
StringGenerator objects in RegistryConfig.ConfigModule, which don't belong
there because they aren't configuration options.

This also removes insecure random entirely; it was only used in a
single place to generate 24 bytes a couple times per day. We can live with the
lower speed if it means we don't have to worry about multiple types of Random,
or possibly using an insecure random accidentally in a place that security
actually does matter.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229751915
2019-01-17 19:20:52 -05:00
weiminyuandBen McIlwain a80a44cd06 Define TestRule that manages cache use in tests
All current tests that use caches with custom data expiry values
now restore the default config when teardown. We need to prevent
new unsafe uses from being introduced.

Restoration code have also been added to a few other tests that modifies
static fields.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228888041
2019-01-11 11:50:33 -05:00
mcilwainandjianglai 015c854a92 Make loadByForeignKeyCached() return an Optional type
Next up (and a much larger commit) will be giving loadByForeignKey() the same
treatment.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225182377
2018-12-12 13:22:34 -05:00
shicongandjianglai 5bc70cbc99 Update lastEppUpdateTime after certain grace period is passed
The lastEppUpdateTime should be updated asynchronously in the situations below:

 - Implicit transfer success after 5 day pending transfer period
 - Implicit end of any grace period

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224831791
2018-12-12 13:22:34 -05:00
mcilwain 43ed2cd7b3 Clean up annotation imports in BUILD files
This affects JSR305, JSR330, and Guava annotations.

The exact command run to generate this CL was:

build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/jsr330_inject,//third_party/java/jsr305_annotations,[]'

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202322747
2018-06-27 15:28:53 -04:00
mcilwain 4c7bc3b18c Improve internal build system speed
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202016862
2018-06-27 15:28:52 -04:00
mcilwain 481790bc91 Always display the registrar abuse contact fields in WHOIS
ICANN's Consistent Labeling & Display policy requires that these fields be
present for all domains. We're currently tracking down the last of the data on
our registrars so that we can display these, but if we don't have the data we
should still at least be displaying the field names to make it more clear that
we do support the field, we just don't have the data for this domain yet.

The two fields in question are:

Registrar Abuse Contact Email:
Registrar Abuse Contact Phone:

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201730290
2018-06-27 15:28:52 -04:00
mcilwain 8b263baefa Delete MultiplyingCloudDnsWriter
Now that the large zone re-signing test is complete, we no longer need it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199507075
2018-06-27 15:28:06 -04:00
mcilwain 5d80f124ca Remove unnecessary "throws" declarations
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201058582
2018-06-18 18:17:56 -04:00
mcilwain ccdf8c4b26 Use -ROID for contact roids in tests instead of -TLD
-TLD is only used for domain roids.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200109501
2018-06-18 18:00:34 -04:00
jianglaiandBen McIlwain 643b30d31f Redact WHOIS output instead of removing the fields altogether
See https://www.icann.org/resources/pages/gtld-registration-data-specs-en/#appendixA for details on how certain fields are redacted.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199295355
2018-06-06 15:14:38 -04:00
jianglai 4be4fb0082 Migrate to Flogger (yellow)
This is a 'yellow' Flogger migration CL. Yellow CLs should be mostly safe
but include changes that are notable for one reason or another. Manual
intervention may be required to address small issues.

The comments in this CL indicate cases where suggested code changes
should be double checked, or even modified. There may even be cases where
files outside this CL are affected by changes to things such as logger
visibility. However if a change does not have an associated comment then
it should be safe.

For more information, see []
Base CL: 197826149

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198097990
2018-05-30 12:18:54 -04:00
jianglai fc60890136 Migrate to internal FormattingLogger in preparation of migration to Flogger
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=197744904
2018-05-30 12:18:54 -04:00
jianglai 86dd6bd59e Add ability to show full WHOIS output in nomulus command
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=197731992
2018-05-30 12:18:54 -04:00
jianglai 7388958df7 Do not escape WHOIS output
Both WhoisAction and WhoisHttpAction set the HTTP response content type to "text/plain". There is no need to defensively escape the content. In fact, by escaping the content, it creates more problems down the line.

When used in a website, the response should be written into a DOM node by setting the textContent of the node, which automatically escapes the content.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=196743398
2018-05-17 21:52:35 -04:00
larryruiliandjianglai 6cdbde107f Redirect Registrar.referralUrl UI actions to url field
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=196597051
2018-05-17 21:52:35 -04:00
larryruiliandjianglai 4657b8ab51 Replace 'referralUrl' with 'url' in WHOIS responses
The 'referralUrl' Datastore field is filled with mostly junk data, whereas
'url' contains real registrar web addresses. This makes the long needed fix to
display the proper url in WHOIS.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194398003
2018-05-05 23:32:40 -04:00
mcilwain 07d38340f3 Cache domains, contacts, and hosts in WHOIS queries
This should prevent having issues with hot key paths on entities that
experience a heavy WHOIS volume (e.g. contacts that registrars reuse on
many domains).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191506124
2018-04-10 16:24:03 -04:00
jianglaiandBen McIlwain 839e4aa682 Remove some fields from WHOIS output
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191445626
2018-04-10 16:17:42 -04:00
jianglai c40eda3235 Rename WhoisServer and WhoisHttpServer to actions
These servlets are converted to actions during daggerization. Calling them servers are misleading.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190942237
2018-04-02 16:52:14 -04:00
cushonandjianglai 606b470cd0 Merge JUnitBackport's expectThrows into assertThrows
More information: https://github.com/junit-team/junit5/issues/531

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=187034408
2018-03-06 18:56:15 -05:00
mcilwain c416b3892d Migrate final try/catch test assertions to use assert/expectThrows
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=182091814
2018-01-19 14:43:12 -05:00
mcilwain 7dc224627f Automatically refactor more exception testing to use new JUnit rules
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178911894
2017-12-27 10:42:36 -05:00
mcilwainandjianglai 03c782f38e Replace ExceptionRule with ExpectedException
This is in preparation for running the automatic refactoring script that
will replace all ExpectedExceptions with use of JUnit 4.13's assertThrows/
expectThrows.

Note that I have recorded the callsites of assertions about EppExceptions
being marshallable and will edit those specific assertions back in after
running the automatic refactoring script (which do not understand these).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178812403
2017-12-13 12:43:45 -05:00
jianglai 1c1f95992a Move backported JUnit file to third_party (part 2)
Last commit did not pick up all the changes because MOE incorrectly attributed some changes to the wrong commit. This commit should reconcile these. Also picked up some changes to how hamcrest library is depended upon in BUILD file, which should have been included in previous commits.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177637931
2017-12-02 11:37:46 -05:00
guybenandjianglai b8970bfe1b Rename all testdata loading files to conform to a single naming scheme
The scheme is:
- loadBytes: returns a ByteSource of the data
- loadFile: returns a string using UTF8 encoding, optionally applying
  substitutions

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177606406
2017-12-01 22:14:06 -05:00
mcilwainandjianglai e2db3f914e Clean up some code quality issues
This removes some qualifiers that aren't necessary (e.g. public/abstract on interfaces, private on enum constructors, final on private methods, static on nested interfaces/enums), uses Java 8 lambdas and features where that's an improvement

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177182945
2017-12-01 22:14:06 -05:00
guybenandjianglai 95647528b8 Clean up test "load resources" methods and classes
There is a big mix of different "load Resources" from different libraries
depending on where you were and what type of resource you want. Now there is a
clear hirarchy:

ResourceUtils:
  for use in actual (non-test) code, reads a file from a context directory

TestDataHelper (uses ResourceUtils):
  for use in tests, reads a file from a context directory + "/testdata". Also
  caches the resource so calling it multiple times with the same file will
  not read the file multiple times.

Library specific helpers (e.g. ToolsTestData) (uses TestDataHelper):
  for use in that library's tests only, reads from a specific testdata directory.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177027533
2017-12-01 22:14:05 -05:00
mcilwainandjianglai 30bfcf9c55 Remove more unused Truth8.assertThat() static imports in tests
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=173562809
2017-11-07 17:22:56 -05:00
mcilwainandjianglai c0f8da0c6e Switch from Guava Optionals to Java 8 Optionals
This was a surprisingly involved change. Some of the difficulties included
java.util.Optional purposely not being Serializable (so I had to move a
few Optionals in mapreduce classes to @Nullable) and having to add the Truth
Java8 extension library for assertion support.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171863777
2017-10-24 16:53:47 -04:00
guybenandjianglai 840d53c819 Allow EventSample.record to accept numSamples=0
There's really no reason not to.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171037754
2017-10-24 16:50:30 -04:00
bbilboandjianglai 263aea3b2a Add logging to WhoisReader
Logging will make it easier to troubleshoot WhoisServer exceptions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=167596453
2017-09-12 15:51:50 -04:00
mcilwain d536cef20f Make Registrar load methods return Optionals instead of Nullables
This makes the code more understandable from callsites, and also forces
users of this function to deal with the situation where the registrar
with a given client ID might not be present (it was previously silently
NPEing from some of the callsites).

This also adds a test helper method loadRegistrar(clientId) that retains
the old functionality for terseness in tests. It also fixes some instances
of using the load method with the wrong cachedness -- some uses in high-
traffic situations (WHOIS) that should have caching, but also low-traffic
reporting that don't benefit from caching so might as well always be
current.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162990468
2017-08-01 16:58:59 -04:00
mountfordandBen McIlwain e8eabe01cf Update WHOIS output format to reflect CL&D and PDT requirements
PDT testing revealed a couple ways in which our WHOIS output was non-compliant. First, the Consistent Labeling & Display policy dictates that the contact IDs must be ROIDs. See rule 11 in https://www.icann.org/resources/pages/rdds-labeling-policy-2017-02-01-en. Second, PDT tests expect that a WHOIS response will treat missing values either by omitting the line entirely, or by including the line with a blank value, but not both. So this is legal:

Phone Number: 123-4567
Phone Number Ext:
Fax Number: 123-4568
Fax Number Ext:

and this is legal:

Phone Number: 123-4567
Fax Number: 123-4568

but this is not:

Phone Number: 123-4567
Phone Number Ext:
Fax Number: 123-4568

In the last example, one extension line is present with a blank value, while the other extension line is omitted. We cannot do both. Therefore, we should update our code to omit lines with no value. Since we can't guarantee that we will always emit all lines that the parse might expect to see, it is safe to use the policy of omitting lines with no value.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158184150
2017-06-14 10:29:55 -04:00
jianglaiandBen McIlwain 3f5735ed4b Add Carriage Return in WHOIS disclaimer text
WHOIS disclaimer is read from the config yaml file directly and the line breaks
within the text does not contain carriage return, which ICANN requires. This
CL fixes the non-compliance.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156359526
2017-05-23 17:22:49 -04:00
mcilwain ef1487cb57 Make name and address fields required on Registrar
The absence of these fields causes RDE failures, so they are in effect
required on any functioning registry system. We are currently
experiencing problems in sandbox caused by null values on these fields.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155474895
2017-05-17 11:44:46 -04:00
jianglaiandBen McIlwain 8892656722 Make WHOIS domain query return conform to ICANN CL&D policy
ICANN's policy can be found here:
https://www.icann.org/resources/pages/rdds-labeling-policy-2017-02-01-en

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155375811
2017-05-17 11:38:28 -04:00
mcilwain 08009e755f Don't include "Command" suffix in WHOIS metrics command name label
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152414152
2017-04-10 13:46:21 -04:00
larryruiliandBen McIlwain a5c9ef56d3 Fix appengine dependency for WhoisServerTest
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152050380
2017-04-05 10:19:50 -04:00
larryruiliandBen McIlwain 7359cc13b8 Implement retry for transient errors in WHOIS server
We now attempt to retry Whois queries in the event of a short-lived error. Currently, we consider 'DatastoreTimeoutException' and 'DatastoreFailureException' as transient.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152044934
2017-04-05 10:18:23 -04:00
cgoldfederandBen McIlwain 9174855a47 Remove the ofy().load() inside of HostResource.cloneProjectedAtTime
In fact, completely eviscerate cloneProjectedAtTime (to be removed in
a followup CL) in favor of doing the projection of transfers and the
loading of values from the superordinate domain at call sites. This
is one of the issues that blocked the memcache audit work, since the
load inside of cloneProjectedAtTime could not be controlled by the
caller.

Note: fixed a minor bug where a subordinate host created after its superordinate domain was last transferred should have lastTransferTime==null but was previously reporting the domain's lastTransferTime.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149769125
2017-03-13 11:22:55 -04:00
mcilwain 815dae2749 Refactor WhoisReader.readCommand() method to take a now param
This obviates the need for the use of @AutoFactory.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149570411
2017-03-13 10:45:40 -04:00
mcilwain 3fcb564251 Record metrics for WHOIS commands
Note that this does not write out metrics for invocations of the
nomulus tool.

This requires a slight refactoring of the existing WhoisResponse
interface so as to also support returning the number of results found
by the WHOIS query.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149461208
2017-03-13 10:30:53 -04:00
mcilwain bd7db61606 Use Dagger to @Inject WhoisReader instances
This is a precursor to adding metrics to WHOIS queries (as I'd like
to be able to @Inject the metrics builders).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149418018
2017-03-07 14:02:46 -05:00
mcilwain d2bc569b4b Tighten up restrictions on domain/host naming on EPP resources
All domain/host names should be stored in their canonical forms (puny-
coded and lower-cased). This validation is already in the flows, but
this adds protection against bad data from other sources, e.g. admin
consoles or RDE imports.

This also removes an old work-around that temporarily suspended this
validation for superusers, because we used to have non-canonicalized
data in the system. The non-canonicalized data has since all been
cleaned up, so this work-around is no longer necessary.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146799558
2017-02-07 13:29:03 -05:00