* Add mosapi client to intract with ICANN's monitoring system
This change introduces a comprehensive client to interact with ICANN's Monitoring System API (MoSAPI). This provides direct, automated access to critical registry health and compliance data, moving Nomulus towards a more proactive monitoring posture.
A core, stateless MosApiClient that manages communication and authentication with the MoSAPI service using TLS client certificates.
* Resolve review feedback & upgrade to OkHttp3 client
This commit addresses and resolves all outstanding review comments, primarily encompassing a shift to OkHttp3, security configuration cleanup, and general code improvements.
* **Review:** Addressed and resolved all pending review comments.
* **Feature:** Switched the underlying HTTP client implementation to [OkHttp3](https://square.github.io/okhttp/).
* **Configuration:** Consolidated TLS Certificates-related configuration into the dedicated configuration area.
* **Cleanup:** Removed unused components (`HttpUtils` and `HttpModule`) and performed general code cleanup.
* **Quality:** Improved exception handling logic for better robustness.
* Refactor and fix Mosapi exception handling
Addresses code review feedback and resulting test failures.
- Flattens package structure by moving MosApiException and its test.
- Corrects exception handling to ensure MosApiAuthorizationException
propagates correctly, before the general exception handler.
- Adds a default case to the MosApiException factory for robustness.
- Uses lowercase for placeholder TLDs in default-config.yaml.
* Refactor and improve Mosapi client implementation
Simplifying URL validation with Guava
Preconditions and refining exception handling to use `Throwables`.
* Refactor precondition checks using project specific utility
This will be necessary if we wish to do larger BTAPPA transfers (or
other types of transfers, I suppose). The nomulus command-line tool is
not fast enough to quickly transfer thousands of domains within a
reasonable timeframe.
After each deployment in sandbox or production, move the artifacts from
the corresponding release to a well-known location so that they can be
mapped to Kokoro in presubmit tests. The Kokoro-mapping does not need
public access to the GCS bucket.
The artifacts include the postgresql schema jar, the nomulus release
jar, and the uber jar of the nomulus schema integration test classes.
Every jar name consists of a fixed prefix and the environment. Each jar
of a new deployment overrides the previous copy.
These are old/pointless now that we've migrated to GKE. Note that this
doesn't update anything in the docs/ folder, as that's a much larger
project that should be done on its own.
Add a flag indicating that a Sandbox TLD should use the production
servers.
No additional TLD name pattern checks. Cloud DNS has an allowlist for
names that may use production servers.
Also updated default descriptive name generation: dropping the trailing
'.', and replacing remaining dots with '_'.
This contains the same codepoints from the
core/src/main/java/google/registry/idn/Latin-IDN.xml file, just in the old .txt
IDN format that Nomulus actually ingests.
We don't need to support the mix of GAE and GKE any more so we can get
rid of the GaeService bits and unify everything under one constant
service. This also allows us to reduce the number of services down to
four (FE, BE, PUBAPI, console) which is nice.
We've previously been using Scrypt since PR #2191 which, while being a
memory-hard slow function, isn't the optimal solution according to the
OWASP recommendations. While we could get away with increasing the
parallelization parameter to 3, it's better to just switch to the
most-recommended solution if we're switching things up anyway.
For the transition, we do something similar to PR #2191 where if the
previous-algorithm's hash is successful, we re-hash with Argo2id and
store that version. By doing this, we should not need any intervention
for registrars who log in at any point during the transition period.
Much of this PR, especially the parts where we re-hash the passwords in
Argon2 instead of Scrypt upon login, is based on the code that was
eventually removed in #2310.
Reformat the current schema file for RFC 8748 final version. This was
adapted from v0.12 is not fully consistent with the final schema
This helps highlight the differences we missed in PR 2855 when we check
in the official schema.
The servlets, at this point now that we're off GAE, are only used for
the test server (and, indirectly, in one BSA test). Instead of having
them all remain separate, we can unify them in one test servlet that
lives in the test/ folder.
This removes one avenue of potential confusion w/r/t how request routing
actually works and where we would want to add new routing.
We need to stop using maven repo on GCS to store artifacts for the
schema compatibility tests. After public access is removed from GCS
buckets, Kokoro won't be able to access it: normal access will be
denied, and the repo is too large to map (copy) to Kokoro VM as a
resource.
This PR uploads the relevant jars to each release's folder. See
go/dr-gcs-public-access-prevention for details.
With GKE, we don't need the individual servlets because the services
aren't partitioned out the same way they were in GAE.
We keep FrontendServlet and BackendServlet around for now as they serve
as the backbone for the local RegistryTestServer (for testing things
like the console).
did some cursory tests on alpha and things seem to be unaffected -- I
was able to curl RDAP (pubapi) and create domains
It's been over half a year now since we last used any of these and we definitely
no longer have any intentions of ever using App Engine again.
BUG= http://b/457471639
Still part of b/454947209, removing references to WHOIS where we can. We
keep the registrar type and the column names (at least for now) because
changing those is much more complicated.
We no longer need this now that no contacts can be applied to any domains at all.
A follow-up PR in subsequent weeks will delete the column from the DB schema.
BUG= http://b/448619572
Previously, we would have separate database calls for mapping from
foreign key to repo ID and then from repo ID to object. This PR modifies
those calls to load the resource directly (the old system was an
artifact of the Datastore key-value storage system).
In this PR, we merge the load-resource-by-foreign-key calls into a
single database load, as well as adding a separate cache object for
(foreign key) -> (resource). Now we cache, and have separate cleaner
code paths, for fk -> resource, fk -> repo ID, and repo ID -> resource.
Also removes the unused RdeFragmenter class
* Support Fee Extension standard in rfc 8748
Adding support to the final version of RFC 8748.
Compared with draft-0.12, the only meaningful change is in the namespace.
The rest is either schema-tightening that reflects actual usage, or
optional server-side features that we do not support.
We reuse draft-0.12 tests, only changing namespace uris in the input and
output files for the new version.
* Addressing reviews
We haven't been serving this for a while, let's finally get rid of them.
We keep some Soy rules around in the presubmits file because we use some
Soy files as XML templates for EPP actions.
This doesn't make any underlying implementation details, and is mainly useful to
reduce the number of diffs in PR #2852 (which does change implementation
details) thus making that easier to review.
This allows us to specify a getter delegation to bypass Hibernate's limitations
on field types for the purposes of, e.g., using a sorted set in toString()
output rather than the base Hibernate unsorted HashSet type.
BUG=http://b/448631639
This affects FKs pointing to both Contact and ContactHistory. This is in
preparation to us deleting all rows in those two tables, and then subsequently
removing all application logic having to do with contacts entirely.
This is steps one and two of b/454947209
We already haven't been serving WHOIS for a while, so there's no point
in keeping the old code around. This can simplify some code paths in the
future (like, certain foreign-key-loads that are only used in WHOIS
queries).
Basically, what happened is that the cache's expireAfterWrite was being
called some number of milliseconds (say, 50-100) after the transaction
was started. That method used the transaction time instead of the
current time, so as a result the entries were sticking around 50-100ms
longer in the cache than they should have been.
This fix contains two parts, each of which I believe would be sufficient
on their own to fix the issue:
1. Use the currentTime passed in in Expiry::expireAfterCreate
2. Use the transaction time in the cache's Ticker. This keeps everything
on the same schedule.
During the release process, we are seeing the message "Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)" which seemingly can be caused by OOMs
I analyzed SQL statements run during the following flows and EXPLAIN
ANALYZEd each of them to figure out if there are any additional hash
indexes we could add that could be particularly helpful. Note: it's not
worth adding a hash index on the host_repo_id field in DomainHost
because so many rows (domains) use the same host.
- domain create
- domain delete
- domain info
- domain renew
- domain update
- host create
- host delete
- host update
I skipped the ones that use the read-only replica, as well as contact
flows (we're getting rid of them), and domain transfer/restore-related
flows as those are extremely infrequent.
Updates (AKA merges) run an extra SELECT statement to figure out if the
resource exists so that it can merge the entity into the existing object
in Hibernate's schema. When we're inserting new rows (such as new poll
messages or resource creates), we know that we don't need to do that
merge. Doing this should save us some SELECT statements (this has borne
out to be the truth in alpha)
This should help in instances of popular domains dropping, since we
won't need to do an additional two database loads every time (assuming
the deletion time is in the future).
