* More Gradle 9 preparations
Fix additional compatibility warnings after upgrading to Gradle 8.14.3
from 8.13.
* More Gradle 9 compatibility fix
More fixes after upgrading Gradle from 8.13 to 8.14.3.
Upgraded the gradle-license-report plugin, and handled config leaking
issues.
Researched using gemini web and manually applied the fixes. Gemini-cli
could not find the right solution.
This ended up being wayyyy more complicated than expected due to
issues with Hibernate, various dependencies having conflicts with the
proto dependency version, and other breaking changes.
Notes:
- Hibernate 7 switches up the user type / converter system and for us,
this means we must be / want to be more explicit with how we convert
and store things. For example, we need to add Postgres types to @Column
definitions.
- Hibernate 7.3 has an issue with generic MappedSuperclasses -- we have
issues with BaseDomainLabelList. I'll investigate that, but for now
let's stick with 7.2.x
- H7 is more strict with annotations and prevents us from storing mapped
superclasses embedded within other objects. This kinda makes sense but
makes the History objects a bit more difficult. We had to add "concrete"
embeddable DomainBase and HostBase objects that we can store/retrieve
from the DB.
- We convert some of the calls to "Query" to "TypedQuery" -- in
Hibernate 8 / JPA 4.0 these will be super-deprecated and we'll need to
shift everything over, so this is necessary.
- You aren't supposed to put callback listeners on embedded entities
(because it can be not obvious what's happening). We don't like that,
so we add our own annotations that are processed recursively for
embedded entities, so we get things like the update / create
timestamps.
- Hibernate doesn't allow for multiple converters to be auto-applied to
the same "type" and it counts all VKey converters as one type.
Unfortunately, this means we have to explicitly mark each one.
- A bunch of other dependency changes were required to keep from having
the proto 3/4 conflict
Previously we omit the fee class in responses if the class is standard.
After a rereading of Section 3.7 of RFC 8748, we determined that the fee
class element should be included in all cases.
Note: the `collison` class now becomes `standard-collision`.
Refactor Gradle scripts to replace usages incompatible with Gradle 9.
PR prepared mostly with gemini-cli, with one issue (project.exec)
researched with gemini web and manually applied.
The actual upgrade to Gradle 9 will be in another PR.
Verified: none of the issues reported in build/reports/problems/problems-report.html
is related to Gradle 9.
PR 2917 added two `get(tld)` methods to ClaimsListDao and
SignedMarkRevocationList so that RST test TLDs can have separate claims
and smdr lists.
RST tests are completed and this functionality is no longer needed. we
are replaceing all invocations of the above to `get()`.
The only type of thing that can be transferred now is a domain, so
there's no point in having this abstract class / redirection.
This does not include deletion of the contact-response-related XML
classes; that can come next.
This doesn't remove everything -- there are still other contact-related
objects that we'll need to remove (e.g.
ContactPendingActionNotificationResponse) and simplifications we'll need to make
(e.g. only domains can be transferred now, so all transfer data can move
there instead of being generic)
But this removes the bulk of the remaining contact-related code. We'll
keep around the XML request objects, since it's still nice to route them
to the appropriate (exception-throwing but logging) flow class.
For all flows that use Fee extensions, normalize the fee tags in all
non-prod environments.
For flows that do not use fee extensions but with fee tags in the
header, e.g., HostInfo flows, normalization is not performed.
This avoids changing any functionality, including the bits of
DomainCommand (representations of XML files) that reference contacts.
Currently, we "allow" parsing of contacts in DomainCommands and fail
later as part of the domain flow, even though in practice the parsing itself will fail now that no
contacts exist in the database.
Because we wish to keep the "contacts aren't allowed in flows" tests
active (e.g.
DomainUpdateFlowTest::testFailure_minimumDataset_whenAddingNewContacts)
we have to keep the usages of contacts in DomainCommand active for now.
We have a restriction in our system that registrar display names be unique (as
the display name is how registrars are queried through RDAP). And, the
uniqueness constraint is enforced on the canonicalized version of the display
name (with spaces and non alphanumeric characters removed). However, in the
check enforcing this uniqueness, we were incorrectly checking against the
existing saved entity of the same registrar, meaning that you couldn't update
the display name of a single registrar to a new value that canonicalized the
same (you would instead have to rename it to something else first that doesn't
canonicalize the same, and then afterwards to the new desired value).
That didn't make sense, so now we exclude the existing registrar entity from
consideration when checking if there are conflicts.
The current SQL credential store was designed to support automatic
password rotation without any disruption to the applications. For that
goal, the credentials are stored with one level of indirection, and the
secret name of the actual credential data may change automatically.
The automatic password rotation feature has been dropped. In the
meantime, the need arises that we use sidecar SQL proxy to get around
the Enterprise Plus edition's post-maintenance reconnection failures
by the socket factory library. This is hampered by the indirection in
storage.
This PR removes the indirection. This change is transparent to the rest
of the code base. We will manually populate the secret manager with the
new secrets in all environments after submissiion of this PR.
* Normalize Fee extension XML tags in EPP response
Nomulus currently supports multiple versions of the Fee extensions. Our
current tooling requires that each version must use a unique namespace
tag, e.g., fee11, fee12, etc.
Some client registrars are sensitive to the tag literal used by the
version of the extension they use. For example, a few registrars
currently using v0.6 have requested that the `fee` literal be used
on the versions they currently use. With registrars upgrading at their
own schedule, this kind of requests are impossible to satisfy.
This PR instroduces a namespace normalizer class for EPP responses. The
key optimization is that each EPP response never mixes multiple versions
of a service extension. Therefore we can define a canonical tag for each
extension, and change the tag of the extension in use in a response to
that. This normalizer only handles Fee extensions right now, but the
idea can be extended to others if use cases come up.
This normalizer will be applied to all flows in a future PR.
* Addressing reviews
* A faster implementation with regex.
b/478848482
This primarily affects the EPP greeting. We already were erroring out when any
contact flows attempted to be run; this should just prevent registrars from even
trying them at all.
This PR is designed to be minimally invasive, and does not remove any of the
contact flows or Jakarta XML/XJC objects/files themselves. That can be done
later as a follow-up.
Also note that the contact namespace urn:ietf:params:xml:ns:contact-1.0 is still
present for now in RDE exports, but I'll remove that subsequently as well.
This is a redo of PR #2932, which had been reverted, but now controlled via
FeatureFlag so that it won't be enabled until we schedule it to do so (and only
after sufficient time has passed after notifying registrars in advance).
BUG= http://b/475506288
We've moved on from contacts entirely now so the only thing we really
need to do is make sure that people don't include contacts in domain
creates or updates. This also makes auth code checking easier too,
because now the only auth code that you're allowed to provide is the
domain auth code (not a contact auth code)
* Add MosApiMetrics exporter with status code mapping
Introduces the metrics exporter for the MoSAPI system.
- Implements `MosApiMetrics` to export TLD and service states to Cloud Monitoring.
- Maps ICANN status codes to numeric gauges: 1 (UP), 0 (DOWN), and 2 (DISABLED/INCONCLUSIVE).
- Sets `MAX_TIMESERIES_PER_REQUEST` to 195 to respect Cloud Monitoring API limits
* Automate metric descriptor creation on startup in Cloud Monitoring
* Refactor MoSAPI metrics for resilience and standards
* Refactor and nits
- Kept projectName as part constant instead of inside method signature
- Added Summary logs for metrics execution
- Metric Executor defaults to Single Threaded
* junit test refactoring
* Fix Metric kind to GAUGE for all metrics
* Refactor MosApiMetrics to remove async ExecutorService
* Add LockHandler for Metric Descriptor creation
* Update LockHandler lease time to one hour and refactoring
Now that we've passed the RST testing (or at least the EPP portion of
it) we are no longer bound by the restriction to only use the fee
extension version 1.0 on sandbox.
For now, in order to avoid changing prod behavior, this does not enable
advertisement of the fee extension version 1.0 in production. We can
change this at any point in the future.
The primary annoyance with this is that it means we need (or at least,
should) split all tests that use the fee extension into two separate
tests -- one that simulates non-prod environments, and one that
simulates prod environments. This leads to duplication of many tests but
that's fine since this is theoretically temporary.
* Configure cloud scheduler to trigger MoSAPI SLA status to cloud monitoring in production
- We have kept this job to trigger for every 3 minutes so that we get near to real time update for our task.
- This will not trigger metrics for now as we have not written Metrics triggering logic yet
- Logs are added
* Change Trigger scheduling from 3 minutes to 5 minutes
This primarily affects the EPP greeting. We already were erroring out when any
contact flows attempted to be run; this should just prevent registrars from even
trying them at all.
This PR is designed to be minimally invasive, and does not remove any of the
contact flows or Jakarta XML/XJC objects/files themselves. That can be done
later as a follow-up.
Also note that the contact namespace urn:ietf:params:xml:ns:contact-1.0 is still
present for now in RDE exports, but I'll remove that subsequently as well.
BUG= http://b/475506288
This means that attempting to add a status that is already present will now
fail, and attempting to remove a status that is not present will also now fail.
This also refactors the existing checks into a single verify method, rather than
having to call three separate methods from every callsite.
BUG= http://b/474645068
Many of the actual fee extension changes are based off Weimin's PR
https://github.com/google/nomulus/pull/2912, though this makes some
additional changes based on the XML schema and description from RFC 8748.
This adds tests for the DomainCheckFlow which is the most complex and
thorough user of the fee extension, but we'll want to add further tests
to the other domain flows to make sure they're handled correctly.
The RST testing expects us to fail if they try to remove an IP from a
host that already doesn't that have that IP, or to add one that already
exists (ditto on both for a domain's nameservers). I don't really see an
issue with our previous no-op implementation, but we need to do this to
pass the tests.
We need to have this enabled in sandbox, but we wish to wait to enable
it for production to make sure that the implementation is correct and
that clients can use it.
Soon we'll want to do something similar (but the opposite) with the old
fee extensions, where we **only** serve them in production (or maybe
unit test as well). That will allow us to pass the RST tests that depend
on only having the fee extension 1.0.
This is / will be required in https://datatracker.ietf.org/doc/rfc8748/.
I split this out from the rest of the fee-extension testing so that it
can be easily visible.
This commit introduces a new backend endpoint at `/_dr/task/triggerMosApiServiceState` that initiates the process of fetching the latest service states for all TLDs from the MoSAPI endpoint and exporting them as metrics to Cloud Monitoring.
The key changes include:
- A new `TriggerServiceStateAction` class that handles the GET request to the new endpoint.
- Logic within `MosApiStateService` to concurrently fetch states for all configured TLDs.
- A new `MosApiMetrics` class (currently a placeholder) responsible for sending the collected states to the monitoring service.
- Unit tests for the new action and the updated service logic.
This endpoint will be called periodically to ensure that the MosApi service health metrics are kept up-to-date.
this is coming from the schema https://datatracker.ietf.org/doc/rfc8748/
section 6.1. The class, that we use for "premium" notes, moved from the
command to the object itself.
I don't know where in the spec these are explicitly disallowed, but it
seems like good practice and we'll fail the RST tests if we don't
disallow them.
* Add GetServiceState action for MoSAPI service monitoring
Implements the `/api/mosapi/getServiceState` endpoint to retrieve service health summaries for TLDs from the MoSAPI system.
- Introduces `GetServiceStateAction` to fetch TLD service status.
- Implements `MosApiStateService` to transform raw MoSAPI responses into a curated `ServiceStateSummary`.
- Uses concurrent processing with a fixed thread pool to fetch states for all configured TLDs efficiently while respecting MoSAPI rate limits.
junit test added
* Refactor MoSAPI models to records and address review nits
- Convert model classes to Java records for conciseness and immutability.
- Update unit tests to use Java text blocks for improved JSON readability.
- Simplify service and action layers by removing redundant logic and logging.
- Fix configuration nits regarding primitive types and comment formatting.
* Consolidate MoSAPI models and enhance null-safety
- Moves model records into a single MosApiModels.java file.
- Switches to ImmutableList/ImmutableMap with non-null defaults in constructors.
- Removes redundant pass-through methods in MosApiStateService.
- Updates tests to use Java Text Blocks and non-null collection assertions.
* Improve MoSAPI client error handling and clean up data models
Refactors the MoSAPI monitoring client to be more robust against
infrastructure failures
* Refactor: use nullToEmptyImmutableCopy() for MoSAPI models
Standardize null-handling in model classes by using the Nomulus
`nullToEmptyImmutableCopy()` utility. This ensures consistent API
responses with empty lists instead of omitted fields.
* Add RST support in Sandbox
Added RST test label files as resources.
Added a RstTmchUtils class that loads appropriate labels according to
TLD pattern.
Temporarily changed label fetching in production to include the TLD
string, so that the new class may know which set of labels to use.
* Addressing comments
* Addressing comments
We do most of these on host create already so we should also do them on
host checks. The only added change is the character validation (our
existing hostnames all match these).
2306 signifies something that is syntactically valid but semantically
invalid (like if someone tried to register a .com domain). These errors
are for domain syntax that could never be valid, thus we should throw a
syntax exception instead of a policy exception.
Now that we have transitioned to the minimum dataset, we no longer
support any actions on contacts (and by the time this is merged /
deployed, all contacts will be deleted). We should just throw an
appropriate exception on all contact-related flows. We don't delete the
flows themselves, so that we can have an appropriate error message.
We also keep all the flows and XML templates around individually for now because we may be
required to continue to differentiate the requests in ICANN activity
reporting (e.g. srs-cont-create vs srs-cont-delete)
