This avoids changing any functionality, including the bits of
DomainCommand (representations of XML files) that reference contacts.
Currently, we "allow" parsing of contacts in DomainCommands and fail
later as part of the domain flow, even though in practice the parsing itself will fail now that no
contacts exist in the database.
Because we wish to keep the "contacts aren't allowed in flows" tests
active (e.g.
DomainUpdateFlowTest::testFailure_minimumDataset_whenAddingNewContacts)
we have to keep the usages of contacts in DomainCommand active for now.
We have a restriction in our system that registrar display names be unique (as
the display name is how registrars are queried through RDAP). And, the
uniqueness constraint is enforced on the canonicalized version of the display
name (with spaces and non alphanumeric characters removed). However, in the
check enforcing this uniqueness, we were incorrectly checking against the
existing saved entity of the same registrar, meaning that you couldn't update
the display name of a single registrar to a new value that canonicalized the
same (you would instead have to rename it to something else first that doesn't
canonicalize the same, and then afterwards to the new desired value).
That didn't make sense, so now we exclude the existing registrar entity from
consideration when checking if there are conflicts.
The current SQL credential store was designed to support automatic
password rotation without any disruption to the applications. For that
goal, the credentials are stored with one level of indirection, and the
secret name of the actual credential data may change automatically.
The automatic password rotation feature has been dropped. In the
meantime, the need arises that we use sidecar SQL proxy to get around
the Enterprise Plus edition's post-maintenance reconnection failures
by the socket factory library. This is hampered by the indirection in
storage.
This PR removes the indirection. This change is transparent to the rest
of the code base. We will manually populate the secret manager with the
new secrets in all environments after submissiion of this PR.
* Normalize Fee extension XML tags in EPP response
Nomulus currently supports multiple versions of the Fee extensions. Our
current tooling requires that each version must use a unique namespace
tag, e.g., fee11, fee12, etc.
Some client registrars are sensitive to the tag literal used by the
version of the extension they use. For example, a few registrars
currently using v0.6 have requested that the `fee` literal be used
on the versions they currently use. With registrars upgrading at their
own schedule, this kind of requests are impossible to satisfy.
This PR instroduces a namespace normalizer class for EPP responses. The
key optimization is that each EPP response never mixes multiple versions
of a service extension. Therefore we can define a canonical tag for each
extension, and change the tag of the extension in use in a response to
that. This normalizer only handles Fee extensions right now, but the
idea can be extended to others if use cases come up.
This normalizer will be applied to all flows in a future PR.
* Addressing reviews
* A faster implementation with regex.
b/478848482
This primarily affects the EPP greeting. We already were erroring out when any
contact flows attempted to be run; this should just prevent registrars from even
trying them at all.
This PR is designed to be minimally invasive, and does not remove any of the
contact flows or Jakarta XML/XJC objects/files themselves. That can be done
later as a follow-up.
Also note that the contact namespace urn:ietf:params:xml:ns:contact-1.0 is still
present for now in RDE exports, but I'll remove that subsequently as well.
This is a redo of PR #2932, which had been reverted, but now controlled via
FeatureFlag so that it won't be enabled until we schedule it to do so (and only
after sufficient time has passed after notifying registrars in advance).
BUG= http://b/475506288
We've moved on from contacts entirely now so the only thing we really
need to do is make sure that people don't include contacts in domain
creates or updates. This also makes auth code checking easier too,
because now the only auth code that you're allowed to provide is the
domain auth code (not a contact auth code)
* Add MosApiMetrics exporter with status code mapping
Introduces the metrics exporter for the MoSAPI system.
- Implements `MosApiMetrics` to export TLD and service states to Cloud Monitoring.
- Maps ICANN status codes to numeric gauges: 1 (UP), 0 (DOWN), and 2 (DISABLED/INCONCLUSIVE).
- Sets `MAX_TIMESERIES_PER_REQUEST` to 195 to respect Cloud Monitoring API limits
* Automate metric descriptor creation on startup in Cloud Monitoring
* Refactor MoSAPI metrics for resilience and standards
* Refactor and nits
- Kept projectName as part constant instead of inside method signature
- Added Summary logs for metrics execution
- Metric Executor defaults to Single Threaded
* junit test refactoring
* Fix Metric kind to GAUGE for all metrics
* Refactor MosApiMetrics to remove async ExecutorService
* Add LockHandler for Metric Descriptor creation
* Update LockHandler lease time to one hour and refactoring
Now that we've passed the RST testing (or at least the EPP portion of
it) we are no longer bound by the restriction to only use the fee
extension version 1.0 on sandbox.
For now, in order to avoid changing prod behavior, this does not enable
advertisement of the fee extension version 1.0 in production. We can
change this at any point in the future.
The primary annoyance with this is that it means we need (or at least,
should) split all tests that use the fee extension into two separate
tests -- one that simulates non-prod environments, and one that
simulates prod environments. This leads to duplication of many tests but
that's fine since this is theoretically temporary.
* Configure cloud scheduler to trigger MoSAPI SLA status to cloud monitoring in production
- We have kept this job to trigger for every 3 minutes so that we get near to real time update for our task.
- This will not trigger metrics for now as we have not written Metrics triggering logic yet
- Logs are added
* Change Trigger scheduling from 3 minutes to 5 minutes
This primarily affects the EPP greeting. We already were erroring out when any
contact flows attempted to be run; this should just prevent registrars from even
trying them at all.
This PR is designed to be minimally invasive, and does not remove any of the
contact flows or Jakarta XML/XJC objects/files themselves. That can be done
later as a follow-up.
Also note that the contact namespace urn:ietf:params:xml:ns:contact-1.0 is still
present for now in RDE exports, but I'll remove that subsequently as well.
BUG= http://b/475506288
This means that attempting to add a status that is already present will now
fail, and attempting to remove a status that is not present will also now fail.
This also refactors the existing checks into a single verify method, rather than
having to call three separate methods from every callsite.
BUG= http://b/474645068
* add cloud profiler to dockerfile and start script
* add apt-get update
* change in cb machine type for nomulus
* fix typo
* add max worker limit to gradle tests
* Switch to root before doing apt-get
* correct dockerfile
* jetty/Dockerfile
* profiler service conditional to kubernetes container name
Many of the actual fee extension changes are based off Weimin's PR
https://github.com/google/nomulus/pull/2912, though this makes some
additional changes based on the XML schema and description from RFC 8748.
This adds tests for the DomainCheckFlow which is the most complex and
thorough user of the fee extension, but we'll want to add further tests
to the other domain flows to make sure they're handled correctly.
It just makes it possible to delete allocation tokens, otherwise we need
to do a linear search over the entire Domain and DomainHistory tables if
we ever want to delete something.
The RST testing expects us to fail if they try to remove an IP from a
host that already doesn't that have that IP, or to add one that already
exists (ditto on both for a domain's nameservers). I don't really see an
issue with our previous no-op implementation, but we need to do this to
pass the tests.
We need to have this enabled in sandbox, but we wish to wait to enable
it for production to make sure that the implementation is correct and
that clients can use it.
Soon we'll want to do something similar (but the opposite) with the old
fee extensions, where we **only** serve them in production (or maybe
unit test as well). That will allow us to pass the RST tests that depend
on only having the fee extension 1.0.
This is / will be required in https://datatracker.ietf.org/doc/rfc8748/.
I split this out from the rest of the fee-extension testing so that it
can be easily visible.
This commit introduces a new backend endpoint at `/_dr/task/triggerMosApiServiceState` that initiates the process of fetching the latest service states for all TLDs from the MoSAPI endpoint and exporting them as metrics to Cloud Monitoring.
The key changes include:
- A new `TriggerServiceStateAction` class that handles the GET request to the new endpoint.
- Logic within `MosApiStateService` to concurrently fetch states for all configured TLDs.
- A new `MosApiMetrics` class (currently a placeholder) responsible for sending the collected states to the monitoring service.
- Unit tests for the new action and the updated service logic.
This endpoint will be called periodically to ensure that the MosApi service health metrics are kept up-to-date.
this is coming from the schema https://datatracker.ietf.org/doc/rfc8748/
section 6.1. The class, that we use for "premium" notes, moved from the
command to the object itself.
I don't know where in the spec these are explicitly disallowed, but it
seems like good practice and we'll fail the RST tests if we don't
disallow them.
* Add GetServiceState action for MoSAPI service monitoring
Implements the `/api/mosapi/getServiceState` endpoint to retrieve service health summaries for TLDs from the MoSAPI system.
- Introduces `GetServiceStateAction` to fetch TLD service status.
- Implements `MosApiStateService` to transform raw MoSAPI responses into a curated `ServiceStateSummary`.
- Uses concurrent processing with a fixed thread pool to fetch states for all configured TLDs efficiently while respecting MoSAPI rate limits.
junit test added
* Refactor MoSAPI models to records and address review nits
- Convert model classes to Java records for conciseness and immutability.
- Update unit tests to use Java text blocks for improved JSON readability.
- Simplify service and action layers by removing redundant logic and logging.
- Fix configuration nits regarding primitive types and comment formatting.
* Consolidate MoSAPI models and enhance null-safety
- Moves model records into a single MosApiModels.java file.
- Switches to ImmutableList/ImmutableMap with non-null defaults in constructors.
- Removes redundant pass-through methods in MosApiStateService.
- Updates tests to use Java Text Blocks and non-null collection assertions.
* Improve MoSAPI client error handling and clean up data models
Refactors the MoSAPI monitoring client to be more robust against
infrastructure failures
* Refactor: use nullToEmptyImmutableCopy() for MoSAPI models
Standardize null-handling in model classes by using the Nomulus
`nullToEmptyImmutableCopy()` utility. This ensures consistent API
responses with empty lists instead of omitted fields.
* Add RST support in Sandbox
Added RST test label files as resources.
Added a RstTmchUtils class that loads appropriate labels according to
TLD pattern.
Temporarily changed label fetching in production to include the TLD
string, so that the new class may know which set of labels to use.
* Addressing comments
* Addressing comments
We do most of these on host create already so we should also do them on
host checks. The only added change is the character validation (our
existing hostnames all match these).
2306 signifies something that is syntactically valid but semantically
invalid (like if someone tried to register a .com domain). These errors
are for domain syntax that could never be valid, thus we should throw a
syntax exception instead of a policy exception.
Now that we have transitioned to the minimum dataset, we no longer
support any actions on contacts (and by the time this is merged /
deployed, all contacts will be deleted). We should just throw an
appropriate exception on all contact-related flows. We don't delete the
flows themselves, so that we can have an appropriate error message.
We also keep all the flows and XML templates around individually for now because we may be
required to continue to differentiate the requests in ICANN activity
reporting (e.g. srs-cont-create vs srs-cont-delete)
This is a follow-up to PR #2908, which relaxed this restriction on bare TLDs
only, but now we also allow it systemwide on domains and hostnames as well. The
rules against hyphens in these positions are still enforced on all parts of the
domain name except the last one. Correct handling of multi-part TLDs in this
regard is out of scope in this PR; a multi-part TLD that looked something like
".zz--foobar.foobar" would still fail validation. (But of course you cannot a
priori know just from looking at a 3-part string whether it might be a hostname
on a normal TLD, or a domain name on a 2-part TLD.)
This also has some annoying interactions with a trailing dot (indicating the
root), which need to be preserved, but otherwise don't affect how TLD validation
is handled.
BUG= http://b/471013082
The canonicalizeHostname() helper method is only suitable for use with domain
names or host names. It does not work on bare TLDs, because a bare TLD can
have hyphens in the third and fourth position without necessarily being an IDN.
Note that the configure TLD command already correctly allows TLDs with such
names to be created.
Note that we are still enforcing that the TLDs to be added exist, so they have
to pass all TLD naming requirements that are enforced on creating TLDs, and we
are still lowercasing the TLD names passed as arguments here (though we're no
longer punycoding them, although arguably that's not super useful on
command-line params anyway).
BUG= http://b/471013082
We used to publish test artifacts to a Maven repo on GCS, for use by
schema tests. For this to work with Kokoro, the GCS bucket must be
accessible to all users.
To comply with the no-public-user requirement, we store the necessary
jars at at well-known bucket and map them into Kokoro. This strategy
cannot be used on the Maven repo because only a small number of files
with fixed names may be mapped. With the Maven repo, there are too many
files to map.
This is a decently simple wrapper around the previously-created
BulkDomainTransferAction that batches a provided list of domains up and
sends them along to be transferred.
