mirrors/nomulus
1
0
Fork 0
mirror of https://github.com/google/nomulus synced 2026-09-30 03:36:25 +00:00
Code Issues Releases Wiki Activity

Update some Nomulus documentation (#2970)

Browse Source 
This doesn't update everything -- it leaves out some of the more
complicated changes (architecture, code-structure, configuration,
install, and proxy-setup). Those will require more complete rewrites, so
I'm punting them to a future PR.
This commit is contained in:
gbrodman
2026-02-26 19:05:22 +00:00
committed by GitHub
parent 8a3ab00e58
commit c52983fb61
16 changed files with 629 additions and 872 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/operational-procedures/brda-deposits.md
+5 -5
View File
@@ -28,8 +28,8 @@ The BRDA copy task reads the previous file and creates two files:
```
If you see an `xml.ghostryde` file but not the others, an error has occurred
during the process. If you see the files in the
{PROJECT-ID}-icann-brda bucket as well, the process has completed successfully.
during the process. If you see the files in the {PROJECT-ID}-icann-brda bucket
as well, the process has completed successfully.
Once the files have been created, they must be stored on an sFTP server from
which ICANN can pull the files. The Nomulus project does not provide this last
@@ -40,11 +40,11 @@ The cursor can be checked using the `nomulus pending_escrow` command.
## Generating BRDA deposits manually
* Get a list of "REAL" (as opposed to TEST) TLDs. Doublecheck that the command
output doesn't contain any TLDs for tests.
* Get a list of "REAL" (as opposed to TEST) TLDs. Double-check that the
command output doesn't contain any TLDs for tests.
```shell
$ registry-tool -e production list_tlds --fields=tldStr,tldType | grep REAL | awk '{print $1}' > realtlds.txt`
$ nomulus -e production list_tlds --fields=tldStr,tldType | grep REAL | awk '{print $1}' > realtlds.txt`
```
* Generate .ryde and .sig files of TLDs specified for given date(s) in the
docs/operational-procedures/modifying-tlds.md
+26
View File
@@ -0,0 +1,26 @@
# Creating or Modifying TLDs
Nomulus stores YAML representations of TLDs, in an effort to make sure that any
(potentially significant) modifications to TLDs go through source control and
code review. We recommend storing these TLD YAML representations in a separate
private repository so that changes can be verified by multiple people before
being merged
([here is an example TLD](https://github.com/google/nomulus/blob/master/core/src/test/resources/google/registry/tools/tld.yaml))
Creating and updating a TLD use the same process -- the only difference is
whether you're creating a TLD YAML file from scratch or modifying an existing
one.
Similar to [premium lists](premium-list-management.md) and
[reserved lists](reserved-list-management.md), we recommend modifying TLDs as a
part of an automated build process after the desired changes have been merged
into the TLD YAML files. The automated process should run:
```shell
nomulus -e {ENVIRONMENT} configure_tld --build_environment --input=path/to/my/file/tld.yaml
```
The `build_environment` flag signals that this is being run as part of an
automated build process and should ideally not be used manually. There is an
additional `--break_glass` argument that can be used in emergencies to modify
TLDs outside a normal build process.
docs/operational-procedures/premium-list-management.md
+24 -13
View File
@@ -69,6 +69,18 @@ Perform this command? (y/N): y
Successfully saved premium list exampletld
```
### Note:
We recommend only updating premium lists manually in the case of emergencies.
Instead, we run the `update_premium_list` command (as well as `configure_tld`
and `update_reserved_list` commands) as part of the build process after a pull
request has been merged into the private source code repository that contains
the files. The `--build_environment` flag is used to signal that the command is
being run in one of those automated environments, and thus allowed to modify
production. Without that flag, commands against production will fail.
This is similar to the process for [updating TLDs](modifying-tlds.md).
If this premium list is already applied to a TLD, then changes will take up to
60 minutes to take effect (depending on how you've configured the relevant
caching interval; 60 minutes is the default).
@@ -80,16 +92,15 @@ premium list must first be applied to a TLD before it will take effect. You will
only need to do this when first creating a premium list; once it has been
applied, it stays applied, and updates to the list are effective automatically.
Note that each TLD can have no more than one premium list applied to it. To
apply a premium list to a TLD, run the `update_tld` command with the following
parameter:
apply a premium list to a TLD,
[update the TLD to set the premium list](modifying-tlds.md):
```shell
$ nomulus -e {ENVIRONMENT} update_tld exampletld --premium_list exampletld
Update Registry@exampletld
premiumList: null -> Key<?>(EntityGroupRoot("cross-tld")/PremiumList("exampletld"))
Perform this command? (y/N): y
Updated 1 entities.
...
pendingDeleteLength: "PT432000S"
premiumListName: "test"
pricingEngineClassName: "google.registry.model.pricing.StaticPremiumListPricingEngine"
...
```
## Checking which premium list is applied to a TLD
@@ -100,7 +111,7 @@ all other information about a TLD). It is used as follows:
```shell
$ nomulus -e {ENVIRONMENT} get_tld exampletld
[ ... snip output ... ]
premiumList=Key<?>(EntityGroupRoot("cross-tld")/PremiumList("exampletld"))
premiumListName: "test"
[ ... snip output ... ]
```
@@ -127,10 +138,10 @@ $ nomulus -e production check_domain {domain_name}
[ ... snip output ... ]
```
**Note that the list can be cached for up to 60 minutes, so the old value may
**Note that the list can be cached for up to 60 minutes, so the old value may
still be returned for a little while**. If it is urgent that the new pricing
changes be applied, and it's OK to potentially interrupt client connections,
then you can use the App Engine web console to kill instances of the `default`
then you can use the GCP web console to kill instances of the `frontend`
service, as the cache is per-instance. Once you've killed all the existing
instances (don't kill them all at once!), all of the newly spun up instances
will now be using the new values you've configured.
instances (don't kill them all at once!), all the newly spun up instances will
now be using the new values you've configured.
docs/operational-procedures/rde-deposits.md
+16 -13
View File
@@ -16,9 +16,9 @@ phases:
3. [Report](https://github.com/google/nomulus/blob/master/java/google/registry/rde/RdeReportAction.java):
Transmit XML *report* file to ICANN via HTTPS.
Each phase happens with an App Engine task queue entry that retries on failure.
When each task succeeds, it automatically enqueues a task for the next phase in
the process. The staging files are stored in Google Cloud Storage indefinitely,
Each phase happens with an GCP task queue entry that retries on failure. When
each task succeeds, it automatically enqueues a task for the next phase in the
process. The staging files are stored in Google Cloud Storage indefinitely,
encrypted with the GhostRyDE container format.
Note that in order for the automated RDE processing to work correctly, you will
@@ -99,9 +99,10 @@ that no cooldown period is necessary.
## Listing deposits in Cloud Storage
You can list the files in Cloud Storage for a given TLD using the gcloud storage tool.
All files are stored in the {PROJECT-ID}-rde bucket, where {PROJECT-ID} is the
name of the App Engine project for the particular environment you are checking.
You can list the files in Cloud Storage for a given TLD using the gcloud storage
tool. All files are stored in the {PROJECT-ID}-rde bucket, where {PROJECT-ID} is
the name of the App Engine project for the particular environment you are
checking.
```shell
$ gcloud storage ls gs://{PROJECT-ID}-rde/zip_2015-05-16*
@@ -116,10 +117,12 @@ Under normal circumstances, RDE is launched by TldFanoutAction, configured in
cron.xml. If the App Engine's cron executor isn't working, you can spawn it
manually by visiting the following URL:
https://backend-dot-{PROJECT-ID}.appspot.com/_dr/task/rdeStaging
```
https://backend.mydomain.com/_dr/task/rdeStaging
```
That will spawn a staging task for each TLD under the backend module in that App
Engine project. You can also run the task from the cron tab of the GAE console.
That will spawn a staging task for each TLD under the backend module in that GCP
project. You can also run the task from the GCP Cloud Scheduler UI.
## Notification of upload problems
@@ -157,7 +160,7 @@ space the uploading at least two hours apart.
Note that this warning only applies when you (re)upload files directly to the
sFTP server. There's an RDE_UPLOAD_SFTP cursor that prevents the production
system from uploading twice in a two hour window, so when you let the production
system from uploading twice in a two-hour window, so when you let the production
job upload missing deposits, it will be safe. Therefore, one safe approach is to
reset the cursor, then kick off the production job manually.
@@ -172,7 +175,7 @@ $ gcloud storage cat gs://{PROJECT-ID}-rde/foo.ghostryde | nomulus -e production
## Identifying which phase of the process failed
Analyze the GAE logs on the backend module.
Analyze the GCP logs on the backend module.
If the rdeStaging task failed, then it's likely the files do not exist in cloud
storage.
@@ -308,8 +311,8 @@ sftp> put ${tld}_2015-05-16_full_S1_R0.sig
It would be convenient to have the following in your `~/.ssh/config` file and
store the SSH private key that you stored in `rde-ssh-client-private` as
`~/.ssh/id_rsa_rde` so that you can simply run `$ sftp rde` to connect to
the sFTP server.
`~/.ssh/id_rsa_rde` so that you can simply run `$ sftp rde` to connect to the
sFTP server.
```
Host rde
docs/operational-procedures/reserved-list-management.md
+29 -46
View File
@@ -5,36 +5,26 @@ for various reasons, usually because of potential abuse.
## Reserved list file format
Reserved lists are handled in a similar way to [premium
lists](./premium-list-management.md), except that instead of each label having
a price, it has a reservation type. The valid values for reservation types are:
Reserved lists are handled in a similar way to
[premium lists](./premium-list-management.md), except that instead of each label
having a price, it has a reservation type. The valid values for reservation
types are:
* **`NAMESERVER_RESTRICTED`** - Only nameservers included here can be set on a
domain with this label. If the a label in this type exists on multiple
reserved lists that are applied to the same TLD. The set of allowed
nameservers for that label in that TLD is the intersection of all applicable
nameservers. Note that this restriction is orthogonal to the TLD-wide
nameserver restrictions that may be otherwise imposed. The ultimate set of
allowed nameservers for a certain domain is the intersection of per-domain
and TLD-wide allowed nameservers set. Furthermore, a TLD can be set in a
domain create restricted mode, in which case **only** domains that are
reserved with this type can be registered.
* **`ALLOWED_IN_SUNRISE`** - The label can be registered during the sunrise
period by a registrant with a valid claim but it is reserved thereafter.
* **`RESERVED_FOR_SPECIFIC_USE`** - The label is reserved for the use of a
specific registrant, and can only be registered by someone sending along the
allocation token at time of registration. This token is configured on an
`AllocationToken` entity with a matching `domainName`, and is sent by the
registrar using the [allocation token EPP
extension](https://tools.ietf.org/id/draft-ietf-regext-allocation-token-07.html).
registrar using the
[allocation token EPP extension](https://tools.ietf.org/id/draft-ietf-regext-allocation-token-07.html).
* **`RESERVED_FOR_ANCHOR_TENANT`** - Like `RESERVED_FOR_SPECIFIC_USE`, except
for an anchor tenant (i.e. a registrant participating in a [Qualified Launch
Program](https://newgtlds.icann.org/en/announcements-and-media/announcement-10apr14-en)),
for an anchor tenant (i.e. a registrant participating in a
[Qualified Launch Program](https://newgtlds.icann.org/en/announcements-and-media/announcement-10apr14-en)),
meaning that registrations can occur during sunrise ahead of GA, and must be
for a two year term.
* **`NAME_COLLISION`** - The label is reserved because it is on an [ICANN
collision
list](https://www.icann.org/resources/pages/name-collision-2013-12-06-en).
* **`NAME_COLLISION`** - The label is reserved because it is on an
[ICANN collision list](https://www.icann.org/resources/pages/name-collision-2013-12-06-en).
It may be registered during sunrise by a registrant with a valid claim but
is reserved thereafter. The `SERVER_HOLD` status is automatically applied
upon registration, which will prevent the domain name from ever resolving in
@@ -53,16 +43,13 @@ label is reserved due to name collision (with message "Cannot be delegated"). In
general `FULLY_BLOCKED` is by far the most widely used reservation type for
typical TLD use cases.
Here's an example of a small reserved list. Note that the
`NAMESERVER_RESTRICTED` label has a third entry, a colon separated list of
nameservers that the label can be delegated to:
Here's an example of a small reserved list:
```
reserveddomain,FULLY_BLOCKED
availableinga,ALLOWED_IN_SUNRISE
fourletterword,FULLY_BLOCKED
acmecorp,RESERVED_FOR_ANCHOR_TENANT
internaldomain,NAMESERVER_RESTRICTED,ns1.internal.tld:ns1.internal.tld
```
# Reserved list file name format
@@ -96,8 +83,8 @@ Updated 1 entities.
Note that `-i` is the input file containing the list. You can optionally specify
the name of the reserved list using `-n`, but when it's omitted as above the
list name is inferred from the name of the filename (minus the file extension).
For ease of tracking track of things, it is recommended to store all lists such
that the filename and list name are identical.
For ease of tracking, it is recommended to store all lists such that the
filename and list name are identical.
You're not done yet! After creating the reserved list you must the apply it to
one or more TLDs (see below) for it to actually be used.
@@ -127,22 +114,16 @@ reserved lists applied. The list of reserved labels for a TLD is the union of
all applied reserved lists, using the precedence rules described earlier when a
label appears in more than one list.
To add a reserved list to a TLD, run the `update_tld` command with the following
parameter:
To add a reserved list to a TLD, [update the TLD](modifying-tlds.md):
```shell
$ nomulus -e {ENVIRONMENT} update_tld exampletld \
--add_reserved_lists common_bad-words
Update Registry@exampletld
reservedLists: null -> [Key<?>(EntityGroupRoot("cross-tld")/ReservedList("common_bad-words"))]
Perform this command? (y/N): y
Updated 1 entities.
...
reservedListNames:
- "common_bad-words"
- "exampletld_specialized-reservations"
...
```
The `--add_reserved_lists` parameter can take a comma-delimited list of reserved
list names if you are applying multiple reserved lists to a TLD. There is also a
`--remove_reserved_lists` parameter that functions as you might expect.
Naming rules are enforced: reserved lists that start with `common_` can be
applied to any TLD (though they don't automatically apply to all TLDs), whereas
reserved lists that start with the name of a TLD can only be applied to the TLD
@@ -156,9 +137,11 @@ purposes here. It is used as follows:
```shell
$ nomulus -e {ENVIRONMENT} get_tld exampletld
[ ... snip output ... ]
reservedLists=[Key<?>(EntityGroupRoot("cross-tld")/ReservedList("common_bad-words"))]
[ ... snip output ... ]
...
reservedListNames:
- "common_bad-words"
- "exampletld_specialized-reservations"
...
```
## Listing all available reserved lists
@@ -184,10 +167,10 @@ $ nomulus -e production check_domain {domain_name}
[ ... snip output ... ]
```
**Note that the list can be cached for up to 60 minutes, so changes may not
take place immediately**. If it is urgent that the new changes be applied, and
it's OK to potentially interrupt client connections, then you can use the App
Engine web console to kill instances of the `default` service, as the cache is
**Note that the list can be cached for up to 60 minutes, so changes may not take
place immediately**. If it is urgent that the new changes be applied, and it's
OK to potentially interrupt client connections, then you can use the GCP web
console to kill instances of the `frontend` service, as the cache is
per-instance. Once you've killed all the existing instances (don't kill them all
at once!), all of the newly spun up instances will now be using the new values
at once!), all the newly spun up instances will now be using the new values
you've configured.
docs/operational-procedures/tld-security-restrictions.md
+21 -105
View File
@@ -1,119 +1,35 @@
# TLD security restrictions
Nomulus has several security features that allow registries to impose additional
restrictions on which domains are allowed on a TLD and what
registrant/nameservers they can have. The restrictions can be applied to an
entire TLD or on a per-domain basis. These restrictions are intended for use on
closed TLDs that need to allow external registrars, and prevent undesired domain
registrations or updates from occurring, e.g. if a registrar makes an error or
is compromised. For closed TLDs that do not need external registrars, a simpler
solution is to not grant any registrars access to the TLD.
restrictions on which domains are allowed on a TLD and what nameservers they can
have. The restrictions can be applied to an entire TLD or on a per-domain basis.
These restrictions are intended for use on closed TLDs that need to allow
external registrars, and prevent undesired domain registrations or updates from
occurring, e.g. if a registrar makes an error or is compromised. For closed TLDs
that do not need external registrars, a simpler solution is to not grant any
registrars access to the TLD.
This document outlines the various restrictions available, their use cases, and
how to apply them.
## TLD-wide nameserver/registrant restrictions
Nomulus allows registry administrators to set registrant contact and/or
nameserver restrictions on a TLD. This is typically desired for brand TLDs on
which all domains are either self-hosted or restricted to a small set of
webhosts.
Nomulus allows registry administrators to set nameserver restrictions on a TLD.
This is typically desired for brand TLDs on which all domains are either
self-hosted or restricted to a small set of webhosts.
To configure allowed nameservers on a TLD, use the
`--allowed_nameservers`, `--add_allowed_nameservers`, and
`--remove_allowed_nameservers` parameters on the `update_tld` command as
follows:
```shell
$ nomulus -e {ENVIRONMENT} update_tld --allowed_nameservers {NS1,NS2,...} {TLD}
```
Note that `--allowed_nameservers` can also be used with the `create_tld` command
when the TLD is initially created.
To set the allowed registrants, use the analogous `--allowed_registrants`,
`--add_allowed_registrants`, and `--remove_allowed_registrants` parameters:
```shell
$ nomulus -e {ENVIRONMENT} update_tld \
--allowed_registrants {CONTACTID1,CONTACTID2,...} {TLD}
```
When nameserver or registrant restrictions are set on a TLD, any domain mutation
flow under that TLD will verify that the supplied nameservers or registrants
are not empty and that they are a strict subset of the allowed nameservers and
registrants on the TLD. If no restrictions are set, domains can be created or
updated without nameservers, but registrant is still always required.
## Per-domain nameserver restrictions
Registries can also elect to impose per-domain nameserver restrictions. This
restriction is orthogonal to the TLD-wide nameserver restriction detailed above.
Any domain mutation must pass both validations (if applicable). In practice, it
is recommended to maintain consistency between the two types of lists by making
the per-domain allowed nameserver list a subset of the TLD-wide one, because any
nameservers that are not included in both lists are effectively disallowed.
The per-domain allowed nameserver lists are configured in [reserved
list](./reserved-list-management.md) entries with the reservation type
`NAMESERVER_RESTRICTED`. The final element in the entry is the colon-delimited
list of nameservers, e.g.:
To [configure allowed nameservers on a TLD](modifying-tlds.md), use the
`allowedFullyQualifiedHostNames` field in the TLD YAML file:
```
restrictedsld,NAMESERVER_RESTRICTED,ns1.mycompany.tld:ns2.mycompany.tld
addGracePeriodLength: "PT432000S"
allowedFullyQualifiedHostNames:
- "ns1.test.goog"
- "ns2.test.goog"
- "ns3.test.goog"
```
Note that multiple reserved lists can be applied to a TLD. If different reserved
lists contain nameserver restrictions for the same label, then the resulting
restriction set is the set intersection of all allowed nameserver lists for that
label.
## Domain create restriction on closed TLDs
Nomulus offers the ability to "lock-down" a TLD so that domain registration is
forbidden except for allow-listed domain names. This is achieved by setting the
"domain create restricted" option on the TLD using the `nomulus` tool. Domains
are allow-listed for registration by adding them to reserved lists with entries
of type `NAMESERVER_RESTRICTED`. Each domain will thus also need to have
explicitly allowed nameservers configured in its reserved list entry, per the
previous section.
To apply domain create restriction when creating/updating a TLD, use the
`--domain_create_restricted` parameter as follows:
```shell
$ nomulus -e {ENVIRONMENT} [create_tld | update_tld] \
--domain_create_restricted [true | false] {TLD}
```
Note that you do **not** have to set a TLD-wide allowed nameservers list with
this option, because it operates independently from the per-domain nameservers
restriction that `NAMESERVER_RESTRICTED` reservation imposes.
In addition to disabling registration of non-allow-listed domains, setting a TLD
as domain create restricted also applies the `SERVER_UPDATE_PROHIBITED` and
`SERVER_TRANSFER_PROHIBITED` statuses to domains upon creation. Any domains on a
domain create restricted TLD are therefore virtually immutable, and must be
unlocked by the registry operator before each change can be made. For more
information on these EPP statuses, see [RFC
5731](https://tools.ietf.org/html/rfc5731#section-2.3).
To an unlock a locked domain so that a registrar can make changes, the registry
operator must remove the status using a `nomulus` tool command as follows:
```shell
$ nomulus -e {ENVIRONMENT} update_server_locks \
--remove SERVER_UPDATE_PROHIBITED,SERVER_TRANSFER_PROHIBITED \
--client {REGISTRAR_CLIENT_ID}
--n {DOMAIN}
```
Note that these statuses will be reapplied immediately after any transfer/update
so long as the TLD is still set to domain create restricted.
Since the domain create restricted facility is intended for use on closed TLDs,
validation/server lock does not happen in domain application and allocate flows.
Most closed TLDs do not have a sunrise period, so this is fine, but for the
unanticipated occasion that a sunrise period is necessary, it suffices to
manually ensure that all domains are correct immediately after entering general
availability, after which no additional disallowed changes can be made.
When nameserver restrictions are set on a TLD, any domain mutation flow under
that TLD will verify that the supplied nameservers are not empty and that they
are a strict subset of the allowed nameservers and registrants on the TLD. If no
restrictions are set, domains can be created or updated without nameservers.