mirror of
https://github.com/google/nomulus
synced 2026-08-02 05:16:08 +00:00
Ben McIlwainandGitHub
dcfe939c38
Summarize all documentation updates across the repository to align with modern GKE, Cloud SQL Proxy v2, standard EPP fee v1.0, and Postgres database environments. Key Updates: - Prerequisites: Bump Java requirement to Java 25. - Architecture & Scaling: Document GKE workloads, Cloud Tasks queues, and scheduled tasks. Replace App Engine references with GKE deployment restart commands (kubectl rollout restart). - Configuration: Update Cloud SQL Proxy instructions to v2, fix keyring verification commands, and document IAP configuration. - Escrow (RDE/BRDA): Fix manual generation and download procedures to match the Dataflow job ID folder structure, and correct deposit encryption/verification command parameters. - Monitoring: Correct metric names and expand the documented metrics list with caching, locking, and reserved list metrics. - Fixes: Standardize lists formatting across markdown files, fix broken webdriver links, and resolve various typos. - Cleanup: Remove leftover cloud scheduler configurations for the deleted wipeOutContactHistoryPii task, and update ICANN reporting documentation to reflect open-sourced DNS query coordinator. TAG=agy CONV=88271e71-e272-40e0-85f8-a075a423b7c2
98 lines
3.5 KiB
Markdown
98 lines
3.5 KiB
Markdown
# BRDA Deposits
|
|
|
|
Bulk Registration Data Access (BRDA) is a backup deposit program mandated by
|
|
ICANN for most gTLD registrars (ccTLDs are not required to provide BRDA
|
|
deposits). Some information related to BRDA can be found at:
|
|
|
|
https://icannwiki.com/Onboarding_Information_Request#BRDA
|
|
|
|
BRDA deposits are generated by the
|
|
[RdeStagingAction][rde-staging-action]
|
|
job. This is the same job that generates RDE deposits. Its Javadoc goes into
|
|
great detail about how it's implemented.
|
|
|
|
The [RDE task](./rde-deposits.md) performs BRDA processing at 00:00:00 UTC every
|
|
Tuesday. RDE runs every day, but only performs the following BRDA steps on
|
|
Tuesday (configurable via the `@Config("brdaDayOfWeek")` parameter):
|
|
|
|
* Check the BRDA `Cursor`
|
|
* Create a staging file named:
|
|
* `gs://{PROJECT-ID}-rde/TLD_YYYY-MM-DD_thin_S1_R0.xml.ghostryde`
|
|
* Enqueue a BrdaCopyTask and roll forward the cursor
|
|
|
|
The BRDA copy task reads the previous file and creates two files:
|
|
|
|
```
|
|
gs://{PROJECT-ID}-icann-brda/TLD_YYYY-MM-DD_thin_S1_R0.ryde
|
|
gs://{PROJECT-ID}-icann-brda/TLD_YYYY-MM-DD_thin_S1_R0.sig
|
|
```
|
|
|
|
If you see an `xml.ghostryde` file but not the others, an error has occurred
|
|
during the process. If you see the files in the {PROJECT-ID}-icann-brda bucket
|
|
as well, the process has completed successfully.
|
|
|
|
Once the files have been created, they must be stored on an sFTP server from
|
|
which ICANN can pull the files. The Nomulus project does not provide this last
|
|
step; you will need to set up an sFTP server yourself, and copy the files from
|
|
Google Cloud Storage to the server.
|
|
|
|
The cursor can be checked using the `nomulus pending_escrow` command.
|
|
|
|
## Generating BRDA deposits manually
|
|
|
|
* Get a list of "REAL" (as opposed to TEST) TLDs. Double-check that the
|
|
command output doesn't contain any TLDs for tests.
|
|
|
|
```shell
|
|
$ nomulus -e production list_tlds --fields=tldStr,tldType \
|
|
| grep REAL | awk '{print $1}' > realtlds.txt
|
|
```
|
|
|
|
* Kick off the server-side generation of thin escrow XML files (GhostRyDE
|
|
encrypted) under the GCS RDE bucket manual directory:
|
|
|
|
```shell
|
|
$ for date in 2015-02-26 2015-03-05; \
|
|
do for tld in $(cat realtlds.txt); \
|
|
do nomulus -e production generate_escrow_deposit \
|
|
--tld=${tld} \
|
|
--watermark=${date}T00:00:00Z \
|
|
--mode=THIN \
|
|
--outdir=manual_brda; \
|
|
done; \
|
|
done
|
|
```
|
|
|
|
* Download and decrypt the GhostRyDE files locally, then encrypt them for
|
|
sending to the escrow provider (note that files are located in
|
|
subdirectories named after the Dataflow job under `manual_brda`, which
|
|
we match using a wildcard):
|
|
|
|
```shell
|
|
$ mkdir /tmp/brda_out; for date in 2015-02-26 2015-03-05; \
|
|
do for tld in $(cat realtlds.txt); \
|
|
do \
|
|
gcloud storage cat \
|
|
gs://{PROJECT-ID}-rde/manual/manual_brda/*/${tld}_${date}_thin_S1_R0.xml.ghostryde \
|
|
| nomulus -e production ghostryde --decrypt \
|
|
> /tmp/${tld}_${date}_thin_S1_R0.xml; \
|
|
nomulus -e production encrypt_escrow_deposit \
|
|
--mode=THIN \
|
|
--tld=${tld} \
|
|
--input=/tmp/${tld}_${date}_thin_S1_R0.xml \
|
|
--outdir=/tmp/brda_out; \
|
|
rm /tmp/${tld}_${date}_thin_S1_R0.xml; \
|
|
done; \
|
|
done
|
|
```
|
|
|
|
* Store the generated `.ryde` and `.sig` files to the BRDA GCS bucket.
|
|
|
|
```shell
|
|
$ gcloud storage cp /tmp/brda_out/*.{ryde,sig} gs://{PROJECT-ID}-icann-brda/
|
|
```
|
|
|
|
* Mirror the files in the GCS bucket to the sFTP server.
|
|
|
|
[rde-staging-action]: https://github.com/google/nomulus/blob/master/core/src/main/java/google/registry/rde/RdeStagingAction.java
|