nomulus/docs/configuration.md

# Configuration

There are multiple different kinds of configuration that go into getting a
working registry system up and running. Broadly speaking, configuration works in
two ways -- globally, for the entire system, and per-TLD. Global configuration
is managed by editing code and deploying a new version, whereas per-TLD
configuration is data that lives in the database in `Tld` entities, and
[is updated](operational-procedures/modifying-tlds.md) without having to deploy
a new version.

## Initial configuration

Here's a checklist of things that need to be configured upon initial
installation of the project:

*   Create Google Cloud Storage buckets (see the
    [Architecture documentation](./architecture.md) for more information).
*   Modify configuration files ("nomulus-config-*.yaml") for all environments
    you wish to deploy.

## Environments

Before getting into the details of configuration, it's important to note that a
lot of configuration is environment-dependent. It is common to see `switch`
statements that operate on the current `RegistryEnvironment`, and return
different values for different environments. This is especially pronounced in
the `UNITTEST` and `LOCAL` environments, which don't run on GCP at all. As an
example, some timeouts may be long in production and short in unit tests.

See the [Architecture documentation](./architecture.md) for more details on
environments as used by Nomulus.

TODO: documentation about how to set up GKE and what config points are necessary
to modify there

## Global configuration

Global configuration is managed through YAML files that are built with and
deployed in the app. The full list of config options and their default values
can be found in the [`default-config.yaml`][default-config] file. If you wish to
change any of these values, do not edit this file. Instead, edit the environment
configuration file named
`core/src/main/java/google/registry/config/files/nomulus-config-ENVIRONMENT.yaml`,
overriding only the options you wish to change. Nomulus ships with blank
placeholders for all standard environments.

You will not need to change most of the default settings. Here is the subset of
settings that you will need to change for all deployed environments, including
development environments. See [`default-config.yaml`][default-config] for a full
description of each option:

```yaml
gcpProject:
  projectId: # Your GCP project ID
  projectIdNumber: # The corresponding ID number, found on the home page
  locationId: # e.g. us-central1
  isLocal: false # Causes saved credentials to be used
  baseDomain: # the base domain from which the registry will be served, e.g. registry.google

gSuite:
  domainName: # Your GSuit domain name, likely same as baseDomain above
  adminAccountEmailAddress: # An admin login for your GSuite account

auth:
  allowedServiceAccountEmails:
    - # a list of service account emails given access to Nomulus
  oauthClientId: # the client ID of the Identity-Aware Proxy

cloudSql:
  jdbcUrl: # path to the Postgres server

```

For fully-featured production environments that need the full range of features
(e.g. RDE, correct contact information on the registrar console, etc.) you will
need to specify *many* more settings.

From a code perspective, all configuration settings ultimately come through the
[`RegistryConfig`][registry-config] class. This includes a Dagger module called
`ConfigModule` that provides injectable configuration options. While most
configuration options can be changed from within the yaml config file, certain
derived options may still need to be overridden by changing the code in this
module.

## OAuth 2 client ID configuration

Nomulus uses OAuth 2 to authenticate and authorize users. This includes the
`nomulus` [command-line tool](admin-tool.md) when it connects to the system to
execute commands as well as the
[Identity-Aware Proxy](https://pantheon.corp.google.com/security/iap) used to
authenticate standard requests. OAuth must be configured before you can use
either system.

OAuth defines the concept of a *client ID*, which identifies the application
which the user wants to authorize. This is so that, when a user clicks in an
OAuth permission dialog and grants access to data, they are not granting access
to every application on their computer (including potentially malicious ones),
but only to the application which they agree needs access. Each environment of
the Nomulus system should have its own pair of client IDs. Multiple
installations of the `nomulus` tool application can share the same client ID for
the same environment.

For the Nomulus tool OAuth configuration, do the following steps:

*   **Create the registry tool client ID in GCP:** Go to your project's
    ["Credentials" page](https://console.developers.google.com/apis/credentials)
    in the Developer's Console. Click "Create credentials" and select "OAuth
    client ID" from the dropdown. In the create credentials window, select an
    application type of "Desktop app". After creating the client ID, copy the
    client ID and client secret which are displayed in the popup window. You may
    also obtain this information by downloading the JSON file for the client ID

*   **Copy the client secret information to the config file:** The *client
    secret file* contains both the client ID and the client secret. Copy the
    respective values to the config file for the environment that the credential
    is created for (e. g. `nomulus-config-production.yaml`) under the
    `registryTool` section. This will make the `nomulus` tool use this
    credential to authenticate itself to the system.

For IAP configuration, do the following steps: * **Create the IAP client ID:**
Follow similar steps from above to create an additional OAuth client ID, but
using an application type of "Web application". Note the client ID and secret. *
**Enable IAP for your HTTPS load balancer:** On the
[IAP page](https://pantheon.corp.google.com/security/iap), enable IAP for all of
the backend services that all use the same HTTPS load balancer. * **Use a custom
OAuth configuration:** For the backend services, under the "Settings" section
(in the three-dot menu) enable custom OAuth and insert the client ID and secret
that we just created * **Save the client ID:** In the configuration file, save
the client ID as `oauthClientId` in the `auth` section

Once these steps are taken, the `nomulus` tool and IAP will both use client IDs
which the server is configured to accept, and authentication should succeed.
Note that many Nomulus commands also require that the user have GCP admin
privileges on the project in question.

## Sensitive global configuration

Some configuration values, such as PGP private keys, are so sensitive that they
should not be written in code as per the configuration methods above, as that
would pose too high a risk of them accidentally being leaked, e.g. in a source
control mishap. We use a secret store to persist these values in a secure
manner, which is backed by the GCP Secret Manager.

The `Keyring` interface contains methods for all sensitive configuration values,
which are primarily credentials used to access various ICANN and
ICANN-affiliated services (such as RDE). These values are only needed for real
production registries and PDT environments. If you are just playing around with
the platform at first, it is OK to put off defining these values until
necessary. This allows the codebase to start and run, but of course any actions
that attempt to connect to external services will fail because if the relevant
key is not found in the Secret Manager.

To configure a production registry system, you will need to add the required
keys to the Secret Manager. To do so, you can use the Nomulus tool's
`update_keyring_secret` command. First, run `nomulus -e ${ENV}
update_keyring_secret` to get the list of all key names (whose meanings should
be obvious); then, for each key to be added or updated, put the data in a file
and run `nomulus -e ${ENV} update_keyring_secret --input ${FILE} --keyname
${KEY_NAME}`.

## Per-TLD configuration

`Tld` entities, which are persisted to the database and stored in YAML files,
are used for per-TLD configuration. They contain any kind of configuration that
is specific to a TLD, such as the create/renew price of a domain name, the
pricing engine implementation, the DNS writer implementation, whether escrow
exports are enabled, the default currency, the reserved label lists, and more.

To create or update TLDs, we use
[YAML files](operational-procedures/modifying-tlds.md) and the `nomulus
configure_tld` command. Because the TLDs are stored as data in the running
system, they do not require code pushes to update.

[app-engine-config]: https://cloud.google.com/appengine/docs/java/configuration-files
[default-config]: https://github.com/google/nomulus/blob/master/java/google/registry/config/files/default-config.yaml
[registry-config]: https://github.com/google/nomulus/blob/master/java/google/registry/config/RegistryConfig.java

## Cloud SQL Configuration

Nomulus requires access to Cloud SQL and thus the necessary configuration must
be applied.

### Create Postgres Cloud SQL Instance

You can create a cloud SQL instance using the gcloud command:

```
$ gcloud sql instances create nomulus --database-version=POSTGRES_17 \
   --cpu=1 --memory=4G
```

Note that for a production instance, you will likely want to be far more
generous with both CPU and memory resources.

Now set the password for the default user:

```
$ gcloud sql users set-password postgres \
    --instance=nomulus --project=$PROJECT_ID \
    --prompt-for-password
```

Store this password somewhere secure.

Now create database users for the tool and for the backend. First, you'll need
to create a password. This can simply be a sequence of random characters. Write
it to the file `/tmp/server.pass` (we'll use a single password for the two user
accounts here, you are encouraged to use different passwords for your production
systems). Make sure that this file does not contain a newline after the
password. Now create the two user accounts:

```
$ gcloud sql users create nomulus --instance=nomulus \
  --project=$PROJECT_ID "--password=`cat /tmp/server.pass`"
$ gcloud sql users create tool --instance=nomulus \
  --project=$PROJECT_ID "--password=`cat /tmp/server.pass`"
```

Now enable access to the Cloud SQL admin APIs:

```
$ gcloud services enable sqladmin.googleapis.com \
    --project=$PROJECT_ID
```

Finally, add the database instance names to the keyring. First, get the
connection name for the new database:

```
$ gcloud sql instances describe nomulus | grep connectionName
connectionName: your-project:us-central1:nomulus
```

Use the `update_keyring_secret` command to update the `SQL_PRIMARY_CONN_NAME`
key with the connection name. If you have created a read-replica, update the
`SQL_REPLICA_CONN_NAME` key with the replica's connection name.

### Installing the Schema

Google's Nomulus team makes use of Spinnaker-based continuous integration to
perform weekly pushes of both the Nomulus software and the SQL database schema.
Organizations wishing to use the Nomulus software will likely want to do
something similar. However, for purposes of this exercise we will push the
schema from the build system.

First, download the
[Cloud SQL Proxy](https://cloud.google.com/sql/docs/mysql/sql-proxy). This will
allow you to connect to your database from a local workstation without a lot of
additional configuration.

Create a service account for use with the proxy:

```
$ gcloud iam service-accounts create sql-proxy \
    --project=$PROJECT_ID \
    --description="Service account for use with Cloud SQL Proxy" \
    --display-name="Cloud SQL Proxy"
```

Give the service account admin permissions:

```
$ gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member=serviceAccount:sql-proxy@$PROJECT_ID.iam.gserviceaccount.com \
    --role=roles/cloudsql.admin
```

Create a JSON key for the service account:

```
$ gcloud iam service-accounts keys create sql-admin.json \
    --project=$PROJECT_ID \
    --iam-account=sql-proxy@$PROJECT_ID.iam.gserviceaccount.com
```

Now start the proxy:

```
$ PORT=3306   # Use a different value for this if you like.
$ ./cloud_sql_proxy -credential_file=sql-admin.json \
    -instances=$PROJECT_ID:nomulus=tcp:$PORT
2020/07/01 12:11:20 current FDs rlimit set to 32768, wanted limit is 8500. Nothing to do here.
2020/07/01 12:11:20 using credential file for authentication; email=sql-proxy@pproject-id.iam.gserviceaccount.com
2020/07/01 12:11:20 Listening on 127.0.0.1:3306 for project-id:nomulus
2020/07/01 12:11:20 Ready for new connections
```

Finally, upload the new database schema:

```
$ ./nom_build :db:flywayMigrate --dbServer=localhost:$PORT \
    --dbName=postgres --dbUser=nomulus --dbPassword=`cat /tmp/server.pass`
```

Now you'll need to give the "tool" user access to all tables. You can do this
either with a locally installed version of PostgreSQL or from the Cloud Shell.
From local postgres, first, with your proxy is still running, connect using
psql.

```
$ psql -h localhost -p 3306 postgres nomulus                                                                                                                         ~/w/nom.admin-docs
Password for user nomulus: <enter the password from /tmp/server.pass>
psql (12.2 (Debian 12.2-1+build2), server 11.6)
Type "help" for help.

postgres=>
```

Enter the following command at the postgres prompt:

```
GRANT SELECT, INSERT, UPDATE, DELETE
ON ALL TABLES IN SCHEMA public
TO tool;
```

From the [Google Cloud Console](https://console.developers.google.com), click
the cloud shell icon in the toolbar (the ">_" icon). You should be able to
connect to your database with gcloud:

```
$ gcloud sql connect nomulus --user=nomulus
```

From this, you should have a postgres prompt and be able to enter the "GRANT"
command specified above.

### Replication and Backups

We highly recommend creating a read-only replica of the database and using the
previously-mentioned `SQL_REPLICA_CONN_NAME` value in the keyring to the name of
that replica. By doing so, you can remove some load from the primary database.

We also recommend enabling
[point-in-time recovery](https://docs.cloud.google.com/sql/docs/postgres/backup-recovery/pitr)
for the instance, just in case something bad happens and you need to restore
from a backup.

### Cloud SecretManager

You'll need to enable the SecretManager API in your project.

#### Install Cloud SQL Passwords in Nomulus Server

Use the update_keyring_secret command to upload the Cloud SQL passwords to the
Nomulus server. We'll use the password same set of passwords we specified above
when creating database user accounts. These should currently be stored in
`/tmp/server.pass`.

Paste the password for the Registry server user to a file, say /tmp/server.pass.
Make sure to avoid any trailing '\n' inserted by the editor.

```
$ set ENV=alpha
$ nomulus -e $ENV update_keyring_secret --keyname CLOUD_SQL_PASSWORD \
    --input /tmp/server.pass
```

Repeat the steps for the tools sql password:

```
$ nomulus -e $ENV update_keyring_secret --keyname TOOLS_CLOUD_SQL_PASSWORD \
    --input /tmp/tools.pass
```

Use get_keyring_secret command to verify the data you put in:

```
$ nomulus -e alpha -e alpha get_keyring_secret --keyname CLOUD_SQL_PASSWORD
[your password]
$ nomulus -e alpha -e alpha get_keyring_secret --keyname CLOUD_SQL_PASSWORD
[your password]
```