mirrors/nomulus
1
0
Fork 0
mirror of https://github.com/google/nomulus synced 2026-08-15 11:46:08 +00:00
Code Issues Releases Wiki Activity

Update more of the documentation (#2974)

Browse Source 
We should be at least at a "good enough" state after this -- I'm sure
there are many updates we could make that would improve the
documentation but this is definitely much improved from before and
should hopefully be good enough to get people started.
This commit is contained in:
gbrodman
2026-03-03 20:25:30 +00:00
committed by GitHub
parent 25fcef8a5b
commit 72016b1e5f
6 changed files with 461 additions and 598 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/proxy-setup.md
+142 -201
View File
@@ -2,22 +2,22 @@
This doc covers procedures to configure, build and deploy the
[Netty](https://netty.io)-based proxy onto [Kubernetes](https://kubernetes.io)
clusters. [Google Kubernetes
Engine](https://cloud.google.com/kubernetes-engine/) is used as deployment
target. Any kubernetes cluster should in theory work, but the user needs to
change some dependencies on other GCP features such as Cloud KMS for key
management and Stackdriver for monitoring.
clusters.
[Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/) is used
as deployment target. Any kubernetes cluster should in theory work, but the user
needs to change some dependencies on other GCP features such as Cloud KMS for
key management and Stackdriver for monitoring.
## Overview
Nomulus runs on Google App Engine, which only supports HTTP(S) traffic. In order
to work with [EPP](https://tools.ietf.org/html/rfc5730.html) (TCP port 700) and
[WHOIS](https://tools.ietf.org/html/rfc3912) (TCP port 43), a proxy is needed to
relay traffic between clients and Nomulus and do protocol translation.
Nomulus runs on GKE, and natively only supports HTTP(S) traffic. In order to
work with [EPP](https://tools.ietf.org/html/rfc5730.html) (TCP port 700), a
proxy is needed to relay traffic between clients and Nomulus and do protocol
translation.
We provide a Netty-based proxy that runs as a standalone service (separate from
Nomulus) either on a VM or Kubernetes clusters. Deploying to kubernetes is
recommended as it provides automatic scaling and management for docker
Nomulus) either on a VM or Kubernetes clusters. Deploying to Kubernetes is
recommended as it provides automatic scaling and management for Docker
containers that alleviates much of the pain of running a production service.
The procedure described here can be used to set up a production environment, as
@@ -26,13 +26,13 @@ However, proper release management (cutting a release, rolling updates, canary
analysis, reliable rollback, etc) is not covered. The user is advised to use a
service like [Spinnaker](https://www.spinnaker.io/) for release management.
## Detailed Instruction
## Detailed Instructions
We use [`gcloud`](https://cloud.google.com/sdk/gcloud/) and
[`terraform`](https://terraform.io) to configure the proxy project on GCP and to create a GCS
bucket for storing the terraform state file. We use
[`kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/) to deploy
the proxy to the project. These instructions assume that all three tools are
[`terraform`](https://terraform.io) to configure the proxy project on GCP and to
create a GCS bucket for storing the terraform state file. We use
[`kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/) to deploy
the proxy to the project. These instructions assume that all three tools are
installed.
### Setup GCP project
@@ -41,9 +41,9 @@ There are three projects involved:
- Nomulus project: the project that hosts Nomulus.
- Proxy project: the project that hosts this proxy.
- GCR ([Google Container
Registry](https://cloud.google.com/container-registry/)) project: the
project from which the proxy pulls its Docker image.
- GCR
([Google Container Registry](https://cloud.google.com/container-registry/))
project: the project from which the proxy pulls its Docker image.
We recommend using the same project for Nomulus and the proxy, so that logs for
both are collected in the same place and easily accessible. If there are
@@ -64,16 +64,16 @@ $ gcloud storage buckets create gs://<bucket-name>/ --project <proxy-project>
### Obtain a domain and SSL certificate
The proxy exposes two endpoints, whois.\<yourdomain.tld\> and
epp.\<yourdomain.tld\>. The base domain \<yourdomain.tld\> needs to be obtained
from a registrar ([Google Domains](https://domains.google) for example). Nomulus
operators can also self-allocate a domain in the TLDs under management.
The proxy exposes one endpoint: `epp.<yourdomain.tld>`. The base domain
`<yourdomain.tld>` needs to be obtained from a registrar (RIP to
[Google Domains](https://domains.google/)). Nomulus operators can also
self-allocate a domain in the TLDs under management.
[EPP protocol over TCP](https://tools.ietf.org/html/rfc5734) requires a
client-authenticated SSL connection. The operator of the proxy needs to obtain
an SSL certificate for domain epp.\<yourdomain.tld\>. [Let's
Encrypt](https://letsencrypt.org) offers SSL certificate free of charge, but any
other CA can fill the role.
an SSL certificate for domain `epp.<yourdomain.tld>`.
[Let's Encrypt](https://letsencrypt.org) offers SSL certificate free of charge,
but any other CA can fill the role.
Concatenate the certificate and its private key into one file:
@@ -82,7 +82,7 @@ $ cat <certificate.pem> <private.key> > <combined_secret.pem>
```
The order between the certificate and the private key inside the combined file
does not matter. However, if the certificate file is chained, i. e. it contains
does not matter. However, if the certificate file is chained, i.e. it contains
not only the certificate for your domain, but also certificates from
intermediate CAs, these certificates must appear in order. The previous
certificate's issuer must be the next certificate's subject.
@@ -92,8 +92,9 @@ bucket will be created automatically by terraform.
### Setup proxy project
First setup the [Application Default
Credential](https://cloud.google.com/docs/authentication/production) locally:
First setup the
[Application Default Credential](https://cloud.google.com/docs/authentication/production)
locally:
```bash
$ gcloud auth application-default login
@@ -102,10 +103,9 @@ $ gcloud auth application-default login
Login with the account that has "Project Owner" role of all three projects
mentioned above.
Navigate to `proxy/terraform`, create a folder called
`envs`, and inside it, create a folder for the environment that proxy is
deployed to ("alpha" for example). Copy `example_config.tf` and `outputs.tf`
to the environment folder.
Navigate to `proxy/terraform`, create a folder called `envs`, and inside it,
create a folder for the environment that proxy is deployed to ("alpha" for
example). Copy `example_config.tf` and `outputs.tf` to the environment folder.
```bash
$ cd proxy/terraform
@@ -132,12 +132,12 @@ takes a couple of minutes.
### Setup Nomulus
After terraform completes, it outputs some information, among which is the
email address of the service account created for the proxy. This needs to be
added to the Nomulus configuration file so that Nomulus accepts traffic from the
proxy. Edit the following section in
`java/google/registry/config/files/nomulus-config-<env>.yaml` and redeploy
Nomulus:
After terraform completes, it outputs some information, among which is the email
address of the service account created for the proxy. This needs to be added to
the Nomulus configuration file so that Nomulus accepts traffic from the proxy.
Edit the following section in
`core/src/main/java/google/registry/config/files/nomulus-config-<env>.yaml` and
redeploy Nomulus:
```yaml
auth:
@@ -148,7 +148,7 @@ auth:
### Setup nameservers
The terraform output (run `terraform output` in the environment folder to show
it again) also shows the nameservers of the proxy domain (\<yourdomain.tld\>).
it again) also shows the nameservers of the proxy domain (`<yourdomain.tld>`).
Delegate this domain to these nameservers (through your registrar). If the
domain is self-allocated by Nomulus, run:
@@ -160,8 +160,8 @@ $ nomulus -e production update_domain <yourdomain.tld> \
### Setup named ports
Unfortunately, terraform currently cannot add named ports on the instance groups
of the GKE clusters it manages. [Named
ports](https://cloud.google.com/compute/docs/load-balancing/http/backend-service#named_ports)
of the GKE clusters it manages.
[Named ports](https://cloud.google.com/compute/docs/load-balancing/http/backend-service#named_ports)
are needed for the load balancer it sets up to route traffic to the proxy. To
set named ports, in the environment folder, do:
@@ -189,8 +189,9 @@ $ gcloud storage cp <combined_secret.pem.enc> gs://<your-certificate-bucket>
### Edit proxy config file
Proxy configuration files are at `java/google/registry/proxy/config/`. There is
a default config that provides most values needed to run the proxy, and several
Proxy configuration files are at
`proxy/src/main/java/google/registry/proxy/config/`. There is a default config
that provides most values needed to run the proxy, and several
environment-specific configs for proxy instances that communicate to different
Nomulus environments. The values specified in the environment-specific file
override those in the default file.
@@ -202,16 +203,33 @@ detailed descriptions on each field.
### Upload proxy docker image to GCR
Edit the `proxy_push` rule in `java/google/registry/proxy/BUILD` to add the GCR
project name and the image name to save to. Note that as currently set up, all
images pushed to GCR will be tagged `bazel` and the GKE deployment object loads
the image tagged as `bazel`. This is fine for testing, but for production one
should give images unique tags (also configured in the `proxy_push` rule).
The GKE deployment manifest is set up to pull the proxy docker image from
[Google Container Registry](https://cloud.google.com/container-registry/) (GCR).
Instead of using `docker` and `gcloud` to build and push images, respectively,
we provide `gradle` rules for the same tasks. To push an image, first use
[`docker-credential-gcr`](https://github.com/GoogleCloudPlatform/docker-credential-gcr)
to obtain necessary credentials. It is used by the Gradle to push the image.
After credentials are configured, verify that Gradle will use the proper
`gcpProject` for deployment in the main `build.gradle` file. We recommend using
the same project and image for proxies intended for different Nomulus
environments, this way one can deploy the same proxy image first to sandbox for
testing, and then to production.
To push to GCR, run:
```bash
$ bazel run java/google/registry/proxy:proxy_push
$ ./gradlew proxy:pushProxyImage
```
If the GCP project to host images (gcr project) is different from the project
that the proxy runs in (proxy project), give the service account "Storage Object
Viewer" role of the gcr project.
```bash
$ gcloud projects add-iam-policy-binding <image-project> \
--member serviceAccount:<service-account-email> \
--role roles/storage.objectViewer
```
### Deploy proxy
@@ -243,9 +261,9 @@ Repeat this for all three clusters.
### Afterwork
Remember to turn on [Stackdriver
Monitoring](https://cloud.google.com/monitoring/docs/) for the proxy project as
we use it to collect metrics from the proxy.
Remember to turn on
[Stackdriver Monitoring](https://cloud.google.com/monitoring/docs/) for the
proxy project as we use it to collect metrics from the proxy.
You are done! The proxy should be running now. You should store the private key
safely, or delete it as you now have the encrypted file shipped with the proxy.
@@ -278,14 +296,14 @@ in multiple zones to provide geographical redundancy.
### Create service account
The proxy will run with the credential of a [service
account](https://cloud.google.com/compute/docs/access/service-accounts). In
theory it can take advantage of [Application Default
Credentials](https://cloud.google.com/docs/authentication/production) and use
the service account that the GCE instance underpinning the GKE cluster uses, but
we recommend creating a separate service account. With a dedicated service
account, one can grant permissions only necessary to the proxy. To create a
service account:
The proxy will run with the credential of a
[service account](https://cloud.google.com/compute/docs/access/service-accounts).
In theory, it can take advantage of
[Application Default Credentials](https://cloud.google.com/docs/authentication/production)
and use the service account that the GCE instance underpinning the GKE cluster
uses, but we recommend creating a separate service account. With a dedicated
service account, one can grant permissions only necessary to the proxy. To
create a service account:
```bash
$ gcloud iam service-accounts create proxy-service-account \
@@ -303,10 +321,10 @@ $ gcloud iam service-accounts keys create proxy-key.json --iam-account \
A `proxy-key.json` file will be created inside the current working directory.
The service account email address needs to be added to the Nomulus
configuration file so that Nomulus accepts the OAuth tokens generated for this
service account. Add its value to
`java/google/registry/config/files/nomulus-config-<env>.yaml`:
The service account email address needs to be added to the Nomulus configuration
file so that Nomulus accepts the OAuth tokens generated for this service
account. Add its value to
`core/src/main/java/google/registry/config/files/nomulus-config-<env>.yaml`:
```yaml
auth:
@@ -325,27 +343,13 @@ $ gcloud projects add-iam-policy-binding <project-id> \
--role roles/logging.logWriter
```
### Obtain a domain and SSL certificate
A domain is needed (if you do not want to rely on IP addresses) for clients to
communicate to the proxy. Domains can be purchased from a domain registrar
([Google Domains](https://domains.google) for example). A Nomulus operator could
also consider self-allocating a domain under an owned TLD insteadl.
An SSL certificate is needed as [EPP over
TCP](https://tools.ietf.org/html/rfc5734) requires SSL. You can apply for an SSL
certificate for the domain name you intended to serve as EPP endpoint
(epp.nic.tld for example) for free from [Let's
Encrypt](https://letsencrypt.org). For now, you will need to manually renew your
certificate before it expires.
### Create keyring and encrypt the certificate/private key
The proxy needs access to both the private key and the certificate. Do *not*
package them directly with the proxy. Instead, use [Cloud
KMS](https://cloud.google.com/kms/) to encrypt them, ship the encrypted file
with the proxy, and call Cloud KMS to decrypt them on the fly. (If you want to
use another keyring solution, you will have to modify the proxy and implement
package them directly with the proxy. Instead, use
[Cloud KMS](https://cloud.google.com/kms/) to encrypt them, ship the encrypted
file with the proxy, and call Cloud KMS to decrypt them on the fly. (If you want
to use another keyring solution, you will have to modify the proxy and implement
yours)
Concatenate the private key file with the certificate. It does not matter which
@@ -378,7 +382,7 @@ A file named `ssl-cert-key.pem.enc` will be created. Upload it to a GCS bucket
in the proxy project. To create a bucket and upload the file:
```bash
$ gcloud storage buckets create gs://<bucket-name> --project <proxy-project>
$ gcloud storage buckets create gs://<bucket-name> --project <proxy-project>
$ gcloud storage cp ssl-cert-key.pem.enc gs://<bucket-name>
```
@@ -402,8 +406,9 @@ $ gcloud storage buckets add-iam-policy-binding gs://<bucket-name> \
### Proxy configuration
Proxy configuration files are at `java/google/registry/proxy/config/`. There is
a default config that provides most values needed to run the proxy, and several
Proxy configuration files are at
`proxy/src/main/java/google/registry/proxy/config/`. There is a default config
that provides most values needed to run the proxy, and several
environment-specific configs for proxy instances that communicate to different
Nomulus environments. The values specified in the environment-specific file
override those in the default file.
@@ -416,12 +421,12 @@ for detailed descriptions on each field.
### Setup Stackdriver for the project
The proxy streams metrics to
[Stackdriver](https://cloud.google.com/stackdriver/). Refer to [Stackdriver
Monitoring](https://cloud.google.com/monitoring/docs/) documentation on how to
enable monitoring on the GCP project.
[Stackdriver](https://cloud.google.com/stackdriver/). Refer to
[Stackdriver Monitoring](https://cloud.google.com/monitoring/docs/)
documentation on how to enable monitoring on the GCP project.
The proxy service account needs to have ["Monitoring Metric
Writer"](https://cloud.google.com/monitoring/access-control#predefined_roles)
The proxy service account needs to have
["Monitoring Metric Writer"](https://cloud.google.com/monitoring/access-control#predefined_roles)
role in order to stream metrics to Stackdriver:
```bash
@@ -464,44 +469,6 @@ tag for all clusters.
Repeat this for all the zones you want to create clusters in.
### Upload proxy docker image to GCR
The GKE deployment manifest is set up to pull the proxy docker image from
[Google Container Registry](https://cloud.google.com/container-registry/) (GCR).
Instead of using `docker` and `gcloud` to build and push images, respectively,
we provide `bazel` rules for the same tasks. To push an image, first use
[`docker-credential-gcr`](https://github.com/GoogleCloudPlatform/docker-credential-gcr)
to obtain necessary credentials. It is used by the [bazel container_push
rules](https://github.com/bazelbuild/rules_docker#authentication) to push the
image.
After credentials are configured, edit the `proxy_push` rule in
`java/google/registry/proxy/BUILD` to add the GCP project name and the image
name to save to. We recommend using the same project and image for proxies
intended for different Nomulus environments, this way one can deploy the same
proxy image first to sandbox for testing, and then to production.
Also note that as currently set up, all images pushed to GCR will be tagged
`bazel` and the GKE deployment object loads the image tagged as `bazel`. This is
fine for testing, but for production one should give images unique tags (also
configured in the `proxy_push` rule).
To push to GCR, run:
```bash
$ bazel run java/google/registry/proxy:proxy_push
```
If the GCP project to host images (gcr project) is different from the project
that the proxy runs in (proxy project), give the service account "Storage Object
Viewer" role of the gcr project.
```bash
$ gcloud projects add-iam-policy-binding <image-project> \
--member serviceAccount:<service-account-email> \
--role roles/storage.objectViewer
```
### Upload proxy service account key to GKE cluster
The kubernetes pods (containers) are configured to read the proxy service
@@ -555,22 +522,22 @@ Repeat the same step for all clusters you want to deploy to.
The proxies running on GKE clusters need to be exposed to the outside. Do not
use Kubernetes
[`LoadBalancer`](https://kubernetes.io/docs/concepts/services-networking/service/#type-loadbalancer).
It will create a GCP [Network Load
Balancer](https://cloud.google.com/compute/docs/load-balancing/network/), which
has several problems:
It will create a GCP
[Network Load Balancer](https://cloud.google.com/compute/docs/load-balancing/network/),
which has several problems:
- This load balancer does not terminate TCP connections. It simply acts as an
edge router that forwards IP packets to a "healthy" node in the cluster. As
such, it does not support IPv6, because GCE instances themselves are
currently IPv4 only.
- IP packets that arrived on the node may be routed to another node for
- IP packets that arrived at the node may be routed to another node for
reasons of capacity and availability. In doing so it will
[SNAT](https://en.wikipedia.org/wiki/Network_address_translation#SNAT) the
packet, therefore losing the source IP information that the proxy needs. The
proxy uses WHOIS source IP address to cap QPS and passes EPP source IP to
Nomulus for validation. Note that a TCP terminating load balancer also has
this problem as the source IP becomes that of the load balancer, but it can
be addressed in other ways (explained later). See
proxy uses source IP address to cap QPS and passes EPP source IP to Nomulus
for validation. Note that a TCP terminating load balancer also has this
problem as the source IP becomes that of the load balancer, but it can be
addressed in other ways (explained later). See
[here](https://kubernetes.io/docs/tutorials/services/source-ip/) for more
details on how Kubernetes route traffic and translate source IPs inside the
cluster.
@@ -581,8 +548,8 @@ has several problems:
Instead, we split the task of exposing the proxy to the Internet into two tasks,
first to expose it within the cluster, then to expose the cluster to the outside
through a [TCP Proxy Load
Balancer](https://cloud.google.com/compute/docs/load-balancing/tcp-ssl/tcp-proxy).
through a
[TCP Proxy Load Balancer](https://cloud.google.com/compute/docs/load-balancing/tcp-ssl/tcp-proxy).
This load balancer terminates TCP connections and allows for the use of a single
anycast IP address (IPv4 and IPv6) to reach any clusters connected to its
backend (it chooses a particular cluster based on geographical proximity). From
@@ -611,8 +578,8 @@ $ kubectl create -f \
proxy/kubernetes/proxy-service.yaml
```
This service object will open up port 30000 (health check), 30001 (WHOIS) and
30002 (EPP) on the nodes, routing to the same ports inside a pod.
This service object will open up port 30000 (health check) and 30002 (EPP) on
the nodes, routing to the same ports inside a pod.
Repeat this for all clusters.
@@ -641,7 +608,7 @@ Then set the named ports:
```bash
$ gcloud compute instance-groups set-named-ports <instance-group> \
--named-ports whois:30001,epp:30002 --zone <zone>
--named-ports epp:30002 --zone <zone>
```
Repeat this for each instance group (cluster).
@@ -689,7 +656,7 @@ routed to the corresponding port on a proxy pod. The backend service codifies
which ports on the node's clusters should receive traffic from the load
balancer.
Create one backend service for EPP and one for WHOIS:
Create a backend service for EPP:
```bash
# EPP backend
@@ -697,28 +664,18 @@ $ gcloud compute backend-services create proxy-epp-loadbalancer \
--global --protocol TCP --health-checks proxy-health --timeout 1h \
--port-name epp
# WHOIS backend
$ gcloud compute backend-services create proxy-whois-loadbalancer \
--global --protocol TCP --health-checks proxy-health --timeout 1h \
--port-name whois
```
These two backend services route packets to the epp named port and whois named
port on any instance group attached to them, respectively.
This ackend service routes packets to the EPP named port on any instance group
attached to it.
Then add (attach) instance groups that the proxies run on to each backend
service:
Then add (attach) instance groups that the proxies run on the backend service:
```bash
# EPP backend
$ gcloud compute backend-services add-backend proxy-epp-loadbalancer \
--global --instance-group <instance-group> --instance-group-zone <zone> \
--balancing-mode UTILIZATION --max-utilization 0.8
# WHOIS backend
$ gcloud compute backend-services add-backend proxy-whois-loadbalancer \
--global --instance-group <instance-group> --instance-group-zone <zone> \
--balancing-mode UTILIZATION --max-utilization 0.8
```
Repeat this for each instance group.
@@ -747,10 +704,10 @@ $ gcloud compute addresses describe proxy-ipv4 --global
$ gcloud compute addresses describe proxy-ipv6 --global
```
Set these IP addresses as the A/AAAA records for both epp.<nic.tld> and
whois.<nic.tld> where <nic.tld> is the domain that was obtained earlier. (If you
use [Cloud DNS](https://cloud.google.com/dns/) as your DNS provider, this step
can also be performed by `gcloud`)
Set these IP addresses as the A/AAAA records for epp.<nic.tld>where <nic.tld> is
the domain that was obtained earlier. (If you use
[Cloud DNS](https://cloud.google.com/dns/) as your DNS provider, this step can
also be performed by `gcloud`)
#### Create load balancer frontend
@@ -761,21 +718,16 @@ First create a TCP proxy (yes, it is confusing, this GCP resource is called
"proxy" as well) which is a TCP termination point. Outside connections terminate
on a TCP proxy, which establishes its own connection to the backend services
defined above. As such, the source IP address from the outside is lost. But the
TCP proxy can add the [PROXY protocol
header](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt) at the
beginning of the connection to the backend. The proxy running on the backend can
parse the header and obtain the original source IP address of a request.
Make one for each protocol (EPP and WHOIS).
TCP proxy can add the
[PROXY protocol header](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt)
at the beginning of the connection to the backend. The proxy running on the
backend can parse the header and obtain the original source IP address of a
request.
```bash
# EPP
$ gcloud compute target-tcp-proxies create proxy-epp-proxy \
--backend-service proxy-epp-loadbalancer --proxy-header PROXY_V1
# WHOIS
$ gcloud compute target-tcp-proxies create proxy-whois-proxy \
--backend-service proxy-whois-loadbalancer --proxy-header PROXY_V1
```
Note the use of the `--proxy-header` flag, which turns on the PROXY protocol
@@ -785,47 +737,36 @@ Next, create the forwarding rule that route outside traffic to a given IP to the
TCP proxy just created:
```bash
$ gcloud compute forwarding-rules create proxy-whois-ipv4 \
--global --target-tcp-proxy proxy-whois-proxy \
--address proxy-ipv4 --ports 43
$ gcloud compute forwarding-rules create proxy-epp-ipv4 \
--global --target-tcp-proxy proxy-epp-proxy \
--address proxy-ipv4 --ports 700
```
The above command sets up a forwarding rule that routes traffic destined to the
static IPv4 address reserved earlier, on port 43 (actual port for WHOIS), to the
TCP proxy that connects to the whois backend service.
static IPv4 address reserved earlier, on port 700 (actual port for EPP), to the
TCP proxy that connects to the EPP backend service.
Repeat the above command another three times, set up IPv6 forwarding for WHOIS,
and IPv4/IPv6 forwarding for EPP.
Repeat the above command to set up IPv6 forwarding for EPP
## Additional steps
### Check if it all works
At this point the proxy should be working and reachable from the Internet. Try
if a whois request to it is successful:
```bash
whois -h whois.<nic.tld> something
```
One can also try to contact the EPP endpoint with an EPP client.
### Check logs and metrics
The proxy saves logs to [Stackdriver
Logging](https://cloud.google.com/logging/), which is the same place that
Nomulus saves it logs to. On GCP console, navigate to Logging - Logs - GKE
Container - <cluster name> - default. Do not choose "All namespace_id" as it
includes logs from the Kubernetes system itself and can be quite overwhelming.
The proxy saves logs to
[Stackdriver Logging](https://cloud.google.com/logging/), which is the same
place that Nomulus saves it logs to. On GCP console, navigate to Logging -
Logs - GKE Container - <cluster name> - default. Do not choose "All
namespace_id" as it includes logs from the Kubernetes system itself and can be
quite overwhelming.
Metrics are stored in [Stackdriver
Monitoring](https://cloud.google.com/monitoring/docs/). To view the metrics, go
to Stackdriver [console](https://app.google.stackdriver.com) (also accessible
from GCE console under Monitoring), navigate to Resources - Metrics Explorer.
Choose resource type "GKE Container" and search for metrics with name "/proxy/"
in it. Currently available metrics include total connection counts, active
connection count, request/response count, request/response size, round-trip
latency and quota rejection count.
Metrics are stored in
[Stackdriver Monitoring](https://cloud.google.com/monitoring/docs/). To view the
metrics, go to Stackdriver [console](https://app.google.stackdriver.com) (also
accessible from GCE console under Monitoring), navigate to Resources - Metrics
Explorer. Choose resource type "GKE Container" and search for metrics with name
"/proxy/" in it. Currently available metrics include total connection counts,
active connection count, request/response count, request/response size,
round-trip latency and quota rejection count.
### Cleanup sensitive files