Update README.md (#2705 )

Log session cookies in metadata (#2703 )
There are two session cookies, JSESSIONID, which is set by Jetty, and GCLB, which is set by the Gateway. In one session, every request other than the first one (the <hello>) should have the same GCLB value, and every request after a successful <login> should have the same JSESSIONID. With these two metadata, we should be able to trace all requests that *should* belong to the same session and debug issues with session mismatch (if any).
2026-07-07 00:27:12 +00:00 · 2025-03-05 16:55:04 +00:00 · 2025-03-04 20:18:15 +00:00 · 2025-03-04 13:15:15 +00:00 · 2025-03-04 04:05:53 +00:00 · 2025-02-28 21:57:49 +00:00
66 changed files with 683 additions and 452 deletions
@@ -12,16 +12,16 @@ Nomulus is an open source, scalable, cloud-based service for operating
 [top-level domains](https://en.wikipedia.org/wiki/Top-level_domain) (TLDs). It
 is the authoritative source for the TLDs that it runs, meaning that it is
 responsible for tracking domain name ownership and handling registrations,
-renewals, availability checks, and WHOIS requests. End-user registrants (i.e.
+renewals, availability checks, and WHOIS requests. End-user registrants (i.e.,
 people or companies that want to register a domain name) use an intermediate
 domain name registrar acting on their behalf to interact with the registry.

-Nomulus runs on [Google App Engine][gae] and is written primarily in Java. It is
-the software that [Google Registry](https://www.registry.google/) uses to
-operate TLDs such as .google, .app, .how, .soy, and .みんな. It can run any
-number of TLDs in a single shared registry system using horizontal scaling. Its
-source code is publicly available in this repository under the [Apache 2.0 free
-and open source license](https://www.apache.org/licenses/LICENSE-2.0).
+Nomulus runs on [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine)
+and is written primarily in Java. It is the software that
+[Google Registry](https://www.registry.google/) uses to operate TLDs such as .google,
+.app, .how, .soy, and .みんな. It can run any number of TLDs in a single shared registry
+system using horizontal scaling. Its source code is publicly available in this
+repository under the [Apache 2.0 free and open source license](https://www.apache.org/licenses/LICENSE-2.0).

 ## Getting started

@@ -54,9 +54,11 @@ Nomulus has the following capabilities:
    checking, updating, and transferring domain names.
 *   **[DNS](https://en.wikipedia.org/wiki/Domain_Name_System) interface**: The
    registry provides a pluggable interface that can be implemented to handle
-    different DNS providers. It includes a sample implementation using Google
-    Cloud DNS as well as an RFC 2136 compliant implementation that works with
-    BIND.
+    different DNS providers. It includes a sample implementation using [Google
+    Cloud DNS](https://cloud.google.com/dns/), as well as an RFC 2136 compliant
+    implementation that works with BIND. If you are using Google Cloud DNS, you
+    may need to understand its capabilities and provide your own
+    multi-[AS](https://en.wikipedia.org/wiki/Autonomous_system_\(Internet\)) solution.
 *   **[WHOIS](https://en.wikipedia.org/wiki/WHOIS)**: A text-based protocol that
    returns ownership and contact information on registered domain names.
 *   **[Registration Data Access Protocol
@@ -68,7 +70,7 @@ Nomulus has the following capabilities:
    provider to allow take-over by another registry operator in the event of
    serious failure. This is required by ICANN for all [new
    gTLDs](https://newgtlds.icann.org/).
-*   **Premium pricing**: Communicates prices for premium domain names (i.e.
+*   **Premium pricing**: Communicates prices for premium domain names (i.e.,
    those that are highly desirable) and supports configurable premium
    registration and renewal prices. An extensible interface allows fully
    programmatic pricing.
@@ -91,56 +93,50 @@ Nomulus has the following capabilities:
 *   **Administrative tool**: Performs the full range of administrative tasks
    needed to manage a running registry system, including creating and
    configuring new TLDs.
-*   **DNS interface**: An interface for DNS operations is provided so you can
-    write an implementation for your chosen provider, along with a sample
-    implementation that uses [Google Cloud DNS](https://cloud.google.com/dns/).
-    If you are using Google Cloud DNS you may need to understand its
-    capabilities and provide your own
-    multi-[AS](https://en.wikipedia.org/wiki/Autonomous_system_\(Internet\))
-    solution.
-*   **GAE Proxy**: App Engine Standard only serves HTTP/S traffic. A proxy to
-    forward traffic on EPP and WHOIS ports to App Engine via HTTPS is provided.
-    Instructions on setting up the proxy on
-    [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/)
-    is [available](https://github.com/google/nomulus/blob/master/docs/proxy-setup.md).
-    Running the proxy on GKE supports IPv4 and IPv6 access, per ICANN's
-    requirements for gTLDs. The proxy can also run as a single jar file, or on
-    other Kubernetes providers, with modifications.
+*   **Secure storage of cryptographic keys**: A keyring interface is
+      provided for plugging in your own implementation (see [configuration
+      doc](https://github.com/google/nomulus/blob/master/docs/configuration.md)
+      for details), and an implementation based on
+      [Google Cloud Secret Manager](https://cloud.google.com/security/products/secret-manager) is
+      available.
+*   **TPC Proxy**: Nomulus is built on top of the [Jetty](https://jetty.org/)
+    container that implements the [Jakarta Servlet](https://jakarta.ee/specifications/servlet/)
+    specification and only serves HTTP/S traffic. A proxy to translate raw TCP traffic (e.g., EPP)
+    to and from HTTP is provided.
+    Instructions on setting up the proxy
+    are [available](https://github.com/google/nomulus/blob/master/docs/proxy-setup.md).
+    The proxy can either run in a separate cluster and communicate to Nomulus public HTTP
+    endpoints via the Internet, or as a sidecar with the Nomulus image in the same pod and
+    communicate to it via loopback.

 ## Additional components

 Registry operators interested in deploying Nomulus will likely require some
-additional components that are need to be configured separately.
+additional components that need to be configured separately.

 *   A way to invoice registrars for domain name registrations and accept
    payments. Nomulus records the information required to generate invoices in
    [billing
    events](https://github.com/google/nomulus/blob/master/docs/code-structure.md#billing-events).
 *   Fully automated reporting to meet ICANN's requirements for gTLDs. Nomulus
-    includes substantial reporting functionality but some additional work will
+    includes substantial reporting functionality, but some additional work will
    be required by the operator in this area.
-*   A secure method for storing cryptographic keys. A keyring interface is
-    provided for plugging in your own implementation (see [configuration
-    doc](https://github.com/google/nomulus/blob/master/docs/configuration.md)
-    for details).
+
 *   System status and uptime monitoring.

 ## Outside references

-*   [Donuts](http://donuts.domains) Registry has helped review the code and
-    provided valuable feedback
+*   [Identity Digital](http://identity.digital) has helped review the code and
+    provided valuable feedback.
 *   [CoCCa](http://cocca.org.nz) and [FRED](https://fred.nic.cz) are other
-    open-source registry platforms in use by many TLDs
+    open-source registry platforms in use by many TLDs.
 *   We are not aware of any fully open source domain registrar projects, but
    open source EPP Toolkits (not yet tested with Nomulus; may require
    integration work) include:
-    *   [EPP RTK Project](http://epp-rtk.sourceforge.net/)
-    *   [CentralNic](https://www.centralnic.com/registry/labs)
+    *   [Universal Registry/Registrar Toolkit](https://sourceforge.net/projects/epp-rtk/)
    *   [ari-toolkit](https://github.com/AusRegistry/ari-toolkit)
    *   [Net::DRI](https://metacpan.org/pod/Net::DRI)
 *   Some Open Source DNS Projects that may be useful, but which we have not
    tested:
-    *   [AtomiaDNS](http://atomiadns.com/)
-    *   [PowerDNS](https://doc.powerdns.com/md/)
-
-[gae]:https://cloud.google.com/appengine/docs/about-the-standard-environment
+    *   [AtomiaDNS](https://github.com/atomia/atomiadns)
+    *   [PowerDNS](https://github.com/PowerDNS/pdns)
@@ -63,6 +63,7 @@ public class TextDiffSubject extends Subject {

  private final ImmutableList<String> actual;
  private DiffFormat diffFormat = DiffFormat.SIDE_BY_SIDE_MARKDOWN;
+  private ImmutableList<String> comments = ImmutableList.of();

  protected TextDiffSubject(FailureMetadata metadata, List<String> actual) {
    super(metadata, actual);
@@ -83,10 +84,22 @@ public class TextDiffSubject extends Subject {
    return this;
  }

+  /** If set, ignore lines that start with the given string. */
+  public TextDiffSubject ignoringLinesStartingWith(String... comments) {
+    this.comments = ImmutableList.copyOf(comments);
+    return this;
+  }
+
+  private ImmutableList<String> filterComments(List<String> lines) {
+    return lines.stream()
+        .filter(line -> comments.stream().noneMatch(line::startsWith))
+        .collect(ImmutableList.toImmutableList());
+  }
+
  public void hasSameContentAs(List<String> expectedContent) {
    checkNotNull(expectedContent, "expectedContent");
-    ImmutableList<String> expected = ImmutableList.copyOf(expectedContent);
-    if (expected.equals(actual)) {
+    ImmutableList<String> expected = filterComments(expectedContent);
+    if (filterComments(expected).equals(filterComments(actual))) {
      return;
    }
    String diffString = diffFormat.generateDiff(expected, actual);
@@ -76,7 +76,7 @@ enum Operation {
  selector: 'app-reason-dialog',
  template: `
    <h2 mat-dialog-title>
-      Please provide a reason for {{ data.operation }} the domain(s):
+      Please provide the (EPP) reason for {{ data.operation }} the domain(s):
    </h2>
    <mat-dialog-content>
      <mat-form-field appearance="outline" style="width:100%">
@@ -27,12 +27,10 @@ export const DISABLED_ELEMENTS_PER_ROLE = {
  NONE: [
    RESTRICTED_ELEMENTS.REGISTRAR_ELEMENT,
    RESTRICTED_ELEMENTS.OTE,
-    RESTRICTED_ELEMENTS.USERS,
-    RESTRICTED_ELEMENTS.BULK_DELETE,
    RESTRICTED_ELEMENTS.SUSPEND,
  ],
-  SUPPORT_LEAD: [RESTRICTED_ELEMENTS.USERS],
-  SUPPORT_AGENT: [RESTRICTED_ELEMENTS.USERS],
+  SUPPORT_LEAD: [],
+  SUPPORT_AGENT: [],
 };

@Directive({
@@ -67,17 +67,24 @@ export class UserDetailsComponent {
  }

  deleteUser() {
-    this.isLoading = true;
-    this.usersService.deleteUser(this.userDetails()).subscribe({
-      error: (err) => {
-        this._snackBar.open(err.error || err.message);
-        this.isLoading = false;
-      },
-      complete: () => {
-        this.isLoading = false;
-        this.goBack();
-      },
-    });
+    if (
+      confirm(
+        'This will permanently delete the user ' +
+          this.userDetails().emailAddress
+      )
+    ) {
+      this.isLoading = true;
+      this.usersService.deleteUser(this.userDetails()).subscribe({
+        error: (err) => {
+          this._snackBar.open(err.error || err.message);
+          this.isLoading = false;
+        },
+        complete: () => {
+          this.isLoading = false;
+          this.goBack();
+        },
+      });
+    }
  }

  goBack() {
@@ -1,32 +0,0 @@
-----BEGIN PGP PRIVATE KEY BLOCK-----
-
-lQHYBFo8aRIBBAC8MA2xQXYvEbeLV1iMo4GC3lRFYvrUCarenhwoWufCYH6dGien
-/HhiB0eiDF672J4MtueHQ2M7UaGJgxAoQTG9c6O90vlmFFhPZ967U1MTdY/NLvDK
-bQEGzjdaUC1T/O6kr0O4GHRAyNyHa39Q75Oaj8MNdPsTmT4tDy+aFO6kKwARAQAB
-AAP9Gd59M12tUmEcGxKBwKuFVSkc6oDlvBosG/geJMoCS+0Z2pzK0MPbBJa9mSAc
-MbRgXZ0TDLwNuwzIqO+UXARCQu1ln/NlCcSzQZd5S80Of6CSoFMdFEb0kcpFW3z9
-rpZdIBpNNk2iyBro9+7JOLJgCUkZQX7jy2K4LM5eTJsnuMECANFBnrMUde43XBiT
-gixOJ5zbekGIIGq4QeRc8fJUDUhkFMq1znNriu30bB0Ld4Btlxzyn56tx8DVgx1+
-4anONuECAOY5nm2G9i46AUxQN3dB8IE0SMMHcRcz60eX68fke+1aYjdSQA/nf9hR
-l2f+gX9+y3cPqo7bFZzrDNECRm3J2IsB/2444JDTnzyME99jRYeEZGM0BXMWZEoO
-hLU7f2V8pdN1po6mZ5bZZv6LeTXWPCIqCuBxNHZAV/xH9oWmkpjnw8Sc77QpVGVz
-dCBSZWdpc3RyeSA8dGVzdC1yZWdpc3RyeUBleGFtcGxlLmNvbT6IuAQTAQIAIgUC
-WjxpEgIbAwYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQvIOfrLbgEN2NegP+
-JV+i4DxPTv3jfRDcpGy8yDANiIoBFSyARpqMqg0+TfV/UypuyjFTGfnLuQv+osce
-OKtPevH8gCc779+OqtqyNDcPooTG5K+eUYR77PYtfzsKk/Z/33tQtEJDb8WWn4G4
-R9Nh51MOz1X17oe3ih9HNvMGIrOG9VeWPsTxjXAzBoidAdgEWjxpEgEEANiasS9p
-bG53M3jeCwLX0PFgWgspMZl3QnU6bvaTsfMAHaklJ55Tj1wuaaQymHqNm6xElCN8
-MK8exDQQvPZwYVQOuoP3cHriCslLGznB943URcuxXz6R7F7WixYUeVVpQ4J0+gFu
-bR8PfThDCtHQyP+uYx9U+EVWIvuIZIchdjl9ABEBAAEAA/4xmt2sorthIf3g9pL1
-e/jfKoZ8i1rPT1NiNvdeE217neFtEPP9i5vni76ISskGOgN2hH8bkE+y7zwWQ2YP
-FyYGlvVcw2KjT7+SrAWCkgR6Y7hWib+RDcVGje+YH5MxGtBIX2W/zcOW5S9+nC3Q
-Y3Tzc3YQxF8sOeaHvrEb1tJ9eQIA5ivEjt43GgZq0nxacKLhleXyA9Z/JmwDg15z
-FCZCnPABmR72wpXzXe2gO18W3iiqwS/WFDbdSFwxDQ0lXSy8VQIA8Okv6Q2BNXEw
-H0hufK8P7aHvuOI1ll4qTw6QkY+z5hRZAcmmID3boQJeJAmVbUissYKUNJudmiUJ
-DPLQod+wiQIAtJWxlRgHvEHRjQS5tH13ERWLObBHdZcQvKcqdtTCZj1EVH7zVHpb
-qBLggo7QwPJTC+UMf/f4nPd1U2O6zXv66p5liJ8EGAECAAkFAlo8aRICGwwACgkQ
-vIOfrLbgEN141gP9GATYCoihm5igbZ0FL8YPPb5WvHpTEA4WgdIIUUCQ0TYJ2ZOC
-dK0i3qbb1xRRBJq006qSiE4vqQ7fHO8HxmEWaPLlsPvebGm39PUuzVyWx8I2w+0/
-qcxt5L2VVzbZFp6+Yoa+meRYsO77gAzUvqUG1yLWo6MD4pSUNYJA867BB/k=
-=mkAP
-----END PGP PRIVATE KEY BLOCK-----
@@ -16,10 +16,14 @@ package google.registry.module;

 import static google.registry.util.GcpJsonFormatter.setCurrentRequest;
 import static google.registry.util.GcpJsonFormatter.setCurrentTraceId;
+import static google.registry.util.GcpJsonFormatter.setLabel;
 import static google.registry.util.GcpJsonFormatter.unsetCurrentRequest;
+import static google.registry.util.GcpJsonFormatter.unsetLabels;
 import static google.registry.util.RandomStringGenerator.insecureRandomStringGenerator;
 import static google.registry.util.StringGenerator.Alphabets.HEX_DIGITS_ONLY;

+import com.google.common.base.Splitter;
+import com.google.common.collect.ImmutableSet;
 import com.google.monitoring.metrics.MetricReporter;
 import dagger.Lazy;
 import google.registry.request.RequestHandler;
@@ -30,6 +34,7 @@ import jakarta.servlet.http.HttpServletRequest;
 import jakarta.servlet.http.HttpServletResponse;
 import java.io.IOException;
 import java.util.Arrays;
+import java.util.Optional;
 import java.util.logging.ConsoleHandler;
 import java.util.logging.Handler;
 import java.util.logging.Level;
@@ -48,6 +53,9 @@ public class RegistryServlet extends ServletBase {
  private static final RequestHandler<RequestComponent> requestHandler = component.requestHandler();
  private static final Lazy<MetricReporter> metricReporter = component.metricReporter();

+  // The names of the session cookies to log.
+  private static final ImmutableSet<String> COOKIES_TO_LOG = ImmutableSet.of("JSESSIONID", "GCLB");
+
  private final String projectId;

  static {
@@ -74,11 +82,20 @@ public class RegistryServlet extends ServletBase {
    String userAgent = String.valueOf(req.getHeader("User-Agent"));
    String protocol = req.getProtocol();
    setCurrentRequest(requestMethod, requestUrl, userAgent, protocol);
+    Optional<String> maybeCookie = Optional.ofNullable(req.getHeader("Cookie"));
+    if (maybeCookie.isPresent() && !maybeCookie.get().isEmpty()) {
+      // GCLB sets the value with a leading and trailing double quote.
+      String cookie = maybeCookie.get().replace("\"", "");
+      Splitter.on(';').trimResults().withKeyValueSeparator('=').split(cookie).entrySet().stream()
+          .filter(e -> COOKIES_TO_LOG.contains(e.getKey()))
+          .forEach(e -> setLabel(e.getKey(), e.getValue()));
+    }
    try {
      super.service(req, rsp);
    } finally {
      setCurrentTraceId(null);
      unsetCurrentRequest();
+      unsetLabels();
    }
  }

@@ -276,6 +276,10 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
      }
      T result = work.call();
      txn.commit();
+      long duration = clock.nowUtc().getMillis() - txnInfo.transactionTime.getMillis();
+      if (duration >= 100) {
+        logger.atInfo().log("Transaction duration: %d milliseconds", duration);
+      }
      return result;
    } catch (Throwable e) {
      // Catch a Throwable here so even Errors would lead to a rollback.
@@ -70,7 +70,7 @@ import google.registry.rdap.RdapObjectClasses.RdapRegistrarEntity;
 import google.registry.rdap.RdapObjectClasses.SecureDns;
 import google.registry.rdap.RdapObjectClasses.Vcard;
 import google.registry.rdap.RdapObjectClasses.VcardArray;
-import google.registry.request.FullServletPath;
+import google.registry.request.RequestServerName;
 import google.registry.util.Clock;
 import jakarta.persistence.Entity;
 import java.net.Inet4Address;
@@ -114,7 +114,7 @@ public class RdapJsonFormatter {
  @Nullable
  String rdapTosStaticUrl;

-  @Inject @FullServletPath String fullServletPath;
+  @Inject @RequestServerName String serverName;
  @Inject RdapAuthorization rdapAuthorization;
  @Inject Clock clock;

@@ -267,7 +267,7 @@ public class RdapJsonFormatter {
            .setDescription(rdapTos)
            .addLink(selfLink);
    if (rdapTosStaticUrl != null) {
-      URI htmlBaseURI = URI.create(fullServletPath);
+      URI htmlBaseURI = URI.create("https//:" + serverName + "/rdap/");
      URI htmlUri = htmlBaseURI.resolve(rdapTosStaticUrl);
      noticeBuilder.addLink(
          Link.builder()
@@ -1071,7 +1071,7 @@ public class RdapJsonFormatter {

  /** Create a link relative to the RDAP server endpoint. */
  String makeRdapServletRelativeUrl(String part, String... moreParts) {
-    return makeServerRelativeUrl(fullServletPath, part, moreParts);
+    return makeServerRelativeUrl("https://" + serverName + "/rdap/", part, moreParts);
  }

  /** Create a link relative to some base server */
@@ -151,17 +151,9 @@ public final class RequestModule {
   * query string.
   */
  @Provides
-  @FullServletPath
-  static String provideFullServletPath(HttpServletRequest req) {
-    // Include the port only if it differs from the default for the scheme.
-    if (("http".equals(req.getScheme()) && (req.getServerPort() == 80))
-        || ("https".equals(req.getScheme()) && (req.getServerPort() == 443))) {
-      return String.format("%s://%s%s", req.getScheme(), req.getServerName(), req.getServletPath());
-    } else {
-      return String.format(
-          "%s://%s:%d%s",
-          req.getScheme(), req.getServerName(), req.getServerPort(), req.getServletPath());
-    }
+  @RequestServerName
+  static String provideServerName(HttpServletRequest req) {
+    return req.getServerName();
  }

  @Provides
@@ -20,11 +20,11 @@ import java.lang.annotation.RetentionPolicy;
 import javax.inject.Qualifier;

 /**
- * Dagger qualifier for the HTTP servlet path, prepended with scheme, host and port.
+ * Dagger qualifier for the server name of the HTTP request.
 *
- * <p>See {@link jakarta.servlet.http.HttpServletRequest#getServletPath}
+ * <p>See {@link jakarta.servlet.http.HttpServletRequest#getServerName()}
 */
@Qualifier
@Documented
@Retention(RetentionPolicy.RUNTIME)
-public @interface FullServletPath {}
+public @interface RequestServerName {}
@@ -106,7 +106,7 @@ public abstract class OidcTokenAuthenticationMechanism implements Authentication

    String email = (String) token.getPayload().get("email");
    if (email == null) {
-      logger.atWarning().log("No email address from the OIDC token:\n%s", token.getPayload());
+      logger.atInfo().log("No email address from the OIDC token:\n%s", token.getPayload());
      return AuthResult.NOT_AUTHENTICATED;
    }
    Optional<User> maybeUser =
@@ -25,7 +25,6 @@ import com.beust.jcommander.Parameters;
 import com.beust.jcommander.ParametersDelegate;
 import com.google.common.base.Throwables;
 import com.google.common.collect.ImmutableMap;
-import com.google.common.collect.ImmutableSet;
 import com.google.common.collect.Iterables;
 import google.registry.persistence.transaction.JpaTransactionManager;
 import google.registry.persistence.transaction.TransactionManagerFactory;
@@ -42,13 +41,6 @@ import org.postgresql.util.PSQLException;
@Parameters(separators = " =", commandDescription = "Command-line interface to the registry")
 final class RegistryCli implements CommandRunner {

-  private static final ImmutableSet<RegistryToolEnvironment> DEFAULT_GKE_ENVIRONMENTS =
-      ImmutableSet.of(
-          RegistryToolEnvironment.ALPHA,
-          RegistryToolEnvironment.CRASH,
-          RegistryToolEnvironment.QA,
-          RegistryToolEnvironment.SANDBOX);
-
  // The environment parameter is parsed twice: once here, and once with {@link
  // RegistryToolEnvironment#parseFromArgs} in the {@link RegistryTool#main} function.
  //
@@ -78,9 +70,6 @@ final class RegistryCli implements CommandRunner {
              + "Beam pipelines")
  private String sqlAccessInfoFile = null;

-  @Parameter(names = "--gke", description = "Whether to use GKE runtime, instead of GAE")
-  private boolean useGke = false;
-
  @Parameter(names = "--gae", description = "Whether to use GAE runtime, instead of GKE")
  private boolean useGae = false;

@@ -161,12 +150,6 @@ final class RegistryCli implements CommandRunner {
      throw e;
    }

-    checkState(!useGke || !useGae, "Cannot specify both --gke and --gae");
-    // Special logic to set the default based on the environment if neither --gae nor --gke is set.
-    if (!useGke && !useGae) {
-      useGke = DEFAULT_GKE_ENVIRONMENTS.contains(environment);
-    }
-
    String parsedCommand = jcommander.getParsedCommand();
    // Show the list of all commands either if requested or if no subcommand name was specified
    // (which does not throw a ParameterException parse error above).
@@ -186,7 +169,7 @@ final class RegistryCli implements CommandRunner {
        DaggerRegistryToolComponent.builder()
            .credentialFilePath(credentialJson)
            .sqlAccessInfoFile(sqlAccessInfoFile)
-            .useGke(useGke)
+            .useGke(!useGae)
            .useCanary(useCanary)
            .build();

@@ -31,12 +31,14 @@ import com.google.api.services.directory.Directory;
 import com.google.api.services.directory.model.UserName;
 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
 import com.google.gson.annotations.Expose;
 import google.registry.config.RegistryConfig.Config;
 import google.registry.model.console.ConsolePermission;
 import google.registry.model.console.RegistrarRole;
 import google.registry.model.console.User;
 import google.registry.model.console.UserRoles;
+import google.registry.model.registrar.Registrar;
 import google.registry.persistence.VKey;
 import google.registry.request.Action;
 import google.registry.request.Action.GkeService;
@@ -44,6 +46,7 @@ import google.registry.request.HttpException.BadRequestException;
 import google.registry.request.Parameter;
 import google.registry.request.auth.Auth;
 import google.registry.tools.IamClient;
+import google.registry.util.DiffUtils;
 import google.registry.util.StringGenerator;
 import java.io.IOException;
 import java.util.List;
@@ -96,24 +99,14 @@ public class ConsoleUsersAction extends ConsoleApiAction {

  @Override
  protected void postHandler(User user) {
-    // Temporary flag while testing
-    if (user.getUserRoles().isAdmin()) {
-      checkPermission(user, registrarId, ConsolePermission.MANAGE_USERS);
-      tm().transact(this::runPostInTransaction);
-    } else {
-      consoleApiParams.response().setStatus(SC_FORBIDDEN);
-    }
+    checkPermission(user, registrarId, ConsolePermission.MANAGE_USERS);
+    tm().transact(this::runPostInTransaction);
  }

  @Override
  protected void putHandler(User user) {
-    // Temporary flag while testing
-    if (user.getUserRoles().isAdmin()) {
-      checkPermission(user, registrarId, ConsolePermission.MANAGE_USERS);
-      tm().transact(this::runUpdateInTransaction);
-    } else {
-      consoleApiParams.response().setStatus(SC_FORBIDDEN);
-    }
+    checkPermission(user, registrarId, ConsolePermission.MANAGE_USERS);
+    tm().transact(this::runUpdateInTransaction);
  }

  @Override
@@ -135,13 +128,8 @@ public class ConsoleUsersAction extends ConsoleApiAction {

  @Override
  protected void deleteHandler(User user) {
-    // Temporary flag while testing
-    if (user.getUserRoles().isAdmin()) {
-      checkPermission(user, registrarId, ConsolePermission.MANAGE_USERS);
-      tm().transact(this::runDeleteInTransaction);
-    } else {
-      consoleApiParams.response().setStatus(SC_FORBIDDEN);
-    }
+    checkPermission(user, registrarId, ConsolePermission.MANAGE_USERS);
+    tm().transact(this::runDeleteInTransaction);
  }

  private void runPostInTransaction() throws IOException {
@@ -163,6 +151,8 @@ public class ConsoleUsersAction extends ConsoleApiAction {
        this.userData.get().emailAddress,
        registrarId,
        RegistrarRole.valueOf(this.userData.get().role));
+
+    sendConfirmationEmail(registrarId, this.userData.get().emailAddress, "Added existing user");
    consoleApiParams.response().setStatus(SC_OK);
  }

@@ -186,6 +176,7 @@ public class ConsoleUsersAction extends ConsoleApiAction {
      VKey<User> key = VKey.create(User.class, email);
      tm().delete(key);
      User.revokeIapPermission(email, maybeGroupEmailAddress, cloudTasksUtils, null, iamClient);
+      sendConfirmationEmail(registrarId, email, "Deleted user");
    }

    consoleApiParams.response().setStatus(SC_OK);
@@ -232,7 +223,7 @@ public class ConsoleUsersAction extends ConsoleApiAction {
    User.Builder builder = new User.Builder().setUserRoles(userRoles).setEmailAddress(newEmail);
    tm().put(builder.build());
    User.grantIapPermission(newEmail, maybeGroupEmailAddress, cloudTasksUtils, null, iamClient);
-
+    sendConfirmationEmail(registrarId, newEmail, "Created user");
    consoleApiParams.response().setStatus(SC_CREATED);
    consoleApiParams
        .response()
@@ -251,6 +242,8 @@ public class ConsoleUsersAction extends ConsoleApiAction {
        this.userData.get().emailAddress,
        registrarId,
        RegistrarRole.valueOf(this.userData.get().role));
+
+    sendConfirmationEmail(registrarId, this.userData.get().emailAddress, "Updated user");
    consoleApiParams.response().setStatus(SC_OK);
  }

@@ -315,6 +308,20 @@ public class ConsoleUsersAction extends ConsoleApiAction {
                    .collect(toImmutableList()));
  }

+  private boolean sendConfirmationEmail(String registrarId, String emailAddress, String operation) {
+    Optional<Registrar> registrar = Registrar.loadByRegistrarId(registrarId);
+    if (registrar.isEmpty()) { // Shouldn't happen, but worth checking
+      setFailedResponse(
+          "Failed to send an email to registrar " + registrarId, SC_INTERNAL_SERVER_ERROR);
+      return false;
+    }
+    sendExternalUpdates(
+        ImmutableMap.of("Console users updated", new DiffUtils.DiffPair(operation, emailAddress)),
+        registrar.get(),
+        ImmutableSet.of());
+    return true;
+  }
+
  public record UserData(
      @Expose String emailAddress, @Expose String role, @Expose @Nullable String password) {}
 }
@@ -15,7 +15,6 @@
 package google.registry.ui.server.console.domains;

 import static com.google.common.collect.ImmutableMap.toImmutableMap;
-import static jakarta.servlet.http.HttpServletResponse.SC_FORBIDDEN;
 import static jakarta.servlet.http.HttpServletResponse.SC_OK;
 import static java.nio.charset.StandardCharsets.UTF_8;

@@ -81,11 +80,6 @@ public class ConsoleBulkDomainAction extends ConsoleApiAction {

  @Override
  protected void postHandler(User user) {
-    // Temporary flag while testing
-    if (!user.getUserRoles().isAdmin()) {
-      consoleApiParams.response().setStatus(SC_FORBIDDEN);
-      return;
-    }
    JsonElement jsonPayload =
        optionalJsonPayload.orElseThrow(
            () -> new IllegalArgumentException("Bulk action payload must be present"));
@@ -161,8 +161,6 @@ import google.registry.model.common.FeatureFlag;
 import google.registry.model.domain.Domain;
 import google.registry.model.domain.DomainHistory;
 import google.registry.model.domain.GracePeriod;
-import google.registry.model.domain.fee.BaseFee.FeeType;
-import google.registry.model.domain.fee.Fee;
 import google.registry.model.domain.fee.FeeQueryCommandExtensionItem.CommandName;
 import google.registry.model.domain.launch.LaunchNotice;
 import google.registry.model.domain.rgp.GracePeriodStatus;
@@ -287,59 +285,45 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
    persistContactsAndHosts("net"); // domain_create.xml uses hosts on "net".
  }

+  private void assertSuccessfulCreate(String domainTld, ImmutableSet<Flag> expectedBillingFlags)
+      throws Exception {
+    assertSuccessfulCreate(domainTld, expectedBillingFlags, null, 24, null);
+  }
+
  private void assertSuccessfulCreate(
-      String domainTld, ImmutableSet<BillingBase.Flag> expectedBillingFlags) throws Exception {
-    assertSuccessfulCreate(domainTld, expectedBillingFlags, null);
+      String domainTld, ImmutableSet<Flag> expectedBillingFlags, double createCost)
+      throws Exception {
+    assertSuccessfulCreate(domainTld, expectedBillingFlags, null, createCost, null);
+  }
+
+  private void assertSuccessfulCreate(
+      String domainTld, ImmutableSet<Flag> expectedBillingFlags, AllocationToken token)
+      throws Exception {
+    assertSuccessfulCreate(domainTld, expectedBillingFlags, token, 24, null);
  }

  private void assertSuccessfulCreate(
      String domainTld,
-      ImmutableSet<BillingBase.Flag> expectedBillingFlags,
-      @Nullable AllocationToken allocationToken)
+      ImmutableSet<Flag> expectedBillingFlags,
+      AllocationToken token,
+      double createCost)
+      throws Exception {
+    assertSuccessfulCreate(domainTld, expectedBillingFlags, token, createCost, null);
+  }
+
+  private void assertSuccessfulCreate(
+      String domainTld,
+      ImmutableSet<Flag> expectedBillingFlags,
+      @Nullable AllocationToken token,
+      double createCost,
+      @Nullable Integer specifiedRenewCost)
      throws Exception {
    Domain domain = reloadResourceByForeignKey();

    boolean isAnchorTenant = expectedBillingFlags.contains(ANCHOR_TENANT);
    // Set up the creation cost.
    boolean isDomainPremium = isDomainPremium(getUniqueIdFromCommand(), clock.nowUtc());
-    BigDecimal createCost = isDomainPremium ? BigDecimal.valueOf(200) : BigDecimal.valueOf(24);
-    if (isAnchorTenant) {
-      createCost = BigDecimal.ZERO;
-    }
-    if (expectedBillingFlags.contains(SUNRISE)) {
-      createCost =
-          createCost.multiply(
-              BigDecimal.valueOf(1 - RegistryConfig.getSunriseDomainCreateDiscount()));
-    }
-    if (allocationToken != null) {
-      if (allocationToken
-          .getRegistrationBehavior()
-          .equals(RegistrationBehavior.NONPREMIUM_CREATE)) {
-        createCost =
-            createCost.subtract(
-                BigDecimal.valueOf(isDomainPremium ? 87 : 0)); // premium is 100, standard 13
-      }
-      if (allocationToken.getRenewalPriceBehavior().equals(NONPREMIUM)) {
-        createCost =
-            createCost.subtract(
-                BigDecimal.valueOf(isDomainPremium ? 89 : 0)); // premium is 100, standard 11
-      }
-      if (allocationToken.getRenewalPriceBehavior().equals(SPECIFIED)) {
-        createCost =
-            createCost
-                .subtract(BigDecimal.valueOf(isDomainPremium ? 100 : 11))
-                .add(allocationToken.getRenewalPrice().get().getAmount());
-      }
-    }
-    FeesAndCredits feesAndCredits =
-        new FeesAndCredits.Builder()
-            .setCurrency(USD)
-            .addFeeOrCredit(
-                Fee.create(
-                    createCost,
-                    FeeType.CREATE,
-                    isDomainPremium(getUniqueIdFromCommand(), clock.nowUtc())))
-            .build();
+
    Money eapFee =
        Money.of(
            Tld.get(domainTld).getCurrency(),
@@ -362,7 +346,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
    RenewalPriceBehavior expectedRenewalPriceBehavior =
        isAnchorTenant
            ? RenewalPriceBehavior.NONPREMIUM
-            : Optional.ofNullable(allocationToken)
+            : Optional.ofNullable(token)
                .map(AllocationToken::getRenewalPriceBehavior)
                .orElse(RenewalPriceBehavior.DEFAULT);
    // There should be one bill for the create and one for the recurrence autorenew event.
@@ -371,13 +355,13 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
            .setReason(Reason.CREATE)
            .setTargetId(getUniqueIdFromCommand())
            .setRegistrarId("TheRegistrar")
-            .setCost(feesAndCredits.getCreateCost())
+            .setCost(Money.of(USD, BigDecimal.valueOf(createCost)))
            .setPeriodYears(2)
            .setEventTime(clock.nowUtc())
            .setBillingTime(billingTime)
            .setFlags(expectedBillingFlags)
            .setDomainHistory(historyEntry)
-            .setAllocationToken(allocationToken == null ? null : allocationToken.createVKey())
+            .setAllocationToken(Optional.ofNullable(token).map(t -> t.createVKey()).orElse(null))
            .build();

    BillingRecurrence renewBillingEvent =
@@ -391,8 +375,8 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
            .setDomainHistory(historyEntry)
            .setRenewalPriceBehavior(expectedRenewalPriceBehavior)
            .setRenewalPrice(
-                Optional.ofNullable(allocationToken)
-                    .flatMap(AllocationToken::getRenewalPrice)
+                Optional.ofNullable(specifiedRenewCost)
+                    .map(r -> Money.of(USD, BigDecimal.valueOf(r)))
                    .orElse(null))
            .build();

@@ -480,7 +464,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
    assertMutatingFlow(true);
    runFlowAssertResponse(
        CommitMode.LIVE, userPrivileges, loadFile(responseXmlFile, substitutions));
-    assertSuccessfulCreate(domainTld, ImmutableSet.of());
+    assertSuccessfulCreate(domainTld, ImmutableSet.of(), 24);
    assertNoLordn();
  }

@@ -886,10 +870,14 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
                    clock.nowUtc().plusDays(1),
                    Money.of(USD, 0)))
            .build());
-    doSuccessfulTest(
-        "example",
-        "domain_create_response_premium_eap.xml",
-        ImmutableMap.of("DOMAIN", "rich.example"));
+    assertMutatingFlow(true);
+    runFlowAssertResponse(
+        CommitMode.LIVE,
+        UserPrivileges.NORMAL,
+        loadFile(
+            "domain_create_response_premium_eap.xml", ImmutableMap.of("DOMAIN", "rich.example")));
+    assertSuccessfulCreate("example", ImmutableSet.of(), 200);
+    assertNoLordn();
  }

  /**
@@ -1336,7 +1324,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
    setEppInput("domain_create_anchor_allocationtoken.xml");
    persistContactsAndHosts();
    runFlowAssertResponse(loadFile("domain_create_anchor_response.xml"));
-    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), allocationToken);
+    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), allocationToken, 0);
    assertNoLordn();
    assertAllocationTokenWasRedeemed("abcDEF23456");
  }
@@ -1350,7 +1338,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
                .setTokenType(SINGLE_USE)
                .setDomainName("resdom.tld")
                .setRenewalPriceBehavior(SPECIFIED)
-                .setRenewalPrice(Money.of(USD, 0))
+                .setRenewalPrice(Money.of(USD, 1))
                .build());
    // Despite the domain being FULLY_BLOCKED, the non-superuser create succeeds the domain is also
    // RESERVED_FOR_SPECIFIC_USE and the correct allocation token is passed.
@@ -1359,7 +1347,8 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
    persistContactsAndHosts();
    runFlowAssertResponse(
        loadFile("domain_create_response.xml", ImmutableMap.of("DOMAIN", "resdom.tld")));
-    assertSuccessfulCreate("tld", ImmutableSet.of(RESERVED), allocationToken);
+    // $13 for the first year plus $1 renewal for the second year =
+    assertSuccessfulCreate("tld", ImmutableSet.of(RESERVED), allocationToken, 14, 1);
    assertNoLordn();
    assertAllocationTokenWasRedeemed("abc123");
  }
@@ -1379,7 +1368,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
    setEppInput("domain_create_anchor_allocationtoken.xml");
    persistContactsAndHosts();
    runFlowAssertResponse(loadFile("domain_create_anchor_response.xml"));
-    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), allocationToken);
+    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), allocationToken, 0);
    assertNoLordn();
    assertAllocationTokenWasRedeemed("abcDEF23456");
  }
@@ -1402,7 +1391,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
    clock.setTo(DateTime.parse("2009-08-16T09:00:00.0Z"));
    persistContactsAndHosts();
    runFlowAssertResponse(loadFile("domain_create_response_claims.xml"));
-    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), allocationToken);
+    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), allocationToken, 0);
    assertDomainDnsRequests("example-one.tld");
    assertClaimsLordn();
    assertAllocationTokenWasRedeemed("abcDEF23456");
@@ -1415,7 +1404,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
    persistContactsAndHosts();
    runFlowAssertResponse(
        loadFile("domain_create_response.xml", ImmutableMap.of("DOMAIN", "example.tld")));
-    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT));
+    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), 0);
    assertNoLordn();
  }

@@ -1450,7 +1439,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
                SMD_VALID_TIME.toString(),
                "EXPIRATION_TIME",
                SMD_VALID_TIME.plusYears(2).toString())));
-    assertSuccessfulCreate("tld", ImmutableSet.of(SUNRISE, ANCHOR_TENANT));
+    assertSuccessfulCreate("tld", ImmutableSet.of(SUNRISE, ANCHOR_TENANT), 0);
  }

  @Test
@@ -1482,7 +1471,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
                SMD_VALID_TIME.toString(),
                "EXPIRATION_TIME",
                SMD_VALID_TIME.plusYears(2).toString())));
-    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT, SUNRISE), allocationToken);
+    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT, SUNRISE), allocationToken, 0);
    assertDomainDnsRequests("test-validate.tld");
    assertSunriseLordn();
    assertAllocationTokenWasRedeemed("abcDEF23456");
@@ -1505,7 +1494,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
    setEppInput("domain_create_anchor_allocationtoken.xml");
    persistContactsAndHosts();
    runFlowAssertResponse(loadFile("domain_create_anchor_response.xml"));
-    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), allocationToken);
+    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), allocationToken, 0);
    assertNoLordn();
    assertAllocationTokenWasRedeemed("abcDEF23456");
  }
@@ -1929,7 +1918,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
        loadFile(
            "domain_create_response_premium.xml",
            ImmutableMap.of("EXDATE", "2001-04-03T22:00:00.0Z", "FEE", "200.00")));
-    assertSuccessfulCreate("example", ImmutableSet.of());
+    assertSuccessfulCreate("example", ImmutableSet.of(), 200);
  }

  private BillingEvent runTest_defaultToken(String token) throws Exception {
@@ -2148,7 +2137,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
        loadFile(
            "domain_create_response_premium.xml",
            ImmutableMap.of("EXDATE", "2001-04-03T22:00:00.0Z", "FEE", "200.00")));
-    assertSuccessfulCreate("example", ImmutableSet.of());
+    assertSuccessfulCreate("example", ImmutableSet.of(), 200);
  }

  @Test
@@ -2572,7 +2561,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
    persistContactsAndHosts();
    persistBsaLabel("anchor");
    runFlowAssertResponse(loadFile("domain_create_anchor_response.xml"));
-    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), allocationToken);
+    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), allocationToken, 0);
    assertNoLordn();
    assertAllocationTokenWasRedeemed("abcDEF23456");
  }
@@ -2738,7 +2727,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
                SMD_VALID_TIME.toString(),
                "EXPIRATION_TIME",
                SMD_VALID_TIME.plusYears(2).toString())));
-    assertSuccessfulCreate("tld", ImmutableSet.of(SUNRISE));
+    assertSuccessfulCreate("tld", ImmutableSet.of(SUNRISE), 20.40);
    assertSunriseLordn();
  }

@@ -2761,7 +2750,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
                SMD_VALID_TIME.toString(),
                "EXPIRATION_TIME",
                SMD_VALID_TIME.plusYears(2).toString())));
-    assertSuccessfulCreate("tld", ImmutableSet.of(SUNRISE));
+    assertSuccessfulCreate("tld", ImmutableSet.of(SUNRISE), 20.40);
    assertSunriseLordn();
  }

@@ -3245,7 +3234,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
        "domain_create_allocationtoken.xml",
        ImmutableMap.of("DOMAIN", "example.tld", "YEARS", "2"));
    runFlow();
-    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), token);
+    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), token, 0);
  }

  @Test
@@ -3265,7 +3254,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
        "domain_create_premium_allocationtoken.xml",
        ImmutableMap.of("YEARS", "2", "FEE", "111.00"));
    runFlow();
-    assertSuccessfulCreate("example", ImmutableSet.of(), token);
+    assertSuccessfulCreate("example", ImmutableSet.of(), token, 111);
  }

  @Test
@@ -3286,7 +3275,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
        "domain_create_premium_allocationtoken.xml",
        ImmutableMap.of("YEARS", "2", "FEE", "101.00"));
    runFlow();
-    assertSuccessfulCreate("example", ImmutableSet.of(), token);
+    assertSuccessfulCreate("example", ImmutableSet.of(), token, 101, 1);
  }

  @Test
@@ -3438,7 +3427,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
        ImmutableMap.of("DOMAIN", "example.tld", "YEARS", "2"));
    persistContactsAndHosts();
    runFlow();
-    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), token);
+    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), token, 0);
  }

  @Test
@@ -3492,7 +3481,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
        "domain_create_allocationtoken.xml",
        ImmutableMap.of("DOMAIN", "example.tld", "YEARS", "2"));
    runFlow();
-    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), token);
+    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), token, 0);
  }

  @Test
@@ -3510,7 +3499,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
            .build());
    persistContactsAndHosts();
    runFlow();
-    assertSuccessfulCreate("tld", ImmutableSet.of(SUNRISE, ANCHOR_TENANT), allocationToken);
+    assertSuccessfulCreate("tld", ImmutableSet.of(SUNRISE, ANCHOR_TENANT), allocationToken, 0);
  }

  @Test
@@ -3572,7 +3561,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
        "domain_create_allocationtoken.xml",
        ImmutableMap.of("DOMAIN", "example.tld", "YEARS", "2"));
    runFlow();
-    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), token);
+    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), token, 0);
  }

  @Test
@@ -3599,7 +3588,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
    setEppInput("domain_create_allocationtoken_claims.xml");
    persistContactsAndHosts();
    runFlow();
-    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), allocationToken);
+    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), allocationToken, 0);
  }

  @Test
@@ -3649,7 +3638,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
        "domain_create_allocationtoken.xml",
        ImmutableMap.of("DOMAIN", "example.tld", "YEARS", "2"));
    runFlow();
-    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), token);
+    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), token, 0);
  }

  @Test
@@ -3663,7 +3652,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
    setEppInput("domain_create_allocationtoken_claims.xml");
    persistContactsAndHosts();
    runFlow();
-    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), allocationToken);
+    assertSuccessfulCreate("tld", ImmutableSet.of(ANCHOR_TENANT), allocationToken, 0);
  }

  @Test
@@ -406,26 +406,6 @@ class RdapJsonFormatterTest {
        .isEqualTo(loadJson("rdapjson_registrant_logged_out.json"));
  }

-  @Test
-  void testRegistrant_baseHasNoTrailingSlash() {
-    // First, make sure we have a trailing slash at the end by default!
-    // This test tries to change the default state, if the default doesn't have a /, then this test
-    // doesn't help.
-    assertThat(rdapJsonFormatter.fullServletPath).endsWith("/");
-    rdapJsonFormatter.fullServletPath =
-        rdapJsonFormatter.fullServletPath.substring(
-            0, rdapJsonFormatter.fullServletPath.length() - 1);
-    assertAboutJson()
-        .that(
-            rdapJsonFormatter
-                .createRdapContactEntity(
-                    contactRegistrant,
-                    ImmutableSet.of(RdapEntity.Role.REGISTRANT),
-                    OutputDataType.FULL)
-                .toJson())
-        .isEqualTo(loadJson("rdapjson_registrant.json"));
-  }
-
  @Test
  void testAdmin() {
    assertAboutJson()
@@ -150,7 +150,7 @@ class RdapTestHelper {
  static RdapJsonFormatter getTestRdapJsonFormatter(Clock clock) {
    RdapJsonFormatter rdapJsonFormatter = new RdapJsonFormatter();
    rdapJsonFormatter.rdapAuthorization = RdapAuthorization.PUBLIC_AUTHORIZATION;
-    rdapJsonFormatter.fullServletPath = "https://example.tld/rdap/";
+    rdapJsonFormatter.serverName = "example.tld";
    rdapJsonFormatter.clock = clock;
    rdapJsonFormatter.rdapTos =
        ImmutableList.of(
@@ -23,7 +23,7 @@ public class NomulusPostgreSql {

  /** The current PostgreSql version in Cloud SQL. */
  // TODO(weiminyu): setup periodic checks to detect version changes in Cloud SQL.
-  private static final String TARGET_VERSION = "11.21-alpine";
+  private static final String TARGET_VERSION = "17-alpine";

  /**
   * Returns the docker image of the targeted Postgresql server version.
@@ -257,11 +257,11 @@ td.section {
   <tbody>
    <tr>
     <td class="property_name">generated by</td>
-     <td class="property_value">SchemaCrawler 16.23.2</td>
+     <td class="property_value">SchemaCrawler 16.25.2</td>
    </tr>
    <tr>
     <td class="property_name">generated on</td>
-     <td class="property_value">2024-12-09 22:31:09</td>
+     <td class="property_value">2025-02-16 20:10:13</td>
    </tr>
    <tr>
     <td class="property_name">last flyway file</td>
@@ -278,9 +278,9 @@ td.section {
    </title>
    <polygon fill="white" stroke="transparent" points="-4,4 -4,-3577.64 4860,-3577.64 4860,4 -4,4" />
    <text text-anchor="start" x="4616" y="-29.8" font-family="Helvetica,sans-Serif" font-size="14.00">generated by</text>
-    <text text-anchor="start" x="4699" y="-29.8" font-family="Helvetica,sans-Serif" font-size="14.00">SchemaCrawler 16.23.2</text>
+    <text text-anchor="start" x="4699" y="-29.8" font-family="Helvetica,sans-Serif" font-size="14.00">SchemaCrawler 16.25.2</text>
    <text text-anchor="start" x="4615" y="-10.8" font-family="Helvetica,sans-Serif" font-size="14.00">generated on</text>
-    <text text-anchor="start" x="4699" y="-10.8" font-family="Helvetica,sans-Serif" font-size="14.00">2024-12-09 22:31:09</text>
+    <text text-anchor="start" x="4699" y="-10.8" font-family="Helvetica,sans-Serif" font-size="14.00">2025-02-16 20:10:13</text>
    <polygon fill="none" stroke="#888888" points="4612,-4 4612,-44 4848,-44 4848,-4 4612,-4" /> <!-- allocationtoken_a08ccbef -->
    <g id="node1" class="node">
     <title>
@@ -2716,6 +2716,11 @@ td.section {
     <td class="minwidth">recurrence_last_expansion</td>
     <td class="minwidth">timestamptz not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default '2021-05-31 20:00:00-04'::timestamp with time zone</td>
+    </tr>
    <tr>
     <td colspan="3"></td>
    </tr>
@@ -2825,6 +2830,11 @@ td.section {
     <td class="minwidth"><b><i>job_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"BsaDomainRefresh_job_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -2861,6 +2871,11 @@ td.section {
     <td class="minwidth"><b><i>job_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"BsaDownload_job_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -3069,6 +3084,11 @@ td.section {
     <td class="minwidth"><b><i>revision_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"ClaimsList_revision_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -3639,6 +3659,11 @@ td.section {
     <td class="minwidth"><b><i>id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"DnsRefreshRequest_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -3785,6 +3810,11 @@ td.section {
     <td class="minwidth">lordn_phase</td>
     <td class="minwidth">text not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default 'NONE'::text</td>
+    </tr>
    <tr>
     <td colspan="3"></td>
    </tr>
@@ -4334,6 +4364,11 @@ td.section {
     <td class="minwidth"><b><i>id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"DomainTransactionRecord_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -4895,6 +4930,11 @@ td.section {
     <td class="minwidth"><b><i>package_promotion_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"Package_promotion_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -5188,6 +5228,11 @@ td.section {
     <td class="minwidth"><b><i>revision_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"PremiumList_revision_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -5873,6 +5918,11 @@ td.section {
     <td class="minwidth"><b><i>revision_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"RegistryLock_revision_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -6006,6 +6056,11 @@ td.section {
     <td class="minwidth"><b><i>revision_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"ReservedList_revision_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -6065,6 +6120,11 @@ td.section {
     <td class="minwidth"><b><i>id</i></b></td>
     <td class="minwidth">int8 not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default 1</td>
+    </tr>
    <tr>
     <td colspan="3"></td>
    </tr>
@@ -6155,6 +6215,11 @@ td.section {
     <td class="minwidth"><b><i>revision_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"SignedMarkRevocationList_revision_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -6209,6 +6274,11 @@ td.section {
     <td class="minwidth"><b><i>id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"SafeBrowsingThreat_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -6321,6 +6391,11 @@ td.section {
     <td class="minwidth"><b><i>id</i></b></td>
     <td class="minwidth">int8 not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default 1</td>
+    </tr>
    <tr>
     <td colspan="3"></td>
    </tr>
@@ -257,11 +257,11 @@ td.section {
   <tbody>
    <tr>
     <td class="property_name">generated by</td>
-     <td class="property_value">SchemaCrawler 16.23.2</td>
+     <td class="property_value">SchemaCrawler 16.25.2</td>
    </tr>
    <tr>
     <td class="property_name">generated on</td>
-     <td class="property_value">2024-12-09 22:31:07</td>
+     <td class="property_value">2025-02-16 20:10:10</td>
    </tr>
    <tr>
     <td class="property_name">last flyway file</td>
@@ -278,9 +278,9 @@ td.section {
    </title>
    <polygon fill="white" stroke="transparent" points="-4,4 -4,-7933.5 5565,-7933.5 5565,4 -4,4" />
    <text text-anchor="start" x="5321" y="-29.8" font-family="Helvetica,sans-Serif" font-size="14.00">generated by</text>
-    <text text-anchor="start" x="5404" y="-29.8" font-family="Helvetica,sans-Serif" font-size="14.00">SchemaCrawler 16.23.2</text>
+    <text text-anchor="start" x="5404" y="-29.8" font-family="Helvetica,sans-Serif" font-size="14.00">SchemaCrawler 16.25.2</text>
    <text text-anchor="start" x="5320" y="-10.8" font-family="Helvetica,sans-Serif" font-size="14.00">generated on</text>
-    <text text-anchor="start" x="5404" y="-10.8" font-family="Helvetica,sans-Serif" font-size="14.00">2024-12-09 22:31:07</text>
+    <text text-anchor="start" x="5404" y="-10.8" font-family="Helvetica,sans-Serif" font-size="14.00">2025-02-16 20:10:10</text>
    <polygon fill="none" stroke="#888888" points="5317,-4 5317,-44 5553,-44 5553,-4 5317,-4" /> <!-- allocationtoken_a08ccbef -->
    <g id="node1" class="node">
     <title>
@@ -3986,11 +3986,21 @@ td.section {
     <td class="minwidth">renewal_price_behavior</td>
     <td class="minwidth">text not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default 'DEFAULT'::text</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth">registration_behavior</td>
     <td class="minwidth">text not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default 'DEFAULT'::text</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth">allowed_epp_actions</td>
@@ -4755,6 +4765,11 @@ td.section {
     <td class="minwidth">renewal_price_behavior</td>
     <td class="minwidth">text not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default 'DEFAULT'::text</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth">renewal_price_currency</td>
@@ -4770,6 +4785,11 @@ td.section {
     <td class="minwidth">recurrence_last_expansion</td>
     <td class="minwidth">timestamptz not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default '2021-05-31 20:00:00-04'::timestamp with time zone</td>
+    </tr>
    <tr>
     <td colspan="3"></td>
    </tr>
@@ -4986,6 +5006,11 @@ td.section {
     <td class="minwidth"><b><i>job_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"BsaDomainRefresh_job_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -5055,6 +5080,11 @@ td.section {
     <td class="minwidth"><b><i>job_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"BsaDownload_job_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -5392,6 +5422,11 @@ td.section {
     <td class="minwidth"><b><i>revision_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"ClaimsList_revision_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -6866,6 +6901,11 @@ td.section {
     <td class="minwidth"><b><i>id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"DnsRefreshRequest_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -7224,6 +7264,11 @@ td.section {
     <td class="minwidth">lordn_phase</td>
     <td class="minwidth">text not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default 'NONE'::text</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth">last_update_time_via_epp</td>
@@ -8154,6 +8199,11 @@ td.section {
     <td class="minwidth">lordn_phase</td>
     <td class="minwidth">text not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default 'NONE'::text</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth">last_update_time_via_epp</td>
@@ -8494,6 +8544,11 @@ td.section {
     <td class="minwidth"><b><i>id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"DomainTransactionRecord_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -9622,6 +9677,11 @@ td.section {
     <td class="minwidth"><b><i>package_promotion_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"Package_promotion_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -10154,6 +10214,11 @@ td.section {
     <td class="minwidth"><b><i>revision_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"PremiumList_revision_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -11688,6 +11753,11 @@ td.section {
     <td class="minwidth"><b><i>revision_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"RegistryLock_revision_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -11958,6 +12028,11 @@ td.section {
     <td class="minwidth"><b><i>revision_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"ReservedList_revision_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -12057,6 +12132,11 @@ td.section {
     <td class="minwidth"><b><i>id</i></b></td>
     <td class="minwidth">int8 not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default 1</td>
+    </tr>
    <tr>
     <td colspan="3"></td>
    </tr>
@@ -12193,6 +12273,11 @@ td.section {
     <td class="minwidth"><b><i>revision_id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"SignedMarkRevocationList_revision_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -12270,6 +12355,11 @@ td.section {
     <td class="minwidth"><b><i>id</i></b></td>
     <td class="minwidth">bigserial not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default nextval('"SafeBrowsingThreat_id_seq"'::regclass)</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth"></td>
@@ -12595,6 +12685,11 @@ td.section {
     <td class="minwidth">breakglass_mode</td>
     <td class="minwidth">bool not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default false</td>
+    </tr>
    <tr>
     <td class="spacer"></td>
     <td class="minwidth">bsa_enroll_start_time</td>
@@ -12699,6 +12794,11 @@ td.section {
     <td class="minwidth"><b><i>id</i></b></td>
     <td class="minwidth">int8 not null</td>
    </tr>
+    <tr>
+     <td class="spacer"></td>
+     <td class="minwidth"></td>
+     <td class="minwidth">default 1</td>
+    </tr>
    <tr>
     <td colspan="3"></td>
    </tr>
@@ -2,12 +2,13 @@
 -- PostgreSQL database dump
 --

-- Dumped from database version 11.21
-- Dumped by pg_dump version 11.21
+-- Dumped from database version 17.4
+-- Dumped by pg_dump version 17.4

 SET statement_timeout = 0;
 SET lock_timeout = 0;
 SET idle_in_transaction_session_timeout = 0;
+SET transaction_timeout = 0;
 SET client_encoding = 'UTF8';
 SET standard_conforming_strings = on;
 SELECT pg_catalog.set_config('search_path', '', false);
@@ -32,7 +33,7 @@ COMMENT ON EXTENSION hstore IS 'data type for storing sets of (key, value) pairs

 SET default_tablespace = '';

-SET default_with_oids = false;
+SET default_table_access_method = heap;

 --
 -- Name: AllocationToken; Type: TABLE; Schema: public; Owner: -
@@ -115,6 +115,7 @@ class SchemaTest {
            Joiner.on(File.separatorChar).join(MOUNTED_RESOURCE_PATH, DUMP_OUTPUT_FILE));

    assertThat(dumpedSchema)
+        .ignoringLinesStartingWith("--")
        .hasSameContentAs(Resources.getResource("sql/schema/nomulus.golden.sql"));
  }

@@ -111,7 +111,7 @@ fi
 if [[ "${SUT}" = "nomulus" ]]; then
  DEPLOYED_SYSTEM="sql"
 elif [[ "${SUT}" = "sql" ]]; then
-  DEPLOYED_SYSTEM="nomulus"
+  DEPLOYED_SYSTEM="nomulus-gke"
 else
  echo "${USAGE}"
  exit 1
@@ -37,15 +37,19 @@ for service in frontend backend pubapi console
 do
  sed s/GCP_PROJECT/"${project}"/g "./kubernetes/nomulus-${service}.yaml" | \
  sed s/ENVIRONMENT/"${environment}"/g | \
+  sed s/PROXY_ENV/"${environment}"/g | \
+  sed s/EPP/"epp"/g | \
  kubectl apply -f -
+  kubectl rollout restart deployment/${service}
  # canary
  sed s/GCP_PROJECT/"${project}"/g "./kubernetes/nomulus-${service}.yaml" | \
  sed s/ENVIRONMENT/"${environment}"/g | \
+  sed s/PROXY_ENV/"${environment}_canary"/g | \
+  sed s/EPP/"epp-canary"/g | \
  sed s/"${service}"/"${service}-canary"/g | \
  kubectl apply -f -
+  kubectl rollout restart deployment/${service}-canary
 done
-# Kills all running pods, new pods created will be pulling the new image.
-kubectl delete pods --all
 kubectl apply -f "./kubernetes/gateway/nomulus-gateway.yaml"
 kubectl apply -f "./kubernetes/gateway/nomulus-iap-${environment}.yaml"
 for service in frontend backend console pubapi
@@ -57,4 +61,16 @@ do
  sed s/SERVICE/"${service}-canary"/g "./kubernetes/gateway/nomulus-backend-policy-${environment}.yaml" | \
  kubectl apply -f -
 done
+
+# Restart proxies
+while read line
+do
+  parts=(${line})
+  echo "Updating cluster ${parts[0]} in location ${parts[1]}..."
+  gcloud container clusters get-credentials ${parts[0]} \
+    --project ${project} --location ${parts[1]}
+  kubectl rollout restart deployment/proxy-deployment
+  kubectl rollout restart deployment/proxy-deployment-canary
+done < <(gcloud container clusters list --project ${project} | grep proxy-cluster)
+
 kubectl config use-context "$current_context"
@@ -1,158 +0,0 @@
-#! /bin/env python3
-# Copyright 2024 The Nomulus Authors. All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-'''
-A script that outputs the IP endpoints of various load balancers, to be run
-after Nomulus is deployed.
-'''
-
-import ipaddress
-import json
-import subprocess
-import sys
-from dataclasses import dataclass
-from ipaddress import IPv4Address
-from ipaddress import IPv6Address
-from operator import attrgetter
-from operator import methodcaller
-
-
-class PreserveContext:
-    def __enter__(self):
-        self._context = run_command('kubectl config current-context')
-
-    def __exit__(self, type, value, traceback):
-        run_command('kubectl config use-context ' + self._context)
-
-
-class UseCluster(PreserveContext):
-    def __init__(self, cluster: str, region: str, project: str):
-        self._cluster = cluster
-        self._region = region
-        self._project = project
-
-    def __enter__(self):
-        super().__enter__()
-        cmd = (f'gcloud container fleet memberships get-credentials'
-                   f' {self._cluster} --project {self._project}')
-        run_command(cmd)
-
-
-def run_command(cmd: str, print_output=False) -> str:
-    proc = subprocess.run(cmd, text=True, shell=True, stdout=subprocess.PIPE,
-                          stderr=subprocess.STDOUT)
-    if print_output:
-        print(proc.stdout)
-    return proc.stdout
-
-
-def get_clusters(project: str) -> dict[str, str]:
-    cmd = f'gcloud container clusters list --project {project} --format=json'
-    content = json.loads(run_command(cmd))
-    res = {}
-    for item in content:
-        name = item['name']
-        region = item['location']
-        if not name.startswith('nomulus-cluster'):
-            continue
-        res[name] = region
-    return res
-
-
-def get_endpoints(resource: str, service: str, jsonpath: str) -> list[
-    str]:
-    content = run_command(
-            f'kubectl get {resource}/{service} -o jsonpath={jsonpath}', )
-    return content.split()
-
-
-def get_region_symbol(region: str) -> str:
-    if region.startswith('us'):
-        return 'amer'
-    if region.startswith('europe'):
-        return 'emea'
-    if region.startswith('asia'):
-        return 'apac'
-    return 'other'
-
-
-@dataclass
-class IP:
-    service: str
-    region: str
-    address: IPv4Address | IPv6Address
-
-    def is_ipv6(self) -> bool:
-        return self.address.version == 6
-
-    def __str__(self) -> str:
-        return f'{self.service} {self.region}: {self.address}'
-
-
-def terraform_str(item) -> str:
-    res = ""
-    if (isinstance(item, dict)):
-        res += '{\n'
-        for key, value in item.items():
-            res += f'{key} = {terraform_str(value)}\n'
-        res += '}'
-    elif (isinstance(item, list)):
-        res += '['
-        for i, value in enumerate(item):
-            if i != 0:
-                res += ', '
-            res += terraform_str(value)
-        res += ']'
-    else:
-        res += f'"{item}"'
-    return res
-
-
-if __name__ == '__main__':
-    if len(sys.argv) != 2:
-        raise ValueError('Usage: get-endpoints.py <project>')
-    project = sys.argv[1]
-    print(f'Project: {project}')
-    clusters = get_clusters(project)
-    ips = []
-    res = {}
-    for cluster, region in clusters.items():
-        with UseCluster(cluster, region, project):
-            for service in ['whois', 'whois-canary', 'epp', 'epp-canary']:
-                map_key = service.replace('-', '_')
-                for ip in get_endpoints('services', service,
-                                        '{.status.loadBalancer.ingress[*].ip}'):
-                    ip = ipaddress.ip_address(ip)
-                    if isinstance(ip, IPv4Address):
-                        map_key_with_iptype = map_key + '_ipv4'
-                    else:
-                        map_key_with_iptype = map_key + '_ipv6'
-                    if map_key_with_iptype not in res:
-                        res[map_key_with_iptype] = {}
-                    res[map_key_with_iptype][get_region_symbol(region)] = [ip]
-                    ips.append(IP(service, get_region_symbol(region), ip))
-            if not region.startswith('us'):
-                continue
-            ip = get_endpoints('gateways.gateway.networking.k8s.io', 'nomulus',
-                               '{.status.addresses[*].value}')[0]
-            print(f'nomulus: {ip}')
-            res['https_ip'] = ipaddress.ip_address(ip)
-    ips.sort(key=attrgetter('region'))
-    ips.sort(key=methodcaller('is_ipv6'))
-    ips.sort(key=attrgetter('service'))
-    for ip in ips:
-        print(ip)
-    print("Terraform friendly output:")
-    print(terraform_str(res))
@@ -4,7 +4,18 @@ metadata:
  name: nomulus
 spec:
  gatewayClassName: gke-l7-global-external-managed-mc
+  addresses:
+  - type: NamedAddress
+    value: nomulus-ipv4-address
+  - type: NamedAddress
+    value: nomulus-ipv6-address
  listeners:
+  - name: http
+    protocol: HTTP
+    port: 80
+    allowedRoutes:
+      kinds:
+      - kind: HTTPRoute
  - name: https
    protocol: HTTPS
    port: 443
@@ -15,3 +26,19 @@ spec:
    allowedRoutes:
      kinds:
      - kind: HTTPRoute
+---
+apiVersion: gateway.networking.k8s.io/v1beta1
+kind: HTTPRoute
+metadata:
+  name: redirect
+spec:
+  parentRefs:
+  - kind: Gateway
+    name: nomulus
+    sectionName: http
+  rules:
+  - filters:
+    - type: RequestRedirect
+      requestRedirect:
+        scheme: https
+
@@ -6,6 +6,7 @@ spec:
  parentRefs:
  - kind: Gateway
    name: nomulus
+    sectionName: https
  hostnames:
    - "backend.BASE_DOMAIN"
  rules:
@@ -6,6 +6,7 @@ spec:
  parentRefs:
  - kind: Gateway
    name: nomulus
+    sectionName: https
  hostnames:
    - "console.BASE_DOMAIN"
  rules:
@@ -6,6 +6,7 @@ spec:
  parentRefs:
  - kind: Gateway
    name: nomulus
+    sectionName: https
  hostnames:
    - "frontend.BASE_DOMAIN"
  rules:
@@ -6,6 +6,7 @@ spec:
  parentRefs:
  - kind: Gateway
    name: nomulus
+    sectionName: https
  hostnames:
    - "pubapi.BASE_DOMAIN"
  rules:
@@ -2,6 +2,8 @@ apiVersion: apps/v1
 kind: Deployment
 metadata:
  name: backend
+  annotations:
+    tag: "latest"
 spec:
  selector:
    matchLabels:
@@ -12,6 +14,8 @@ spec:
        service: backend
    spec:
      serviceAccountName: nomulus
+      nodeSelector:
+        cloud.google.com/compute-class: "Performance"
      containers:
      - name: backend
        image: gcr.io/GCP_PROJECT/nomulus
@@ -20,7 +24,8 @@ spec:
          name: http
        resources:
          requests:
-            cpu: "500m"
+            cpu: "100m"
+            memory: "512Mi"
        args: [ENVIRONMENT]
        env:
        - name: POD_ID
@@ -2,6 +2,8 @@ apiVersion: apps/v1
 kind: Deployment
 metadata:
  name: console
+  annotations:
+    tag: "latest"
 spec:
  selector:
    matchLabels:
@@ -12,6 +14,8 @@ spec:
        service: console
    spec:
      serviceAccountName: nomulus
+      nodeSelector:
+        cloud.google.com/compute-class: "Performance"
      containers:
      - name: console
        image: gcr.io/GCP_PROJECT/nomulus
@@ -20,7 +24,8 @@ spec:
          name: http
        resources:
          requests:
-            cpu: "500m"
+            cpu: "100m"
+            memory: "512Mi"
        args: [ENVIRONMENT]
        env:
        - name: POD_ID
@@ -2,6 +2,8 @@ apiVersion: apps/v1
 kind: Deployment
 metadata:
  name: frontend
+  annotations:
+    tag: "latest"
 spec:
  selector:
    matchLabels:
@@ -12,6 +14,8 @@ spec:
        service: frontend
    spec:
      serviceAccountName: nomulus
+      nodeSelector:
+        cloud.google.com/compute-class: "Performance"
      containers:
      - name: frontend
        image: gcr.io/GCP_PROJECT/nomulus
@@ -20,7 +24,8 @@ spec:
          name: http
        resources:
          requests:
-            cpu: "500m"
+            cpu: "100m"
+            memory: "512Mi"
        args: [ENVIRONMENT]
        env:
        - name: POD_ID
@@ -37,6 +42,27 @@ spec:
              fieldPath: metadata.namespace
        - name: CONTAINER_NAME
          value: frontend
+      - name: EPP
+        image: gcr.io/GCP_PROJECT/proxy
+        ports:
+        - containerPort: 30002
+          name: epp
+        resources:
+          requests:
+            cpu: "100m"
+            memory: "512Mi"
+        args: [--env, PROXY_ENV, --log, --local]
+        env:
+        - name: POD_ID
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.name
+        - name: NAMESPACE_ID
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.namespace
+        - name: CONTAINER_NAME
+          value: EPP
 ---
 # Only need to define the service account once per cluster.
 apiVersion: v1
@@ -55,7 +81,7 @@ spec:
    apiVersion: apps/v1
    kind: Deployment
    name: frontend
-  minReplicas: 5
+  minReplicas: 15
  maxReplicas: 15
  metrics:
    - type: Resource
@@ -77,6 +103,27 @@ spec:
      targetPort: http
      name: http
 ---
+apiVersion: v1
+kind: Service
+metadata:
+  name: EPP
+  annotations:
+    cloud.google.com/l4-rbs: enabled
+    networking.gke.io/weighted-load-balancing: pods-per-node
+    networking.gke.io/load-balancer-ip-addresses: "EPP-ipv6-main,EPP-ipv4-main"
+spec:
+  type: LoadBalancer
+  # Traffic is directly delivered to a node, preserving the original source IP.
+  externalTrafficPolicy: Local
+  ipFamilies: [IPv4, IPv6]
+  ipFamilyPolicy: RequireDualStack
+  selector:
+    service: frontend
+  ports:
+  - port: 700
+    targetPort: epp
+    name: epp
+---
 apiVersion: net.gke.io/v1
 kind: ServiceExport
 metadata:
@@ -2,6 +2,8 @@ apiVersion: apps/v1
 kind: Deployment
 metadata:
  name: pubapi
+  annotations:
+    tag: "latest"
 spec:
  selector:
    matchLabels:
@@ -12,6 +14,8 @@ spec:
        service: pubapi
    spec:
      serviceAccountName: nomulus
+      nodeSelector:
+        cloud.google.com/compute-class: "Performance"
      containers:
      - name: pubapi
        image: gcr.io/GCP_PROJECT/nomulus
@@ -20,7 +24,8 @@ spec:
          name: http
        resources:
          requests:
-            cpu: "500m"
+            cpu: "100m"
+            memory: "512Mi"
        args: [ENVIRONMENT]
        env:
        - name: POD_ID
@@ -0,0 +1,9 @@
+# ---------------------------------------
+# Module: gzip
+# Enables gzip compression of the following mime types
+# ---------------------------------------
+
+--module=gzip
+
+jetty.http.gzip.minGzipSize=200
+jetty.http.gzip.includedMimeTypes=text/html,text/css,text/javascript,application/javascript,application/json,image/svg+xml
@@ -40,7 +40,8 @@ do
    kubectl apply -f -
    kubectl apply -f "./kubernetes/proxy-service-canary.yaml" --force
  fi
-  # Kills all running pods, new pods created will be pulling the new image.
-  kubectl delete pods --all
+  # Restart all running pods, new pods created will be pulling the new image.
+  kubectl rollout restart deployment/proxy-deployment
+  kubectl rollout restart deployment/proxy-deployment-canary
 done < <(gcloud container clusters list --project ${project} | grep proxy-cluster)
 kubectl config use-context "$current_context"
@@ -15,8 +15,10 @@
 package google.registry.proxy.handler;

 import static com.google.common.base.Preconditions.checkArgument;
+import static google.registry.proxy.handler.ProxyProtocolHandler.REMOTE_ADDRESS_KEY;

 import google.registry.proxy.metric.FrontendMetrics;
+import google.registry.util.ProxyHttpHeaders;
 import io.netty.buffer.ByteBuf;
 import io.netty.channel.ChannelFutureListener;
 import io.netty.channel.ChannelHandlerContext;
@@ -30,6 +32,8 @@ import java.util.function.Supplier;
 /** Handler that processes WHOIS protocol logic. */
 public final class WhoisServiceHandler extends HttpsRelayServiceHandler {

+  private String clientAddress;
+
  public WhoisServiceHandler(
      String relayHost,
      String relayPath,
@@ -45,6 +49,12 @@ public final class WhoisServiceHandler extends HttpsRelayServiceHandler {
    super.channelActive(ctx);
  }

+  @Override
+  public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
+    clientAddress = ctx.channel().attr(REMOTE_ADDRESS_KEY).get();
+    super.channelRead(ctx, msg);
+  }
+
  @Override
  protected FullHttpRequest decodeFullHttpRequest(ByteBuf byteBuf) {
    FullHttpRequest request = super.decodeFullHttpRequest(byteBuf);
@@ -52,6 +62,12 @@ public final class WhoisServiceHandler extends HttpsRelayServiceHandler {
        .headers()
        .set(HttpHeaderNames.CONTENT_TYPE, HttpHeaderValues.TEXT_PLAIN)
        .set(HttpHeaderNames.ACCEPT, HttpHeaderValues.TEXT_PLAIN);
+    if (clientAddress != null) {
+      request
+          .headers()
+          .set(ProxyHttpHeaders.IP_ADDRESS, clientAddress)
+          .set(ProxyHttpHeaders.FALLBACK_IP_ADDRESS, clientAddress);
+    }
    return request;
  }

@@ -28,7 +28,7 @@ COPY go.sum ./
 COPY go.mod ./
 RUN go build -o /deployCloudSchedulerAndQueue

-FROM marketplace.gcr.io/google/ubuntu2204
+FROM marketplace.gcr.io/google/ubuntu2404
 ENV DEBIAN_FRONTEND=noninteractive LANG=en_US.UTF-8
 # Add script for cloud scheduler and cloud tasks deployment
 COPY --from=deployCloudSchedulerAndQueueBuilder /deployCloudSchedulerAndQueue /usr/local/bin/deployCloudSchedulerAndQueue
@@ -43,10 +43,6 @@ apt-get install openjdk-21-jdk-headless -y

 # Install Python
 apt-get install python3 -y
-# As of March 2021 python3 is at v3.6. Get pip then install dataclasses
-# (introduced in 3.7) for nom_build
-apt-get install python3-pip -y
-python3 -m pip install dataclasses

 # Install Node
 apt-get install npm -y
@@ -57,11 +53,13 @@ npm install -g n
 for i in {1..5}; do n 22.7.0 && break || sleep 15; done

 # Install gp_dump
-apt-get install postgresql-client-11 procps -y
+apt-get install postgresql-client-17 procps -y

 # Install gcloud
 apt-get install google-cloud-cli -y
 apt-get install google-cloud-sdk-app-engine-java -y
+apt-get install kubectl -y
+apt-get install google-cloud-cli-gke-gcloud-auth-plugin -y

 # Install git
 apt-get install git -y
@@ -198,6 +198,7 @@ artifacts:
    - 'release/cloudbuild-delete-*.yaml'
    - 'release/cloudbuild-schema-deploy-*.yaml'
    - 'release/cloudbuild-schema-verify-*.yaml'
+    - 'release/cloudbuild-restart-proxies-*.yaml'
    - 'jetty/kubernetes/*.yaml'
    - 'jetty/kubernetes/gateway/*.yaml'
 # The images are already uploaded, but we still need to include them there so that
@@ -88,6 +88,7 @@ steps:
    sed -i s/builder:latest/builder@$builder_digest/g release/cloudbuild-schema-deploy.yaml
    sed -i s/builder:latest/builder@$builder_digest/g release/cloudbuild-schema-verify.yaml
    sed -i s/builder:latest/builder@$builder_digest/g release/cloudbuild-delete.yaml
+    sed -i s/builder:latest/builder@$builder_digest/g release/cloudbuild-restart-proxies.yaml
    sed -i s/GCP_PROJECT/${PROJECT_ID}/ proxy/kubernetes/proxy-*.yaml
    sed -i s/'$${TAG_NAME}'/${TAG_NAME}/g release/cloudbuild-sync-and-tag.yaml
    sed -i s/'$${TAG_NAME}'/${TAG_NAME}/g release/cloudbuild-deploy.yaml
@@ -99,6 +100,11 @@ steps:
        > release/cloudbuild-deploy-gke-${environment}.yaml
      sed s/'$${_ENV}'/${environment}/g release/cloudbuild-delete.yaml \
        > release/cloudbuild-delete-${environment}.yaml
+      sed s/'$${_ENV}'/${environment}/g release/cloudbuild-restart-proxies.yaml \
+        > release/cloudbuild-restart-proxies-${environment}.yaml
+      sed s/'$${_ENV}'/${environment}/g release/cloudbuild-restart-proxies.yaml | \
+        sed s/proxy-deployment/proxy-deployment-canary/g \
+        > release/cloudbuild-restart-proxies-${environment}-canary.yaml
    done
 # Build and upload the schema_deployer image.
 - name: 'gcr.io/cloud-builders/docker'
@@ -178,11 +184,14 @@ steps:
      base_domain=$(grep baseDomain \
        ./core/src/main/java/google/registry/config/files/nomulus-config-${env}.yaml | \
        awk '{print $2}')
-      for service in frontend backend pubapi console 
+      for service in frontend backend pubapi console
      do
        # non-canary
        sed s/GCP_PROJECT/${PROJECT_ID}/g ./jetty/kubernetes/nomulus-${service}.yaml | \
-          sed s/ENVIRONMENT/${env}/g > ./jetty/kubernetes/nomulus-${env}-${service}.yaml
+          sed s/latest/${TAG_NAME}/g | \
+          sed s/ENVIRONMENT/${env}/g | \
+          sed s/PROXY_ENV/"${env}"/g | \
+          sed s/EPP/"epp"/g > ./jetty/kubernetes/nomulus-${env}-${service}.yaml
        # Proxy '--log' flag does not work on production.
        if [ ${env} == production ]
        then
@@ -195,7 +204,10 @@ steps:
        fi
        # canary
        sed s/GCP_PROJECT/${PROJECT_ID}/g ./jetty/kubernetes/nomulus-${service}.yaml | \
+          sed s/latest/${TAG_NAME}/g | \
          sed s/ENVIRONMENT/${env}/g | \
+          sed s/PROXY_ENV/"${env}_canary"/g | \
+          sed s/EPP/"epp-canary"/g | \
          sed s/${service}/${service}-canary/g \
          > ./jetty/kubernetes/nomulus-${env}-${service}-canary.yaml
        # Proxy '--log' flag does not work on production.
@@ -264,7 +276,7 @@ steps:
      $(gcloud auth list --format='get(account)' --filter=active)
    git add .
    git commit -m "Release commit for tag ${TAG_NAME}"
-    git push -o nokeycheck origin master 
+    git push -o nokeycheck origin master
    git tag ${TAG_NAME}
    git push -o nokeycheck origin ${TAG_NAME}
 timeout: 3600s
@@ -0,0 +1,64 @@
+# This will do rolling restarts of all proxies. This forces the client to reconnect
+# and resets the sessions.
+#
+# To manually trigger a build on GCB, run:
+# gcloud builds submit --config=cloudbuild-restart-proxies.yaml \
+# --substitutions=_ENV=[ENV] ..
+#
+# To trigger a build automatically, follow the instructions below and add a trigger:
+# https://cloud.google.com/cloud-build/docs/running-builds/automate-builds
+#
+# Note: to work around the issue in Spinnaker's 'Deployment Manifest' stage,
+# variable references must avoid the ${var} format. Valid formats include
+# $var or ${"${var}"}. This file uses the former. Since TAG_NAME and _ENV are
+# expanded in the copies sent to Spinnaker, we preserve the brackets around
+# them for safe pattern matching during release.
+# See https://github.com/spinnaker/spinnaker/issues/3028 for more information.
+steps:
+# Pull the credential for nomulus tool.
+- name: 'gcr.io/$PROJECT_ID/builder:latest'
+  entrypoint: /bin/bash
+  args:
+  - -c
+  - |
+    set -e
+    gcloud secrets versions access latest \
+      --secret nomulus-tool-cloudbuild-credential > tool-credential.json
+# Do rolling restarts of all proxies in all environments.
+- name: 'gcr.io/$PROJECT_ID/builder:latest'
+  entrypoint: /bin/bash
+  args:
+  - -c
+  - |
+    if [ ${_ENV} == production ]
+    then
+      project_id="domain-registry"
+    else
+      project_id="domain-registry-${_ENV}"
+    fi
+
+    gcloud auth activate-service-account --key-file=tool-credential.json
+
+    first=true
+    t=0
+
+    while read line
+    do
+      # Sleep for t seconds for the rollout to stabilize.
+      if [[ -v first ]]
+      then
+        unset first
+      else
+        sleep $t
+      fi
+      name=$(echo $line | awk '{print $1}')
+      location=$(echo $line | awk '{print $2}')
+      echo $name $region
+      echo "Updating cluster $name in location $location..."
+      gcloud container clusters get-credentials $name \
+        --project $project_id --location $location
+      kubectl rollout restart deployment/proxy-deployment
+    done < <(gcloud container clusters list --project $project_id | grep proxy-cluster)
+timeout: 7500s
+options:
+  machineType: 'N1_HIGHCPU_8'
@@ -3,3 +3,6 @@ COMMENT ON EXTENSION pgaudit IS 'provides auditing functionality';
 SET default_with_oids = false;
 CREATE EXTENSION IF NOT EXISTS pg_stat_statements WITH SCHEMA public;
 COMMENT ON EXTENSION pg_stat_statements IS 'track execution statistics of all SQL statements executed';
+SET transaction_timeout = 0;
+SET default_table_access_method = heap;
+SET default_with_oids = false;
@@ -69,6 +69,14 @@ PGPASSWORD=${db_password} pg_dump -h localhost -U "${db_user}" \
    --exclude-table flyway_schema_history \
    postgres

+if [ $? -ne 0 ]; then
+  echo "Failed to dump schema."
+  exit 1
+else
+  echo "Schema dumped."
+fi
+
+
 raw_diff=$(diff /schema/nomulus.golden.sql /schema/nomulus.actual.sql)
 # Clean up the raw_diff:
 # - Remove diff locations (e.g. "5,6c5,6): grep "^[<>]"
@@ -20,6 +20,7 @@ import com.google.common.flogger.backend.system.SimpleLogRecord;
 import com.google.gson.Gson;
 import java.io.PrintWriter;
 import java.io.StringWriter;
+import java.util.HashMap;
 import java.util.LinkedHashMap;
 import java.util.Map;
 import java.util.Optional;
@@ -58,6 +59,13 @@ public class GcpJsonFormatter extends Formatter {
  /** JSON field that stores the parameters of the current request, if any. */
  private static final String HTTP_REQUEST = "httRequest";

+  /**
+   * JSON field that stores the arbitrary labels.
+   *
+   * <p>In practice, we store session cookies so we can identify requests from the same session.
+   */
+  private static final String LABELS = "logging.googleapis.com/labels";
+
  /** JSON field that contains the content, this will show up as the main entry in a log. */
  private static final String MESSAGE = "message";

@@ -83,6 +91,9 @@ public class GcpJsonFormatter extends Formatter {

  private static final ThreadLocal<HttpRequest> request = new ThreadLocal<>();

+  private static final ThreadLocal<Map<String, String>> labels =
+      ThreadLocal.withInitial(HashMap::new);
+
  /**
   * Set the Trace ID associated with any logging done by the current thread.
   *
@@ -107,6 +118,19 @@ public class GcpJsonFormatter extends Formatter {
    request.set(new HttpRequest(requestMethod, requestUrl, userAgent, protocol));
  }

+  /**
+   * Set an arbitrary key-value pair as a label for the current request.
+   *
+   * <p>Existing labels will be overwritten.
+   */
+  public static void setLabel(String key, String value) {
+    labels.get().put(key, value);
+  }
+
+  public static void unsetLabels() {
+    labels.get().clear();
+  }
+
  public static void unsetCurrentRequest() {
    request.remove();
  }
@@ -180,6 +204,9 @@ public class GcpJsonFormatter extends Formatter {
    if (request.get() != null) {
      json.put(HTTP_REQUEST, request.get());
    }
+    if (!labels.get().isEmpty()) {
+      json.put(LABELS, labels.get());
+    }
    // This trailing newline is required for the proxy because otherwise multiple logs might be
    // sent to Stackdriver together (due to the async nature of the proxy), and not parsed
    // correctly.
@@ -72,6 +72,7 @@ class GcpJsonFormatterTest {
    jdkLogger.removeHandler(handler);
    GcpJsonFormatter.setCurrentTraceId(null);
    GcpJsonFormatter.unsetCurrentRequest();
+    GcpJsonFormatter.unsetLabels();
  }

  @Test
@@ -79,7 +80,7 @@ class GcpJsonFormatterTest {
    logger.atInfo().log("Something I have to say");
    handler.close();
    String output = ostream.toString(StandardCharsets.US_ASCII);
-    assertThat(output).isEqualTo(makeJson("INFO", 79, "testSuccess", "Something I have to say"));
+    assertThat(output).isEqualTo(makeJson("INFO", 80, "testSuccess", "Something I have to say"));
  }

  @Test
@@ -88,7 +89,7 @@ class GcpJsonFormatterTest {
    logger.atInfo().log("Something I have to say");
    handler.close();
    String output = ostream.toString(StandardCharsets.US_ASCII);
-    String expected = makeJson("INFO", 88, "testSuccess_traceId", "Something I have to say");
+    String expected = makeJson("INFO", 89, "testSuccess_traceId", "Something I have to say");
    // Remove the last two characters (}, \n) from the template and add the trace ID.
    expected =
        expected.substring(0, expected.length() - 2)
@@ -103,7 +104,7 @@ class GcpJsonFormatterTest {
    handler.close();
    String output = ostream.toString(StandardCharsets.US_ASCII);
    String expected =
-        makeJson("INFO", 102, "testSuccess_currentRequest", "Something I have to say");
+        makeJson("INFO", 103, "testSuccess_currentRequest", "Something I have to say");
    // Remove the last two characters (}, \n) from the template and add the request.
    expected =
        expected.substring(0, expected.length() - 2)
@@ -112,13 +113,28 @@ class GcpJsonFormatterTest {
    assertThat(output).isEqualTo(expected);
  }

+  @Test
+  void testSuccess_labels() {
+    GcpJsonFormatter.setLabel("label1", "value1");
+    GcpJsonFormatter.setLabel("label2", "value2");
+    logger.atInfo().log("Something I have to say");
+    handler.close();
+    String output = ostream.toString(StandardCharsets.US_ASCII);
+    String expected = makeJson("INFO", 120, "testSuccess_labels", "Something I have to say");
+    // Remove the last two characters (}, \n) from the template and add the labels.
+    expected =
+        expected.substring(0, expected.length() - 2)
+            + ",\"logging.googleapis.com/labels\":{\"label1\":\"value1\",\"label2\":\"value2\"}}\n";
+    assertThat(output).isEqualTo(expected);
+  }
+
  @Test
  void testSuccess_logLevel() {
    logger.atSevere().log("Something went terribly wrong");
    handler.close();
    String output = ostream.toString(StandardCharsets.US_ASCII);
    assertThat(output)
-        .isEqualTo(makeJson("ERROR", 117, "testSuccess_logLevel", "Something went terribly wrong"));
+        .isEqualTo(makeJson("ERROR", 133, "testSuccess_logLevel", "Something went terribly wrong"));
  }

  @Test
@@ -127,7 +143,7 @@ class GcpJsonFormatterTest {
    handler.close();
    String output = ostream.toString(StandardCharsets.US_ASCII);
    String prefix =
-        makeJson("ERROR", 126, "testSuccess_withCause", "Something went terribly wrong");
+        makeJson("ERROR", 142, "testSuccess_withCause", "Something went terribly wrong");
    // Remove the last two characters (}, \n) from the template as the actual output contains
    // the full stack trace.
    prefix = prefix.substring(0, prefix.length() - 2);
@@ -141,7 +157,7 @@ class GcpJsonFormatterTest {
    handler.close();
    String output = ostream.toString(StandardCharsets.US_ASCII);
    String prefix =
-        makeJson("ERROR", 140, "testSuccess_withStackTrace", "Something is worth checking");
+        makeJson("ERROR", 156, "testSuccess_withStackTrace", "Something is worth checking");
    // Remove the last three characters (}, \n) from the template as the actual output contains
    // the full stack trace.
    prefix = prefix.substring(0, prefix.length() - 2);
@@ -167,7 +183,7 @@ class GcpJsonFormatterTest {
    TimeLimiter.create().callWithTimeout(this::logSomething, 1, TimeUnit.SECONDS);
    handler.close();
    String output = ostream.toString(StandardCharsets.US_ASCII);
-    assertThat(output).isEqualTo(makeJson("INFO", 188, "logSomething", "Something I have to say"));
+    assertThat(output).isEqualTo(makeJson("INFO", 204, "logSomething", "Something I have to say"));
  }

  @Test
@@ -176,7 +192,7 @@ class GcpJsonFormatterTest {
    TimeLimiter.create().callWithTimeout(this::logSomething, 1, TimeUnit.SECONDS);
    handler.close();
    String output = ostream.toString(StandardCharsets.US_ASCII);
-    String expected = makeJson("INFO", 188, "logSomething", "Something I have to say");
+    String expected = makeJson("INFO", 204, "logSomething", "Something I have to say");
    // Remove the last two characters (}, \n) from the template and add the trace ID.
    expected =
        expected.substring(0, expected.length() - 2)
Author	SHA1	Message	Date
Lai Jiang	12fd206c35	Update README.md (#2705 )	2025-03-05 16:55:04 +00:00
Lai Jiang	a3f510d0db	Log session cookies in metadata (#2703 ) There are two session cookies, JSESSIONID, which is set by Jetty, and GCLB, which is set by the Gateway. In one session, every request other than the first one (the <hello>) should have the same GCLB value, and every request after a successful <login> should have the same JSESSIONID. With these two metadata, we should be able to trace all requests that should belong to the same session and debug issues with session mismatch (if any).	2025-03-04 20:18:15 +00:00
gbrodman	fa54c26ee2	Log transaction durations (#2682 ) There can be delays in releasing predicate locks when we have transactions that are long-lived -- even delays in releasing predicate locks acquired by shorter-lived transactions. Logging the transaction duration will allow us to get a sense as to transaction durations during busy times.	2025-03-04 13:15:15 +00:00
gbrodman	8896fb94f4	Use nomulus-gke tagging mechanism in sql-int tests (#2702 ) Had to temporarily create the files in gs://domain-registry-dev-deployed-tags but the automated release process will take care of that soon	2025-03-04 04:05:53 +00:00
Pavlo Tkach	6c7bf5e5dd	Enable Users and Domains actions, add email notification (#2700 )	2025-02-28 21:57:49 +00:00
Pavlo Tkach	ea1e8d5cc5	Add console gzip compression to js,css and html files (#2696 )	2025-02-27 22:52:10 +00:00
Lai Jiang	7fb846c5b0	Add headers to record WHOIS client IPs (#2695 ) The headers can be used by Cloud Armor to perform IP-based rate limiting.	2025-02-27 22:15:13 +00:00
Lai Jiang	5180095cb6	Reduce log level to info when no email is found from the OIDC token (#2694 ) This can happen on public endpoints (in pubapi) where the service is behind IAP but all users (including not-logged-in ones) are allowed. IAP will add an OIDC token with no email field in the request header.	2025-02-26 22:17:45 +00:00
Lai Jiang	9fe64bf9ec	Make ignoreLinesStartingWith varargs (#2691 ) It still is a list, because we String::startsWith does not benefit from the target being in a set.	2025-02-26 17:12:24 +00:00
Lai Jiang	0f3b62d5ce	Change the sleep time between proxy rollout (#2689 )	2025-02-26 04:48:52 +00:00
Ben McIlwain	bd4701647b	Refactor logic out of domain create flow tests (#2688 ) This removes logic from an inner helper method so that it becomes more clear from callsites within each test exactly which behavior is expected from those test conditions.	2025-02-25 19:54:56 +00:00
Lai Jiang	fb816d7a2c	Make it possible to ignore comment lines when comparing schemas (#2690 ) We now pin to postgreSQL v17 when running tests, which means that minor version might increase without our intervention. This causes (at least) the comment in the golden schema to change, and failing the test as a result. This PR adds the ability to strip lines that we deem as comment from the comparison, so we don't have to do trivial upgrades to the gold schema whenever there's minor version upgrade.	2025-02-25 16:58:26 +00:00
gbrodman	8fbf363195	Remove unused dummy PGP file (#2687 ) This was previously used as a dummy value for testing / compilation but it's not used any more.	2025-02-24 21:45:26 +00:00
Lai Jiang	397f800614	Connect to GKE by default from the tool (#2686 )	2025-02-24 19:01:05 +00:00
Lai Jiang	bcf42bd287	Use static IPs for EPP endpoints (#2685 ) These IPs are now provisioned by Terraform. Also delete the get-endpoints.py script as it is no longer necessary.	2025-02-24 16:38:47 +00:00
Pavlo Tkach	ed95d19b93	Provide prompt for user deletion UI (#2684 )	2025-02-21 20:30:03 +00:00
Lai Jiang	97fc2c0b66	Add an annotation to the deployment (#2683 ) This allows us to easily tell which tag was deployed. Also set the gateway to use named address so they are stable, and so that we can attach an IPv6 record to it. Auto-provisioned addresses are IPv4 only.	2025-02-21 16:30:32 +00:00
Weimin Yu	00728c40ba	Abort schema verifier when pg_dump fails (#2681 ) Failed pg_dump may not leave a file, failing the subsequent diffing and causing the verifier to return success. The verifier should abort in this case.	2025-02-20 17:35:47 +00:00
Lai Jiang	3f2a42ab8d	Expose EPP via saidcar proxy (#2680 )	2025-02-19 18:57:25 +00:00
Lai Jiang	b73e342820	Update PostgreSQL version in builder image and tests (#2667 )	2025-02-18 17:34:41 +00:00
Lai Jiang	df7fec7a3e	Update RDAP TOS link (#2678 )	2025-02-18 17:00:26 +00:00
Lai Jiang	6f7ae1eabc	Redirect HTTP to HTTPS (#2679 ) This opens up port 80 on the load balancer IP and upgrades all HTTP request to HTTPS. TESTED=tested on alpha.	2025-02-18 16:57:18 +00:00