Make small OIDC token javadoc update (#3142)

Just remove a few possibly-insecure logs

GEMINI.md

@@ -44,6 +44,11 @@ This document outlines foundational mandates, architectural patterns, and projec
         - **Test Helpers & Timestamps:** If a static test helper method (like in `DatabaseHelper`) needs the database transaction time but might be called from outside a transaction, using `tm().reTransact(tm()::getTxTime)` is acceptable. However, NEVER wrap it redundantly like `tm().transact(() -> tm().reTransact(tm()::getTxTime))`. If you are just setting an arbitrary timestamp in a test where the exact DB transaction time isn't strictly required, prefer `Instant.now()` or `clock.now()` to avoid creating unnecessary database transactions.
         - **Production Code:** In production code, if a flow fails because it is calling `getTxTime()` outside of a transaction, you must wrap the *caller* in a transaction instead of adding an unnecessary `reTransact()` around `getTxTime()`.
     - **Transactional Time:** Ensure code that relies on `tm().getTransactionTime()` (or `tm().getTxTime()`) is executed within a transaction context.
+- **Database Schema Migrations & 2-PR Split Mandate:**
+    - **Mandatory Consultation of `db/README.md`:** Before planning, drafting, or executing any database schema modifications (e.g., adding/altering columns, creating Flyway `.sql` migration scripts, or modifying JPA entity `@Column` mappings), you **MUST read and strictly adhere to `db/README.md`**.
+    - **Strict 2-PR Deployment Split:** Never propose or submit combining Flyway SQL scripts (`db/src/main/resources/sql/flyway/V*.sql`) and Java ORM changes (`.java` entity files + `db-schema.sql.generated`) into a single PR for submission to `master`. Because live servers during a rolling deployment will fail if Java code attempts to access unmigrated database columns/constraints, all schema additions must be split into two sequential PRs per `db/README.md`:
+        1. **PR #1 (Database Schema Only):** Contains *only* the new Flyway `.sql` script, the `flyway.txt` index update (`:db:generateFlywayIndex`), the `nomulus.golden.sql` dump (`:nom:generate_golden_file`), and ER diagrams (`er_diagram/`). Must contain **zero `.java` files or `db-schema.sql.generated` changes**.
+        2. **PR #2 (Java ORM, EPP Flows & Generated Schema Map):** Submitted *only after* PR #1 is deployed to production. Contains all `.java` entity/flow modifications, tests, and the regenerated `db-schema.sql.generated` (`generateSqlSchema`).
 
 ### 5. Testing Best Practices
 - **Mandatory Proactive Testing:** You MUST automatically write and update tests alongside your code changes WITHOUT waiting for the user to prompt you. If you add a new feature, fix a bug, or change core logic, you are explicitly required to identify the corresponding `*Test.java` file and implement comprehensive test coverage for your changes.
@@ -51,6 +56,7 @@ This document outlines foundational mandates, architectural patterns, and projec
 - **Empirical Reproduction:** Before fixing a bug, always create a test case that reproduces the failure.
 - **Base Classes:** Leverage `CommandTestCase`, `EppToolCommandTestCase`, etc., to reduce boilerplate and ensure consistent setup (e.g., clock initialization).
 - **Gradle Test Patterns:** When running tests to investigate fixes in the "core" directory, try to first use the "standardTest" Gradle task. It is faster than the "test" task, which includes the "fragileTest" task. Only run the full "test" task after "standardTest" succeeds.
+- **Mandatory SQL Integration Verification:** Whenever you modify any database schema, Flyway script (`.sql`), or JPA entity class, you MUST explicitly run both `./gradlew :db:test` and `./gradlew :core:sqlIntegrationTest` **in addition to** the standard test suites (`./gradlew standardTest` or `./gradlew test`) before finalizing the task or declaring completion. Do not rely solely on `standardTest` when database schemas or ORM mappings are touched; all three test suites (`standardTest`, `:db:test`, and `:core:sqlIntegrationTest`) are mandatory.
 
 ### 6. Project Dependencies
 - **Common Module:** When using `Clock` or other core utilities in a new or separate module (like `load-testing`), ensure `implementation project(':common')` is added to the module's `build.gradle`.
@@ -171,3 +177,4 @@ This protocol defines the standard for interacting with GitHub repositories and
 - **One Commit Per PR:** Ensure all changes are squashed into a single, clean commit. Use `git commit --amend --no-edit` for follow-up fixes.
 - **Clean Workspace:** Always run `git status` and verify the repository state before declaring a task complete.
 - **Package Lock:** The Gradle build automatically modifies `console-webapp/package-lock.json` via the `npmInstallDeps` task. ALWAYS revert this file (`git checkout console-webapp/package-lock.json`) before staging changes or finalizing a commit unless you explicitly modified NPM dependencies.
+- **PR Description Synchronization:** Whenever you amend or update a commit's description, you **MUST** check whether the corresponding GitHub PR description (if one exists) matches the previous commit description. If the PR description was not manually customized (i.e., it simply reflects the older commit description), you must automatically update it via `gh pr edit` to keep it in sync with your newly updated commit description—**strictly after the updated commit has been pushed to the remote GitHub PR branch**. Do not update the remote PR description when changes are only committed locally. **CRITICAL:** When updating the PR description, you must strictly preserve any automated review links or footer blocks (such as Reviewable link blocks: `<!-- Reviewable:start -->...<!-- Reviewable:end -->`) at the bottom of the description.

core/src/main/java/google/registry/beam/spec11/SafeBrowsingTransforms.java

@@ -226,18 +226,17 @@ public class SafeBrowsingTransforms {
     private void processResponse(
         CloseableHttpResponse response,
         ImmutableSet.Builder<KV<DomainNameInfo, ThreatMatch>> resultBuilder)
-        throws JSONException, IOException {
+        throws IOException {
       int statusCode = response.getStatusLine().getStatusCode();
       if (statusCode != SC_OK) {
-        logger.atWarning().log("Got unexpected status code %s from response.", statusCode);
-      } else {
-        // Unpack the response body
-        JSONObject responseBody =
-            new JSONObject(
-                CharStreams.toString(
-                    new InputStreamReader(response.getEntity().getContent(), UTF_8)));
-        logger.atInfo().log("Got response: %s", responseBody);
-        if (responseBody.length() == 0) {
+        throw new IOException(
+            String.format("Got unexpected status code %s from response.", statusCode));
+      }
+      // Unpack the response body
+      try (InputStreamReader reader =
+          new InputStreamReader(response.getEntity().getContent(), UTF_8)) {
+        JSONObject responseBody = new JSONObject(CharStreams.toString(reader));
+        if (responseBody.isEmpty()) {
           logger.atInfo().log("Response was empty, no threats detected.");
         } else {
           // Emit all DomainNameInfos with their API results.

core/src/main/java/google/registry/config/RegistryConfig.java

@@ -38,6 +38,7 @@ import google.registry.dns.ReadDnsRefreshRequestsAction;
 import google.registry.model.common.DnsRefreshRequest;
 import google.registry.mosapi.MosApiClient;
 import google.registry.persistence.transaction.JpaTransactionManager;
+import google.registry.rde.RdeUploadUrl;
 import google.registry.request.Action.Service;
 import google.registry.util.RegistryEnvironment;
 import google.registry.util.YamlUtils;
@@ -842,8 +843,8 @@ public final class RegistryConfig {
      */
     @Provides
     @Config("rdeUploadUrl")
-    public static URI provideRdeUploadUrl(RegistryConfigSettings config) {
-      return URI.create(config.rde.uploadUrl);
+    public static RdeUploadUrl provideRdeUploadUrl(RegistryConfigSettings config) {
+      return RdeUploadUrl.create(URI.create(config.rde.uploadUrl));
     }
 
     /**

core/src/main/java/google/registry/flows/CookieSessionMetadata.java

@@ -19,7 +19,6 @@ import static java.nio.charset.StandardCharsets.US_ASCII;
 import com.google.common.base.Joiner;
 import com.google.common.base.Splitter;
 import com.google.common.collect.ImmutableSet;
-import com.google.common.flogger.FluentLogger;
 import com.google.common.io.BaseEncoding;
 import google.registry.request.Response;
 import jakarta.servlet.http.HttpServletRequest;
@@ -55,7 +54,6 @@ public class CookieSessionMetadata extends SessionMetadata {
       Pattern.compile("serviceExtensionUris=([^,\\s}]+)?");
   private static final Pattern FAILED_LOGIN_ATTEMPTS_PATTERN =
       Pattern.compile("failedLoginAttempts=([^,\\s]+)?");
-  private static final FluentLogger logger = FluentLogger.forEnclosingClass();
 
   private final Map<String, String> data = new HashMap<>();
 
@@ -66,7 +64,6 @@ public class CookieSessionMetadata extends SessionMetadata {
               Matcher matcher = COOKIE_PATTERN.matcher(cookie);
               if (matcher.find()) {
                 String sessionInfo = decode(matcher.group(1));
-                logger.atInfo().log("SESSION INFO: %s", sessionInfo);
                 matcher = REGISTRAR_ID_PATTERN.matcher(sessionInfo);
                 if (matcher.find()) {
                   String registrarId = matcher.group(1);

core/src/main/java/google/registry/flows/EppController.java

@@ -68,9 +68,12 @@ public final class EppController {
       try {
         eppInput = unmarshalEpp(EppInput.class, inputXmlBytes);
       } catch (EppException e) {
-        // Log the unmarshalling error, with the raw bytes (in base64) to help with debugging.
+        // Log the unmarshalling error, with the sanitized bytes (in base64) to help with debugging.
+        Optional<String> sanitizedXml = EppXmlSanitizer.sanitizeEppXmlIfValid(inputXmlBytes);
+        String xmlBytesToLog =
+            base64().encode(sanitizedXml.map(xml -> xml.getBytes(UTF_8)).orElse(inputXmlBytes));
         logger.atInfo().withCause(e).log(
-            "EPP request XML unmarshalling failed - \"%s\":\n%s\n%s\n%s\n%s",
+            "EPP request XML unmarshalling failed - \"%s\":\n%s\n%s",
             e.getMessage(),
             lazy(
                 () ->
@@ -83,12 +86,7 @@ public final class EppController {
                             "resultMessage",
                             e.getResult().getCode().msg,
                             "xmlBytes",
-                            base64().encode(inputXmlBytes)))),
-            LOG_SEPARATOR,
-            lazy(
-                () ->
-                    new String(inputXmlBytes, UTF_8)
-                        .trim()), // Charset decoding failures are swallowed.
+                            xmlBytesToLog))),
             LOG_SEPARATOR);
         // Return early by sending an error message, with no clTRID since we couldn't unmarshal it.
         eppMetricBuilder.setStatus(e.getResult().getCode());

core/src/main/java/google/registry/flows/EppXmlSanitizer.java

@@ -87,16 +87,22 @@ public class EppXmlSanitizer {
    *
    * <p>Also, an empty element will be formatted as {@code <tag></tag>} instead of {@code <tag/>}.
    */
-  public static String sanitizeEppXml(byte[] inputXmlBytes) {
+  public static Optional<String> sanitizeEppXmlIfValid(byte[] inputXmlBytes) {
     try {
       // Keep exactly one newline at end of sanitized string.
-      return CharMatcher.whitespace().trimTrailingFrom(sanitizeAndEncode(inputXmlBytes)) + "\n";
+      return Optional.of(
+          CharMatcher.whitespace().trimTrailingFrom(sanitizeAndEncode(inputXmlBytes)) + "\n");
     } catch (XMLStreamException | UnsupportedEncodingException e) {
       logger.atWarning().withCause(e).log("Failed to sanitize EPP XML message.");
-      return Base64.getMimeEncoder().encodeToString(inputXmlBytes);
+      return Optional.empty();
     }
   }
 
+  public static String sanitizeEppXml(byte[] inputXmlBytes) {
+    return sanitizeEppXmlIfValid(inputXmlBytes)
+        .orElseGet(() -> Base64.getMimeEncoder().encodeToString(inputXmlBytes));
+  }
+
   private static String sanitizeAndEncode(byte[] inputXmlBytes)
       throws XMLStreamException, UnsupportedEncodingException {
     XMLEventReader xmlEventReader =

core/src/main/java/google/registry/gcs/GcsUtils.java

@@ -14,12 +14,15 @@
 
 package google.registry.gcs;
 
+import static com.google.common.base.Preconditions.checkState;
 import static com.google.common.collect.ImmutableList.toImmutableList;
 import static com.google.common.collect.Iterables.getLast;
 
 import com.google.cloud.storage.Blob;
 import com.google.cloud.storage.BlobId;
 import com.google.cloud.storage.BlobInfo;
+import com.google.cloud.storage.Bucket;
+import com.google.cloud.storage.BucketInfo;
 import com.google.cloud.storage.Storage;
 import com.google.cloud.storage.Storage.BlobListOption;
 import com.google.cloud.storage.StorageException;
@@ -33,12 +36,14 @@ import com.google.common.flogger.FluentLogger;
 import com.google.common.net.MediaType;
 import google.registry.config.CredentialModule.ApplicationDefaultCredential;
 import google.registry.util.GoogleCredentialsBundle;
+import google.registry.util.RegistryEnvironment;
 import jakarta.inject.Inject;
 import java.io.IOException;
 import java.io.InputStream;
 import java.io.OutputStream;
 import java.io.Serializable;
 import java.nio.channels.Channels;
+import java.util.concurrent.ConcurrentHashMap;
 import javax.annotation.CheckReturnValue;
 
 /**
@@ -50,6 +55,8 @@ public class GcsUtils implements Serializable {
 
   private static final FluentLogger logger = FluentLogger.forEnclosingClass();
 
+  private static final ConcurrentHashMap<String, Boolean> PAP_CACHE = new ConcurrentHashMap<>();
+
   private static final ImmutableMap<String, MediaType> EXTENSIONS =
       new ImmutableMap.Builder<String, MediaType>()
           .put("ghostryde", MediaType.APPLICATION_BINARY)
@@ -85,6 +92,7 @@ public class GcsUtils implements Serializable {
   /** Opens a GCS file for writing as an {@link OutputStream}, overwriting existing files. */
   @CheckReturnValue
   public OutputStream openOutputStream(BlobId blobId) {
+    verifyPublicAccessPrevention(blobId.getBucket());
     return Channels.newOutputStream(storage().writer(createBlobInfo(blobId)));
   }
 
@@ -94,6 +102,7 @@ public class GcsUtils implements Serializable {
    */
   @CheckReturnValue
   public OutputStream openOutputStream(BlobId blobId, ImmutableMap<String, String> metadata) {
+    verifyPublicAccessPrevention(blobId.getBucket());
     return Channels.newOutputStream(
         storage().writer(BlobInfo.newBuilder(blobId).setMetadata(metadata).build()));
   }
@@ -105,6 +114,7 @@ public class GcsUtils implements Serializable {
 
   /** Creates a GCS file with the given byte contents and metadata, overwriting existing files. */
   public void createFromBytes(BlobInfo blobInfo, byte[] bytes) throws StorageException {
+    verifyPublicAccessPrevention(blobInfo.getBucket());
     storage().create(blobInfo, bytes);
   }
 
@@ -120,6 +130,7 @@ public class GcsUtils implements Serializable {
 
   /** Update file content type on existing GCS file */
   public void updateContentType(BlobId blobId, String contentType) throws StorageException {
+    verifyPublicAccessPrevention(blobId.getBucket());
     if (existsAndNotEmpty(blobId)) {
       Blob blob = storage().get(blobId);
       blob.toBuilder().setContentType(contentType).build().update();
@@ -154,12 +165,6 @@ public class GcsUtils implements Serializable {
     }
   }
 
-  /** Returns the user defined metadata of a GCS file if the file exists, or an empty map. */
-  public ImmutableMap<String, String> getMetadata(BlobId blobId) throws StorageException {
-    Blob blob = storage().get(blobId);
-    return blob == null ? ImmutableMap.of() : ImmutableMap.copyOf(blob.getMetadata());
-  }
-
   /**
    * Returns the {@link BlobInfo} of the given GCS file.
    *
@@ -179,6 +184,37 @@ public class GcsUtils implements Serializable {
     return builder.build();
   }
 
+  /**
+   * Asserts that Public Access Prevention (PAP) is enforced on the GCS bucket.
+   *
+   * @throws IllegalStateException if PAP is not ENFORCED.
+   */
+  @VisibleForTesting
+  void verifyPublicAccessPrevention(String bucketName) {
+    if (RegistryEnvironment.get() != RegistryEnvironment.PRODUCTION) {
+      return;
+    }
+    PAP_CACHE.computeIfAbsent(
+        bucketName,
+        name -> {
+          Bucket bucket = storage().get(name);
+          checkState(bucket != null, "Bucket %s does not exist", name);
+          BucketInfo.PublicAccessPrevention pap =
+              bucket.getIamConfiguration().getPublicAccessPrevention();
+          checkState(
+              pap == BucketInfo.PublicAccessPrevention.ENFORCED,
+              "Public Access Prevention is not enforced on bucket %s. Current state: %s",
+              name,
+              pap);
+          return true;
+        });
+  }
+
+  @VisibleForTesting
+  static void clearPapCache() {
+    PAP_CACHE.clear();
+  }
+
   // These two methods are needed to check whether serialization is done correctly in tests.
   @Override
   public boolean equals(Object obj) {

core/src/main/java/google/registry/model/OteStats.java

@@ -176,11 +176,12 @@ public class OteStats {
      * Check if the {@link HistoryEntry} type matches as well as the {@link EppInput} if supplied.
      */
     private boolean matches(HistoryEntry.Type historyType, Optional<EppInput> eppInput) {
-      if (eppInputFilter.isPresent() && eppInput.isPresent()) {
-        return typeFilter.test(historyType) && eppInputFilter.get().test(eppInput.get());
-      } else {
-        return typeFilter.test(historyType);
+      if (!typeFilter.test(historyType)) {
+        return false;
       }
+      return eppInputFilter
+          .map(filter -> eppInput.isPresent() && filter.test(eppInput.get()))
+          .orElse(true);
     }
   }
 

core/src/main/java/google/registry/model/registrar/Registrar.java

@@ -871,7 +871,10 @@ public class Registrar extends UpdateAutoTimestampEntity implements Buildable, J
     }
 
     public Builder setIpAddressAllowList(Iterable<CidrAddressBlock> ipAddressAllowList) {
-      getInstance().ipAddressAllowList = ImmutableList.copyOf(ipAddressAllowList);
+      getInstance().ipAddressAllowList =
+          ipAddressAllowList == null
+              ? ImmutableList.of()
+              : ImmutableList.copyOf(ipAddressAllowList);
       return this;
     }
 
@@ -1030,6 +1033,11 @@ public class Registrar extends UpdateAutoTimestampEntity implements Buildable, J
     return tm().transact(() -> tm().loadAllOf(Registrar.class));
   }
 
+  /** Loads all registrar entities directly from the database, sorted by the given field names. */
+  public static Iterable<Registrar> loadAllSorted(String... sortFields) {
+    return tm().transact(() -> tm().loadAllOfSorted(Registrar.class, sortFields));
+  }
+
   /** Loads all registrar entities using an in-memory cache. */
   public static Iterable<Registrar> loadAllCached() {
     return CACHE_BY_REGISTRAR_ID.get().values();

core/src/main/java/google/registry/model/tld/label/PremiumList.java

@@ -169,6 +169,14 @@ public final class PremiumList extends BaseDomainLabelList<BigDecimal, PremiumEn
         getInstance().revisionId = revisionId;
         return this;
       }
+
+      @Override
+      public PremiumEntry build() {
+        checkArgument(getInstance().price != null, "Price must not be null");
+        checkArgument(
+            getInstance().price.compareTo(BigDecimal.ZERO) >= 0, "Price must not be negative");
+        return super.build();
+      }
     }
   }
 

core/src/main/java/google/registry/persistence/transaction/DelegatingReplicaJpaTransactionManager.java

@@ -257,11 +257,21 @@ public class DelegatingReplicaJpaTransactionManager implements JpaTransactionMan
     return getReplica().loadAllOf(clazz);
   }
 
+  @Override
+  public <T> ImmutableList<T> loadAllOfSorted(Class<T> clazz, String... sortFields) {
+    return getReplica().loadAllOfSorted(clazz, sortFields);
+  }
+
   @Override
   public <T> Stream<T> loadAllOfStream(Class<T> clazz) {
     return getReplica().loadAllOfStream(clazz);
   }
 
+  @Override
+  public <T> Stream<T> loadAllOfSortedStream(Class<T> clazz, String... sortFields) {
+    return getReplica().loadAllOfSortedStream(clazz, sortFields);
+  }
+
   @Override
   public <T> Optional<T> loadSingleton(Class<T> clazz) {
     return getReplica().loadSingleton(clazz);

core/src/main/java/google/registry/persistence/transaction/JpaTransactionManagerImpl.java

@@ -76,6 +76,7 @@ import java.util.concurrent.TimeUnit;
 import java.util.concurrent.atomic.AtomicLong;
 import java.util.function.Supplier;
 import java.util.function.UnaryOperator;
+import java.util.regex.Pattern;
 import java.util.stream.Stream;
 import java.util.stream.StreamSupport;
 import javax.annotation.Nullable;
@@ -88,6 +89,16 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
 
   private static final FluentLogger logger = FluentLogger.forEnclosingClass();
   private static final Retrier retrier = new Retrier(new SystemSleeper(), 6);
+
+  /**
+   * Strict allowlist regex for property/field names in dynamic JPQL ORDER BY clauses.
+   *
+   * <p>JPA and database engines forbid bind parameters (e.g. ? or :param) for schema identifiers or
+   * property names in ORDER BY clauses. To prevent JPQL/SQL injection when dynamically constructing
+   * sort queries, every sort field MUST be validated against this pattern before concatenation.
+   */
+  private static final Pattern VALID_SORT_FIELD_PATTERN = Pattern.compile("^[a-zA-Z0-9_.]+$");
+
   private static final String NESTED_TRANSACTION_MESSAGE =
       "Nested transaction detected. Try refactoring to avoid nested transactions. If unachievable,"
           + " use reTransact() in nested transactions";
@@ -528,15 +539,38 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
 
   @Override
   public <T> ImmutableList<T> loadAllOf(Class<T> clazz) {
-    return loadAllOfStream(clazz).collect(toImmutableList());
+    return loadAllOfSorted(clazz);
   }
 
   @Override
   public <T> Stream<T> loadAllOfStream(Class<T> clazz) {
+    return loadAllOfSortedStream(clazz);
+  }
+
+  @Override
+  public <T> ImmutableList<T> loadAllOfSorted(Class<T> clazz, String... sortFields) {
+    return loadAllOfSortedStream(clazz, sortFields).collect(toImmutableList());
+  }
+
+  @Override
+  public <T> Stream<T> loadAllOfSortedStream(Class<T> clazz, String... sortFields) {
     checkArgumentNotNull(clazz, "clazz must be specified");
+    checkArgumentNotNull(sortFields, "sortFields must not be null");
     assertInTransaction();
+    StringBuilder queryString =
+        new StringBuilder(String.format("FROM %s", getEntityType(clazz).getName()));
+    if (sortFields.length > 0) {
+      for (String field : sortFields) {
+        checkArgument(
+            VALID_SORT_FIELD_PATTERN.matcher(field).matches(),
+            "Invalid sort field name: %s",
+            field);
+      }
+      queryString.append(" ORDER BY ");
+      queryString.append(String.join(", ", sortFields));
+    }
     return getEntityManager()
-        .createQuery(String.format("FROM %s", getEntityType(clazz).getName()), clazz)
+        .createQuery(queryString.toString(), clazz)
         .getResultStream()
         .map(this::detach);
   }

core/src/main/java/google/registry/persistence/transaction/TransactionManager.java

@@ -219,6 +219,14 @@ public interface TransactionManager {
    */
   <T> ImmutableList<T> loadAllOf(Class<T> clazz);
 
+  /**
+   * Returns a list of all entities of the given type that exist in the database, ordered by the
+   * specified field names in ascending order.
+   *
+   * <p>The resulting list is empty if there are no entities of this type.
+   */
+  <T> ImmutableList<T> loadAllOfSorted(Class<T> clazz, String... sortFields);
+
   /**
    * Returns a stream of all entities of the given type that exist in the database.
    *
@@ -226,6 +234,14 @@ public interface TransactionManager {
    */
   <T> Stream<T> loadAllOfStream(Class<T> clazz);
 
+  /**
+   * Returns a stream of all entities of the given type that exist in the database, ordered by the
+   * specified field names in ascending order.
+   *
+   * <p>The resulting stream is empty if there are no entities of this type.
+   */
+  <T> Stream<T> loadAllOfSortedStream(Class<T> clazz, String... sortFields);
+
   /**
    * Loads the only instance of this particular class, or empty if none exists.
    *

core/src/main/java/google/registry/rde/JSchSshSession.java

@@ -24,7 +24,6 @@ import com.jcraft.jsch.SftpException;
 import google.registry.config.RegistryConfig.Config;
 import jakarta.inject.Inject;
 import java.io.Closeable;
-import java.net.URI;
 import java.time.Duration;
 
 /**
@@ -57,8 +56,7 @@ final class JSchSshSession implements Closeable {
      *
      * @throws JSchException if we fail to open the connection.
      */
-    JSchSshSession create(JSch jsch, URI uri) throws JSchException {
-      RdeUploadUrl url = RdeUploadUrl.create(uri);
+    JSchSshSession create(JSch jsch, RdeUploadUrl url) throws JSchException {
       logger.atInfo().log("Connecting to SSH endpoint: %s", url);
       Session session = jsch.getSession(
           url.getUser().orElse("domain-registry"),

core/src/main/java/google/registry/rde/RdeStagingAction.java

@@ -312,7 +312,8 @@ public final class RdeStagingAction implements Runnable {
                             jobRegion,
                             new LaunchFlexTemplateRequest().setLaunchParameter(parameter))
                         .execute();
-                logger.atInfo().log("Got response: %s", launchResponse.getJob().toPrettyString());
+                logger.atInfo().log(
+                    "Launched RDE staging Dataflow job: %s", launchResponse.getJob().getId());
                 jobNameBuilder.add(launchResponse.getJob().getId());
               } catch (IOException e) {
                 logger.atWarning().withCause(e).log("Pipeline Launch failed");

core/src/main/java/google/registry/rde/RdeUploadAction.java

@@ -64,7 +64,6 @@ import java.io.ByteArrayOutputStream;
 import java.io.IOException;
 import java.io.InputStream;
 import java.io.OutputStream;
-import java.net.URI;
 import java.time.Duration;
 import java.time.Instant;
 import java.util.Optional;
@@ -116,11 +115,16 @@ public final class RdeUploadAction implements Runnable, EscrowTask {
   @Inject @Config("rdeInterval") Duration interval;
   @Inject @Config("rdeUploadLockTimeout") Duration timeout;
   @Inject @Config("rdeUploadSftpCooldown") Duration sftpCooldown;
-  @Inject @Config("rdeUploadUrl") URI uploadUrl;
   @Inject @Key("rdeReceiverKey") PGPPublicKey receiverKey;
   @Inject @Key("rdeSigningKey") PGPKeyPair signingKey;
   @Inject @Key("rdeStagingDecryptionKey") PGPPrivateKey stagingDecryptionKey;
+
+  @Inject
+  @Config("rdeUploadUrl")
+  RdeUploadUrl rdeUploadUrl;
+
   @Inject RdeUploadAction() {}
+
   @Override
   public void run() {
     logger.atInfo().log("Attempting to acquire RDE upload lock for TLD '%s'.", tld);
@@ -135,7 +139,7 @@ public final class RdeUploadAction implements Runnable, EscrowTask {
   @Override
   public void runWithLock(Instant watermark) throws Exception {
     // If a prefix is not provided,try to determine the prefix. This should only happen when the RDE
-    // upload cron job runs to catch up any un-retried (i. e. expected) RDE failures.
+    // upload cron job runs to catch up any un-retried (i.e. expected) RDE failures.
     String actualPrefix =
         prefix.orElseGet(() -> findMostRecentPrefixForWatermark(watermark, bucket, tld, gcsUtils));
     logger.atInfo().log("Verifying readiness to upload the RDE deposit.");
@@ -181,7 +185,7 @@ public final class RdeUploadAction implements Runnable, EscrowTask {
     verifyFileExists(xmlFilename);
     verifyFileExists(xmlLengthFilename);
     verifyFileExists(reportFilename);
-    logger.atInfo().log("Commencing RDE upload for TLD '%s' to '%s'.", tld, uploadUrl);
+    logger.atInfo().log("Commencing RDE upload for TLD '%s' to '%s'.", tld, rdeUploadUrl);
     final long xmlLength = readXmlLength(xmlLengthFilename);
     retrier.callWithRetry(
         () -> upload(xmlFilename, xmlLength, watermark, name, nameWithoutPrefix),
@@ -221,10 +225,10 @@ public final class RdeUploadAction implements Runnable, EscrowTask {
   private void upload(
       BlobId xmlFile, long xmlLength, Instant watermark, String name, String nameWithoutPrefix)
       throws Exception {
-    logger.atInfo().log("Uploading XML file '%s' to remote path '%s'.", xmlFile, uploadUrl);
+    logger.atInfo().log("Uploading XML file '%s' to remote path '%s'.", xmlFile, rdeUploadUrl);
     try (InputStream gcsInput = gcsUtils.openInputStream(xmlFile);
         InputStream ghostrydeDecoder = Ghostryde.decoder(gcsInput, stagingDecryptionKey)) {
-      try (JSchSshSession session = jschSshSessionFactory.create(lazyJsch.get(), uploadUrl);
+      try (JSchSshSession session = jschSshSessionFactory.create(lazyJsch.get(), rdeUploadUrl);
           JSchSftpChannel ftpChan = session.openSftpChannel()) {
         ByteArrayOutputStream sigOut = new ByteArrayOutputStream();
         String rydeFilename = nameWithoutPrefix + ".ryde";

core/src/main/java/google/registry/rde/RdeUploadUrl.java

@@ -37,7 +37,7 @@ import javax.annotation.concurrent.Immutable;
  * @see RdeUploadAction
  */
 @Immutable
-final class RdeUploadUrl implements Comparable<RdeUploadUrl> {
+public final class RdeUploadUrl implements Comparable<RdeUploadUrl> {
 
   public static final Protocol SFTP = new Protocol("sftp", 22);
   private static final ImmutableMap<String, Protocol> ALLOWED_PROTOCOLS =

core/src/main/java/google/registry/reporting/billing/GenerateInvoicesAction.java

@@ -14,10 +14,14 @@
 
 package google.registry.reporting.billing;
 
+import static com.google.common.base.Preconditions.checkState;
 import static google.registry.beam.BeamUtils.createJobName;
+import static google.registry.model.common.Cursor.CursorType.RECURRING_BILLING;
+import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
 import static google.registry.request.Action.Method.POST;
 import static jakarta.servlet.http.HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
 import static jakarta.servlet.http.HttpServletResponse.SC_OK;
+import static java.time.ZoneOffset.UTC;
 
 import com.google.api.services.dataflow.Dataflow;
 import com.google.api.services.dataflow.model.LaunchFlexTemplateParameter;
@@ -29,6 +33,7 @@ import com.google.common.flogger.FluentLogger;
 import com.google.common.net.MediaType;
 import google.registry.batch.CloudTasksUtils;
 import google.registry.config.RegistryConfig.Config;
+import google.registry.model.common.Cursor;
 import google.registry.persistence.PersistenceModule;
 import google.registry.reporting.ReportingModule;
 import google.registry.request.Action;
@@ -40,7 +45,9 @@ import google.registry.util.RegistryEnvironment;
 import jakarta.inject.Inject;
 import java.io.IOException;
 import java.time.Duration;
+import java.time.Instant;
 import java.time.YearMonth;
+import java.util.Optional;
 
 /**
  * Invokes the {@code InvoicingPipeline} beam template via the REST api, and enqueues the {@link
@@ -107,6 +114,7 @@ public class GenerateInvoicesAction implements Runnable {
     response.setContentType(MediaType.PLAIN_TEXT_UTF_8);
     logger.atInfo().log("Launching invoicing pipeline for %s.", yearMonth);
     try {
+      checkBillingRecurrenceCursor();
       LaunchFlexTemplateParameter parameter =
           new LaunchFlexTemplateParameter()
               .setJobName(createJobName("invoicing", clock))
@@ -156,4 +164,20 @@ public class GenerateInvoicesAction implements Runnable {
       response.setPayload(String.format("Pipeline launch failed: %s", e.getMessage()));
     }
   }
+
+  private void checkBillingRecurrenceCursor() {
+    Optional<Cursor> previousCursor =
+        tm().transact(() -> tm().loadByKeyIfPresent(Cursor.createGlobalVKey(RECURRING_BILLING)));
+    checkState(
+        previousCursor.isPresent(),
+        "BillingRecurrence expansion cursor is not present. Run ExpandBillingRecurrencesAction.");
+    Instant startOfNextMonth = yearMonth.plusMonths(1).atDay(1).atStartOfDay(UTC).toInstant();
+    Instant previousCursorTime = previousCursor.get().getCursorTime();
+    checkState(
+        !previousCursorTime.isBefore(startOfNextMonth),
+        "BillingRecurrence expansion cursor (%s) is before the start of the next month (%s). "
+            + "Run ExpandBillingRecurrencesAction.",
+        previousCursorTime,
+        startOfNextMonth);
+  }
 }

core/src/main/java/google/registry/reporting/spec11/GenerateSpec11ReportAction.java

@@ -127,7 +127,7 @@ public class GenerateSpec11ReportAction implements Runnable {
                   jobRegion,
                   new LaunchFlexTemplateRequest().setLaunchParameter(parameter))
               .execute();
-      logger.atInfo().log("Got response: %s", launchResponse.getJob().toPrettyString());
+      logger.atInfo().log("Launched Spec11 Dataflow job: %s", launchResponse.getJob().getId());
       String jobId = launchResponse.getJob().getId();
       if (sendEmail) {
         cloudTasksUtils.enqueue(

core/src/main/java/google/registry/request/auth/OidcTokenAuthenticationMechanism.java

@@ -40,10 +40,9 @@ import javax.annotation.Nullable;
  * An authentication mechanism that verifies the OIDC token.
  *
  * <p>Currently, two flavors are supported: one that checks for the OIDC token as a regular bearer
- * token, and another that checks for the OIDC token passed by IAP. In both cases, the {@link
- * AuthResult} with the highest {@link AuthLevel} possible is returned. So, if the email address for
- * which the token is minted exists both as a {@link User} and as a service account, the returned
- * {@link AuthResult} is at {@link AuthLevel#USER}.
+ * token, and another that checks for the OIDC token passed by IAP. In the case where the email
+ * address for which the token is minted exists both as a {link User} and as a service account, the
+ * returned {@link AuthResult} is at {@link AuthLevel#APP} to avoid database lookups when possible.
  *
  * @see <a href="https://developers.google.com/identity/openid-connect/openid-connect">OpenID
  *     Connect </a>

core/src/main/java/google/registry/tools/server/GenerateZoneFilesAction.java

@@ -33,6 +33,7 @@ import google.registry.model.domain.Domain;
 import google.registry.model.domain.secdns.DomainDsData;
 import google.registry.model.host.Host;
 import google.registry.model.tld.Tld;
+import google.registry.model.tld.Tlds;
 import google.registry.request.Action;
 import google.registry.request.HttpException.BadRequestException;
 import google.registry.request.JsonActionRunner;
@@ -119,6 +120,7 @@ public class GenerateZoneFilesAction implements Runnable, JsonActionRunner.JsonA
   public Map<String, Object> handleJsonRequest(Map<String, ?> json) {
     @SuppressWarnings("unchecked")
     ImmutableSet<String> tlds = ImmutableSet.copyOf((List<String>) json.get("tlds"));
+    Tlds.assertTldsExist(tlds);
     Instant exportTime = Instant.parse(json.get("exportTime").toString());
     // We disallow exporting within the past 2 minutes because there might be outstanding writes.
     // We can only reliably call loadAtPointInTime at times that are UTC midnight and >
@@ -233,15 +235,20 @@ public class GenerateZoneFilesAction implements Runnable, JsonActionRunner.JsonA
     String domainLabel = stripTld(domain.getDomainName(), domain.getTld());
     Tld tld = Tld.get(domain.getTld());
     for (Host nameserver : tm().loadByKeys(domain.getNameservers()).values()) {
+      // Load the nameservers at the export time in case they've been renamed or deleted
+      Host host = loadAtPointInTime(nameserver, exportTime);
+      if (host == null) {
+        log.atSevere().log(
+            "Domain %s contained nameserver %s that didn't exist at time %s",
+            domain.getRepoId(), nameserver.getRepoId(), exportTime);
+        continue;
+      }
       result.append(
           String.format(
               NS_FORMAT,
               domainLabel,
-              tld.getDnsNsTtl()
-                  .orElse(dnsDefaultNsTtl)
-                  .toSeconds(),
-              // Load the nameservers at the export time in case they've been renamed or deleted.
-              loadAtPointInTime(nameserver, exportTime).getHostName()));
+              tld.getDnsNsTtl().orElse(dnsDefaultNsTtl).toSeconds(),
+              host.getHostName()));
     }
     for (DomainDsData dsData : domain.getDsData()) {
       result.append(

core/src/main/java/google/registry/ui/server/console/ConsoleUsersAction.java

@@ -102,7 +102,7 @@ public class ConsoleUsersAction extends ConsoleApiAction {
   @Override
   protected void postHandler(User user) {
     checkPermission(user, registrarId, ConsolePermission.MANAGE_USERS);
-    tm().transact(this::runPostInTransaction);
+    tm().transact(() -> runPostInTransaction(user));
   }
 
   @Override
@@ -135,16 +135,16 @@ public class ConsoleUsersAction extends ConsoleApiAction {
     tm().transact(this::runDeleteInTransaction);
   }
 
-  private void runPostInTransaction() throws IOException {
+  private void runPostInTransaction(User callingUser) throws IOException {
     validateRequestParams();
     if (!tm().exists(VKey.create(User.class, this.userData.get().emailAddress))) {
       this.runCreate();
     } else {
-      this.runAppendUserToExistingRegistrar();
+      this.runAppendUserToExistingRegistrar(callingUser);
     }
   }
 
-  private void runAppendUserToExistingRegistrar() {
+  private void runAppendUserToExistingRegistrar(User callingUser) {
     ImmutableList<User> allRegistrarUsers = getAllRegistrarUsers(registrarId);
     if (allRegistrarUsers.size() >= 4) {
       throw new BadRequestException("Total users amount per registrar is limited to 4");
@@ -156,6 +156,9 @@ public class ConsoleUsersAction extends ConsoleApiAction {
       throw new BadRequestException(
           "Cannot append a global administrator or user with a global role to a registrar");
     }
+    for (String existingRegistrarId : userToAppend.getUserRoles().getRegistrarRoles().keySet()) {
+      checkPermission(callingUser, existingRegistrarId, ConsolePermission.MANAGE_USERS);
+    }
 
     updateUserRegistrarRoles(
         userToAppend, registrarId, requestRoleToAllowedRoles(this.userData.get().role));
@@ -263,10 +266,15 @@ public class ConsoleUsersAction extends ConsoleApiAction {
       return;
     }
 
+    User userToUpdate = verifyUserExists(this.userData.get().emailAddress);
+    if (userToUpdate.getUserRoles().isAdmin()
+        || !userToUpdate.getUserRoles().getGlobalRole().equals(GlobalRole.NONE)) {
+      throw new BadRequestException(
+          "Cannot update a global administrator or user with a global role");
+    }
+
     updateUserRegistrarRoles(
-        verifyUserExists(this.userData.get().emailAddress),
-        registrarId,
-        requestRoleToAllowedRoles(this.userData.get().role));
+        userToUpdate, registrarId, requestRoleToAllowedRoles(this.userData.get().role));
 
     sendConfirmationEmail(registrarId, this.userData.get().emailAddress, "Updated user");
     consoleApiParams.response().setStatus(SC_OK);

core/src/main/java/google/registry/ui/server/console/RegistrarsAction.java

@@ -59,6 +59,7 @@ public class RegistrarsAction extends ConsoleApiAction {
       """
             SELECT * FROM "Registrar"
             WHERE registrar_id in :registrarIds
+            ORDER BY registrar_name ASC, registrar_id ASC
       """;
   static final String PATH = "/console-api/registrars";
   private final Optional<Registrar> registrar;
@@ -83,7 +84,7 @@ public class RegistrarsAction extends ConsoleApiAction {
       ImmutableSet<Registrar.Type> allowedRegistrarTypes =
           user.getUserRoles().isAdmin() ? TYPES_ALLOWED_FOR_ADMINS : TYPES_ALLOWED_FOR_USERS;
       ImmutableList<Registrar> registrars =
-          Streams.stream(Registrar.loadAll())
+          Streams.stream(Registrar.loadAllSorted("registrarName", "registrarId"))
               .filter(r -> allowedRegistrarTypes.contains(r.getType()))
               .collect(ImmutableList.toImmutableList());
       consoleApiParams.response().setPayload(consoleApiParams.gson().toJson(registrars));

core/src/main/java/google/registry/ui/server/console/domains/ConsoleDomainActionType.java

@@ -87,6 +87,6 @@ public abstract class ConsoleDomainActionType {
   }
 
   private String replaceValue(String xml, String key, String value) {
-    return xml.replaceAll("%" + key + "%", XML_ESCAPER.escape(value));
+    return xml.replace("%" + key + "%", XML_ESCAPER.escape(value));
   }
 }

core/src/main/resources/google/registry/beam/billing/sql/cloud_sql_billing_events.sql

@@ -30,7 +30,7 @@ JOIN Registrar r ON b.clientId = r.registrarId
 JOIN Domain d ON b.domainRepoId = d.repoId
 JOIN Tld t ON t.tldStr = d.tld
 LEFT JOIN BillingCancellation c ON b.id = c.billingEvent
-LEFT JOIN BillingCancellation cr ON b.cancellationMatchingBillingEvent = cr.billingRecurrence
+LEFT JOIN BillingCancellation cr ON b.cancellationMatchingBillingEvent = cr.billingRecurrence AND b.billingTime = cr.billingTime
 WHERE r.billingAccountMap IS NOT NULL
 AND r.type = 'REAL'
 AND t.invoicingEnabled IS TRUE

core/src/test/java/google/registry/beam/billing/InvoicingPipelineTest.java

@@ -229,7 +229,22 @@ class InvoicingPipelineTest {
               3,
               "USD",
               20.5,
-              ""));
+              ""),
+          google.registry.beam.billing.BillingEvent.create(
+              17,
+              Instant.parse("2017-10-04T00:00:00Z"),
+              Instant.parse("2017-10-04T00:00:00Z"),
+              "theRegistrar",
+              "234",
+              "",
+              "test",
+              "RENEW",
+              "recurrence-collision.test",
+              "REPO-ID",
+              3,
+              "USD",
+              20.5,
+              "SYNTHETIC"));
 
   private static final ImmutableMap<String, ImmutableList<String>> EXPECTED_DETAILED_REPORT_MAP =
       ImmutableMap.of(
@@ -239,6 +254,8 @@ class InvoicingPipelineTest {
                   + "test,RENEW,mydomain2.test,REPO-ID,3,USD,20.50,",
               "1,2017-10-04 00:00:00 UTC,2017-10-04 00:00:00 UTC,theRegistrar,234,,"
                   + "test,RENEW,mydomain.test,REPO-ID,3,USD,20.50,",
+              "17,2017-10-04 00:00:00 UTC,2017-10-04 00:00:00 UTC,theRegistrar,234,,"
+                  + "test,RENEW,recurrence-collision.test,REPO-ID,3,USD,20.50,",
               "7,2017-10-04 00:00:00 UTC,2017-10-04 00:00:00 UTC,theRegistrar,234,,"
                   + "test,SERVER_STATUS,update-prohibited.test,REPO-ID,0,USD,20.00,",
               "6,2017-10-04 00:00:00 UTC,2017-10-04 00:00:00 UTC,theRegistrar,234,,"
@@ -264,7 +281,7 @@ class InvoicingPipelineTest {
 
   private static final ImmutableList<String> EXPECTED_INVOICE_OUTPUT =
       ImmutableList.of(
-          "2017-10-01,2020-09-30,234,61.50,USD,10125,1,PURCHASE,,3,"
+          "2017-10-01,2020-09-30,234,82.00,USD,10125,1,PURCHASE,,4,"
               + "RENEW | TLD: test | TERM: 3-year,20.50,USD,",
           "2017-10-01,2022-09-30,234,70.00,JPY,10125,1,PURCHASE,,1,"
               + "CREATE | TLD: hello | TERM: 5-year,70.00,JPY,",
@@ -398,7 +415,8 @@ JOIN Registrar r ON b.clientId = r.registrarId
 JOIN Domain d ON b.domainRepoId = d.repoId
 JOIN Tld t ON t.tldStr = d.tld
 LEFT JOIN BillingCancellation c ON b.id = c.billingEvent
-LEFT JOIN BillingCancellation cr ON b.cancellationMatchingBillingEvent = cr.billingRecurrence
+LEFT JOIN BillingCancellation cr ON b.cancellationMatchingBillingEvent = cr.billingRecurrence AND \
+b.billingTime = cr.billingTime
 WHERE r.billingAccountMap IS NOT NULL
 AND r.type = 'REAL'
 AND t.invoicingEnabled IS TRUE
@@ -607,6 +625,51 @@ AND cr.id IS NULL
         Instant.parse("2017-10-04T00:00:00.0Z"),
         Instant.parse("2017-10-02T00:00:00.0Z"));
     persistBillingEvent(16, domain15, registrar11, Reason.RENEW, 3, Money.of(USD, 20.5));
+
+    // Add a billing event in Year 1 and a cancellation in Year 2 for the same recurrence.
+    // The Year 1 event should NOT be cancelled.
+    Domain domain17 = persistActiveDomain("recurrence-collision.test");
+    DomainHistory domainHistoryCollision = persistDomainHistory(domain17, registrar1);
+
+    BillingRecurrence billingRecurrenceCollision =
+        new BillingRecurrence()
+            .asBuilder()
+            .setRegistrarId(registrar1.getRegistrarId())
+            .setRecurrenceEndTime(END_INSTANT)
+            .setId(100)
+            .setDomainHistory(domainHistoryCollision)
+            .setTargetId(domain17.getDomainName())
+            .setEventTime(Instant.parse("2017-10-04T00:00:00.0Z"))
+            .setReason(Reason.RENEW)
+            .build();
+    persistResource(billingRecurrenceCollision);
+
+    // Year 1 Billing Event (October 2017)
+    BillingEvent billingEventYear1 =
+        persistBillingEvent(17, domain17, registrar1, Reason.RENEW, 3, Money.of(USD, 20.5));
+    billingEventYear1 =
+        billingEventYear1
+            .asBuilder()
+            .setCancellationMatchingBillingEvent(billingRecurrenceCollision)
+            .setFlags(ImmutableSet.of(Flag.SYNTHETIC))
+            .setSyntheticCreationTime(Instant.parse("2017-10-03T00:00:00.0Z"))
+            .build();
+    persistResource(billingEventYear1);
+
+    // Year 2 Billing Cancellation (October 2018)
+    BillingCancellation cancellationYear2 =
+        new BillingCancellation()
+            .asBuilder()
+            .setId(101)
+            .setRegistrarId(registrar1.getRegistrarId())
+            .setEventTime(Instant.parse("2018-10-05T00:00:00.0Z"))
+            .setBillingTime(Instant.parse("2018-10-04T00:00:00.0Z"))
+            .setBillingRecurrence(billingRecurrenceCollision.createVKey())
+            .setTargetId(domain17.getDomainName())
+            .setReason(Reason.RENEW)
+            .setDomainHistory(domainHistoryCollision)
+            .build();
+    persistResource(cancellationYear2);
   }
 
   private static DomainHistory persistDomainHistory(Domain domain, Registrar registrar) {

core/src/test/java/google/registry/gcs/GcsUtilsTest.java

@@ -15,27 +15,40 @@
 package google.registry.gcs;
 
 import static com.google.common.truth.Truth.assertThat;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
 
 import com.google.cloud.storage.BlobId;
 import com.google.cloud.storage.BlobInfo;
+import com.google.cloud.storage.Bucket;
+import com.google.cloud.storage.BucketInfo;
+import com.google.cloud.storage.Storage;
 import com.google.cloud.storage.StorageOptions;
 import com.google.cloud.storage.contrib.nio.testing.LocalStorageHelper;
 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableMap;
 import com.google.common.io.ByteStreams;
 import com.google.common.net.MediaType;
+import google.registry.testing.SystemPropertyExtension;
+import google.registry.util.RegistryEnvironment;
 import java.io.ByteArrayInputStream;
 import java.io.ByteArrayOutputStream;
 import java.io.IOException;
 import java.io.ObjectInputStream;
 import java.io.ObjectOutputStream;
 import java.io.OutputStream;
+import org.junit.jupiter.api.AfterEach;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.RegisterExtension;
 
 /** Unit tests for {@link GcsUtilsTest}. */
 class GcsUtilsTest {
 
+  @RegisterExtension
+  final SystemPropertyExtension systemPropertyExtension = new SystemPropertyExtension();
+
   private GcsUtils gcsUtils = new GcsUtils(LocalStorageHelper.getOptions());
 
   private String bucket = "my-bucket";
@@ -43,9 +56,18 @@ class GcsUtilsTest {
   private BlobId blobId = BlobId.of(bucket, filename);
   private ImmutableMap<String, String> metadata = ImmutableMap.of("key1", "val1", "Key2", "val2");
   private final byte[] bytes = new byte[] {'a', 'b', 'c'};
+  private RegistryEnvironment previousEnvironment;
 
   @BeforeEach
-  void beforeEach() {}
+  void beforeEach() {
+    previousEnvironment = RegistryEnvironment.get();
+    GcsUtils.clearPapCache();
+  }
+
+  @AfterEach
+  void afterEach() {
+    previousEnvironment.setup();
+  }
 
   @Test
   void testSerialization_testStorage() throws Exception {
@@ -111,6 +133,88 @@ class GcsUtilsTest {
     assertThat(gcsUtils.existsAndNotEmpty(blobId)).isFalse();
   }
 
+  @Test
+  void testVerifyPublicAccessPrevention_enforced() {
+    RegistryEnvironment.PRODUCTION.setup(systemPropertyExtension);
+    Storage mockStorage = mock(Storage.class);
+    StorageOptions mockOptions = mock(StorageOptions.class);
+    when(mockOptions.getService()).thenReturn(mockStorage);
+
+    Bucket mockBucket = mock(Bucket.class);
+    when(mockStorage.get("my-bucket")).thenReturn(mockBucket);
+
+    BucketInfo.IamConfiguration mockIamConfig = mock(BucketInfo.IamConfiguration.class);
+    when(mockBucket.getIamConfiguration()).thenReturn(mockIamConfig);
+    when(mockIamConfig.getPublicAccessPrevention())
+        .thenReturn(BucketInfo.PublicAccessPrevention.ENFORCED);
+
+    GcsUtils utils = new GcsUtils(mockOptions);
+    utils.verifyPublicAccessPrevention("my-bucket");
+  }
+
+  @Test
+  void testVerifyPublicAccessPrevention_notCheckedInNonProd() {
+    RegistryEnvironment.SANDBOX.setup(systemPropertyExtension);
+    Storage mockStorage = mock(Storage.class);
+    StorageOptions mockOptions = mock(StorageOptions.class);
+    when(mockOptions.getService()).thenReturn(mockStorage);
+
+    Bucket mockBucket = mock(Bucket.class);
+    when(mockStorage.get("my-bucket")).thenReturn(mockBucket);
+
+    BucketInfo.IamConfiguration mockIamConfig = mock(BucketInfo.IamConfiguration.class);
+    when(mockBucket.getIamConfiguration()).thenReturn(mockIamConfig);
+    when(mockIamConfig.getPublicAccessPrevention())
+        .thenReturn(BucketInfo.PublicAccessPrevention.INHERITED);
+
+    GcsUtils utils = new GcsUtils(mockOptions);
+    // no exception thrown even though PAP isn't enforced
+    utils.verifyPublicAccessPrevention("my-bucket");
+  }
+
+  @Test
+  void testVerifyPublicAccessPrevention_notEnforced() {
+    RegistryEnvironment.PRODUCTION.setup(systemPropertyExtension);
+    Storage mockStorage = mock(Storage.class);
+    StorageOptions mockOptions = mock(StorageOptions.class);
+    when(mockOptions.getService()).thenReturn(mockStorage);
+
+    Bucket mockBucket = mock(Bucket.class);
+    when(mockStorage.get("my-bucket")).thenReturn(mockBucket);
+
+    BucketInfo.IamConfiguration mockIamConfig = mock(BucketInfo.IamConfiguration.class);
+    when(mockBucket.getIamConfiguration()).thenReturn(mockIamConfig);
+    when(mockIamConfig.getPublicAccessPrevention())
+        .thenReturn(BucketInfo.PublicAccessPrevention.INHERITED);
+
+    GcsUtils utils = new GcsUtils(mockOptions);
+
+    IllegalStateException thrown =
+        assertThrows(
+            IllegalStateException.class, () -> utils.verifyPublicAccessPrevention("my-bucket"));
+    assertThat(thrown)
+        .hasMessageThat()
+        .contains("Public Access Prevention is not enforced on bucket my-bucket");
+  }
+
+  @Test
+  void testVerifyPublicAccessPrevention_nonexistentBucket() {
+    RegistryEnvironment.PRODUCTION.setup(systemPropertyExtension);
+    Storage mockStorage = mock(Storage.class);
+    StorageOptions mockOptions = mock(StorageOptions.class);
+    when(mockOptions.getService()).thenReturn(mockStorage);
+
+    when(mockStorage.get("nonexistent-bucket")).thenReturn(null);
+
+    GcsUtils utils = new GcsUtils(mockOptions);
+
+    IllegalStateException thrown =
+        assertThrows(
+            IllegalStateException.class,
+            () -> utils.verifyPublicAccessPrevention("nonexistent-bucket"));
+    assertThat(thrown).hasMessageThat().contains("Bucket nonexistent-bucket does not exist");
+  }
+
   private static byte[] serialize(Object object) throws IOException {
     try (ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
       ObjectOutputStream oos = new ObjectOutputStream(baos);

core/src/test/java/google/registry/model/OteStatsTest.java

@@ -16,10 +16,17 @@ package google.registry.model;
 
 import static com.google.common.truth.Truth.assertThat;
 import static google.registry.testing.DatabaseHelper.createTld;
+import static google.registry.testing.DatabaseHelper.persistActiveDomain;
+import static google.registry.testing.DatabaseHelper.persistPremiumList;
+import static google.registry.testing.DatabaseHelper.persistResource;
+import static org.joda.money.CurrencyUnit.USD;
 
 import google.registry.model.OteStats.StatType;
+import google.registry.model.domain.DomainHistory;
+import google.registry.model.reporting.HistoryEntry.Type;
 import google.registry.persistence.transaction.JpaTestExtensions;
 import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
+import java.time.Instant;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.RegisterExtension;
@@ -60,39 +67,40 @@ public final class OteStatsTest {
     OteStats stats = OteStats.getFromRegistrar("blobio");
     String expected =
         """
-            contact creates: 0
-            contact deletes: 0
-            contact transfer approves: 0
-            contact transfer cancels: 0
-            contact transfer rejects: 0
-            contact transfer requests: 0
-            contact updates: 0
-            domain autorenews: 0
-            domain creates: 5
-            domain creates ascii: 4
-            domain creates idn: 1
-            domain creates start date sunrise: 1
-            domain creates with claims notice: 1
-            domain creates with fee: 1
-            domain creates with sec dns: 1
-            domain creates without sec dns: 4
-            domain deletes: 1
-            domain renews: 0
-            domain restores: 1
-            domain transfer approves: 1
-            domain transfer cancels: 1
-            domain transfer rejects: 1
-            domain transfer requests: 1
-            domain updates: 1
-            domain updates with sec dns: 1
-            domain updates without sec dns: 0
-            host creates: 1
-            host creates external: 0
-            host creates subordinate: 1
-            host deletes: 1
-            host updates: 1
-            unclassified flows: 0
-            TOTAL: 30""";
+        contact creates: 0
+        contact deletes: 0
+        contact transfer approves: 0
+        contact transfer cancels: 0
+        contact transfer rejects: 0
+        contact transfer requests: 0
+        contact updates: 0
+        domain autorenews: 0
+        domain creates: 5
+        domain creates ascii: 4
+        domain creates idn: 1
+        domain creates start date sunrise: 1
+        domain creates with claims notice: 1
+        domain creates with fee: 1
+        domain creates with sec dns: 1
+        domain creates without sec dns: 4
+        domain deletes: 1
+        domain renews: 0
+        domain restores: 1
+        domain transfer approves: 1
+        domain transfer cancels: 1
+        domain transfer rejects: 1
+        domain transfer requests: 1
+        domain updates: 1
+        domain updates with sec dns: 1
+        domain updates without sec dns: 0
+        host creates: 1
+        host creates external: 0
+        host creates subordinate: 1
+        host deletes: 1
+        host updates: 1
+        unclassified flows: 0
+        TOTAL: 30\
+        """;
     assertThat(stats.toString()).isEqualTo(expected);
   }
 
@@ -102,39 +110,72 @@ public final class OteStatsTest {
     OteStats stats = OteStats.getFromRegistrar("blobio");
     String expected =
         """
-            contact creates: 0
-            contact deletes: 0
-            contact transfer approves: 0
-            contact transfer cancels: 0
-            contact transfer rejects: 0
-            contact transfer requests: 0
-            contact updates: 0
-            domain autorenews: 0
-            domain creates: 4
-            domain creates ascii: 4
-            domain creates idn: 0
-            domain creates start date sunrise: 1
-            domain creates with claims notice: 1
-            domain creates with fee: 1
-            domain creates with sec dns: 1
-            domain creates without sec dns: 3
-            domain deletes: 1
-            domain renews: 0
-            domain restores: 0
-            domain transfer approves: 1
-            domain transfer cancels: 1
-            domain transfer rejects: 1
-            domain transfer requests: 1
-            domain updates: 1
-            domain updates with sec dns: 1
-            domain updates without sec dns: 0
-            host creates: 1
-            host creates external: 0
-            host creates subordinate: 1
-            host deletes: 0
-            host updates: 10
-            unclassified flows: 0
-            TOTAL: 34""";
+        contact creates: 0
+        contact deletes: 0
+        contact transfer approves: 0
+        contact transfer cancels: 0
+        contact transfer rejects: 0
+        contact transfer requests: 0
+        contact updates: 0
+        domain autorenews: 0
+        domain creates: 4
+        domain creates ascii: 4
+        domain creates idn: 0
+        domain creates start date sunrise: 1
+        domain creates with claims notice: 1
+        domain creates with fee: 1
+        domain creates with sec dns: 1
+        domain creates without sec dns: 3
+        domain deletes: 1
+        domain renews: 0
+        domain restores: 0
+        domain transfer approves: 1
+        domain transfer cancels: 1
+        domain transfer rejects: 1
+        domain transfer requests: 1
+        domain updates: 1
+        domain updates with sec dns: 1
+        domain updates without sec dns: 0
+        host creates: 1
+        host creates external: 0
+        host creates subordinate: 1
+        host deletes: 0
+        host updates: 10
+        unclassified flows: 0
+        TOTAL: 34\
+        """;
     assertThat(stats.toString()).isEqualTo(expected);
   }
+
+  @Test
+  void testDomainCreateWithoutXmlBytes_doesNotSatisfyComplexRequirements() throws Exception {
+    persistPremiumList("default_sandbox_list", USD, "sandbox,USD 1000");
+    OteAccountBuilder.forRegistrarId("blobio").buildAndPersist();
+    String oteAccount1 = "blobio-1";
+    Instant now = Instant.parse("2026-06-25T10:00:00Z");
+
+    // Persist a DOMAIN_CREATE history entry without XML bytes
+    persistResource(
+        new DomainHistory.Builder()
+            .setDomain(persistActiveDomain("example.tld"))
+            .setRegistrarId(oteAccount1)
+            .setType(Type.DOMAIN_CREATE)
+            .setXmlBytes(null) // explicitly null
+            .setModificationTime(now)
+            .build());
+
+    OteStats stats = OteStats.getFromRegistrar("blobio");
+
+    // It should count towards the basic DOMAIN_CREATES
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES)).isEqualTo(1);
+
+    // It should NOT count towards any stat type that requires EPP input filtering
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_ASCII)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_IDN)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_START_DATE_SUNRISE)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_WITH_CLAIMS_NOTICE)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_WITH_FEE)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_WITH_SEC_DNS)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_WITHOUT_SEC_DNS)).isEqualTo(0);
+  }
 }

core/src/test/java/google/registry/model/tld/label/PremiumListTest.java

@@ -110,28 +110,44 @@ public class PremiumListTest {
 
   @Test
   void testValidation_labelMustBeLowercase() {
-    Exception e =
-        assertThrows(
-            IllegalArgumentException.class,
-            () ->
-                new PremiumEntry.Builder()
-                    .setPrice(BigDecimal.valueOf(399))
-                    .setLabel("UPPER.tld")
-                    .build());
-    assertThat(e).hasMessageThat().contains("must be in puny-coded, lower-case form");
+    assertThat(
+            assertThrows(
+                IllegalArgumentException.class,
+                () ->
+                    new PremiumEntry.Builder()
+                        .setPrice(BigDecimal.valueOf(399))
+                        .setLabel("UPPER.tld")
+                        .build()))
+        .hasMessageThat()
+        .contains("must be in puny-coded, lower-case form");
+  }
+
+  @Test
+  void testValidation_priceMustNotBeNegative() {
+    assertThat(
+            assertThrows(
+                IllegalArgumentException.class,
+                () ->
+                    new PremiumEntry.Builder()
+                        .setPrice(BigDecimal.valueOf(-100))
+                        .setLabel("anchor")
+                        .build()))
+        .hasMessageThat()
+        .isEqualTo("Price must not be negative");
   }
 
   @Test
   void testValidation_labelMustBePunyCoded() {
-    Exception e =
-        assertThrows(
-            IllegalArgumentException.class,
-            () ->
-                new PremiumEntry.Builder()
-                    .setPrice(BigDecimal.valueOf(399))
-                    .setLabel("lower.みんな")
-                    .build());
-    assertThat(e).hasMessageThat().contains("must be in puny-coded, lower-case form");
+    assertThat(
+            assertThrows(
+                IllegalArgumentException.class,
+                () ->
+                    new PremiumEntry.Builder()
+                        .setPrice(BigDecimal.valueOf(399))
+                        .setLabel("lower.みんな")
+                        .build()))
+        .hasMessageThat()
+        .contains("must be in puny-coded, lower-case form");
   }
 
   @Test

core/src/test/java/google/registry/persistence/transaction/JpaTransactionManagerImplTest.java

@@ -404,6 +404,39 @@ class JpaTransactionManagerImplTest {
         .containsExactlyElementsIn(moreEntities);
   }
 
+  @Test
+  void loadAllOfSorted() {
+    TestEntity entityB = new TestEntity("b_entity", "gamma");
+    TestEntity entityC = new TestEntity("c_entity", "alpha");
+    TestEntity entityA = new TestEntity("a_entity", "beta");
+    persistResources(ImmutableList.of(entityB, entityC, entityA));
+
+    assertThat(tm().transact(() -> tm().loadAllOfSorted(TestEntity.class, "name")))
+        .containsExactly(entityA, entityB, entityC)
+        .inOrder();
+
+    assertThat(tm().transact(() -> tm().loadAllOfSorted(TestEntity.class, "data")))
+        .containsExactly(entityC, entityA, entityB)
+        .inOrder();
+
+    assertThat(tm().transact(() -> tm().loadAllOfSorted(TestEntity.class, "data", "name")))
+        .containsExactly(entityC, entityA, entityB)
+        .inOrder();
+  }
+
+  @Test
+  void loadAllOfSorted_invalidFieldName_throwsException() {
+    IllegalArgumentException thrown =
+        assertThrows(
+            IllegalArgumentException.class,
+            () ->
+                tm().transact(
+                        () ->
+                            tm().loadAllOfSorted(
+                                    TestEntity.class, "name; DROP TABLE TestEntity;")));
+    assertThat(thrown).hasMessageThat().contains("Invalid sort field name");
+  }
+
   @Test
   void saveAllNew_rollsBackWhenFailure() {
     moreEntities.forEach(entity -> assertThat(tm().transact(() -> tm().exists(entity))).isFalse());

core/src/test/java/google/registry/rde/RdeUploadActionTest.java

@@ -155,7 +155,7 @@ public class RdeUploadActionTest {
       action.timeout = Duration.ofSeconds(23);
       action.tld = "tld";
       action.sftpCooldown = Duration.ofSeconds(7);
-      action.uploadUrl = uploadUrl;
+      action.rdeUploadUrl = RdeUploadUrl.create(uploadUrl);
       action.receiverKey = keyring.getRdeReceiverKey();
       action.signingKey = keyring.getRdeSigningKey();
       action.stagingDecryptionKey = keyring.getRdeStagingDecryptionKey();
@@ -212,7 +212,7 @@ public class RdeUploadActionTest {
   @Test
   void testRun() {
     createTld("lol");
-    RdeUploadAction action = createAction(null);
+    RdeUploadAction action = createAction(URI.create("sftp://user:password@localhost:5432"));
     action.tld = "lol";
     action.run();
     verify(runner)
@@ -231,7 +231,7 @@ public class RdeUploadActionTest {
   @Test
   void testRun_withPrefix() throws Exception {
     createTld("lol");
-    RdeUploadAction action = createAction(null);
+    RdeUploadAction action = createAction(URI.create("sftp://user:password@localhost:5432"));
     action.prefix = Optional.of("job-name/");
     action.tld = "lol";
     action.run();

core/src/test/java/google/registry/reporting/billing/GenerateInvoicesActionTest.java

@@ -15,8 +15,11 @@
 package google.registry.reporting.billing;
 
 import static com.google.common.truth.Truth.assertThat;
+import static google.registry.model.common.Cursor.CursorType.RECURRING_BILLING;
+import static google.registry.testing.DatabaseHelper.persistResource;
 import static jakarta.servlet.http.HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
 import static jakarta.servlet.http.HttpServletResponse.SC_OK;
+import static org.mockito.ArgumentMatchers.startsWith;
 import static org.mockito.Mockito.mock;
 import static org.mockito.Mockito.verify;
 import static org.mockito.Mockito.when;
@@ -25,6 +28,7 @@ import com.google.cloud.tasks.v2.HttpMethod;
 import com.google.common.net.MediaType;
 import google.registry.batch.CloudTasksUtils;
 import google.registry.beam.BeamActionTestBase;
+import google.registry.model.common.Cursor;
 import google.registry.persistence.transaction.JpaTestExtensions;
 import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
 import google.registry.reporting.ReportingModule;
@@ -33,6 +37,7 @@ import google.registry.testing.CloudTasksHelper.TaskMatcher;
 import google.registry.testing.FakeClock;
 import java.io.IOException;
 import java.time.Duration;
+import java.time.Instant;
 import java.time.YearMonth;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.RegisterExtension;
@@ -50,8 +55,13 @@ class GenerateInvoicesActionTest extends BeamActionTestBase {
   private CloudTasksUtils cloudTasksUtils = cloudTasksHelper.getTestCloudTasksUtils();
   private GenerateInvoicesAction action;
 
+  private void setCursor(Instant cursorTime) {
+    persistResource(Cursor.createGlobal(RECURRING_BILLING, cursorTime));
+  }
+
   @Test
   void testLaunchTemplateJob_withPublish() throws Exception {
+    setCursor(Instant.parse("2017-11-01T00:00:00Z"));
     action =
         new GenerateInvoicesAction(
             "test-project",
@@ -84,6 +94,7 @@ class GenerateInvoicesActionTest extends BeamActionTestBase {
 
   @Test
   void testLaunchTemplateJob_withoutPublish() throws Exception {
+    setCursor(Instant.parse("2017-11-01T00:00:00Z"));
     action =
         new GenerateInvoicesAction(
             "test-project",
@@ -107,6 +118,7 @@ class GenerateInvoicesActionTest extends BeamActionTestBase {
 
   @Test
   void testCaughtIOException() throws IOException {
+    setCursor(Instant.parse("2017-11-01T00:00:00Z"));
     when(launch.execute()).thenThrow(new IOException("Pipeline error"));
     action =
         new GenerateInvoicesAction(
@@ -128,4 +140,58 @@ class GenerateInvoicesActionTest extends BeamActionTestBase {
     verify(emailUtils).sendAlertEmail("Pipeline Launch failed due to Pipeline error");
     cloudTasksHelper.assertNoTasksEnqueued("beam-reporting");
   }
+
+  @Test
+  void testFailure_cursorLagging() {
+    setCursor(Instant.parse("2017-10-31T23:59:59.999Z"));
+    action =
+        new GenerateInvoicesAction(
+            "test-project",
+            "test-region",
+            "staging_bucket",
+            "billing_bucket",
+            "REG-INV",
+            false,
+            YearMonth.of(2017, 10),
+            emailUtils,
+            cloudTasksUtils,
+            clock,
+            response,
+            dataflow);
+    action.run();
+    assertThat(response.getStatus()).isEqualTo(SC_INTERNAL_SERVER_ERROR);
+    assertThat(response.getPayload()).contains("Pipeline launch failed");
+    assertThat(response.getPayload()).contains("BillingRecurrence expansion cursor");
+    verify(emailUtils)
+        .sendAlertEmail(
+            startsWith("Pipeline Launch failed due to BillingRecurrence expansion cursor"));
+    cloudTasksHelper.assertNoTasksEnqueued("beam-reporting");
+  }
+
+  @Test
+  void testFailure_cursorMissing() {
+    // Do not set cursor, should default to START_INSTANT (1970)
+    action =
+        new GenerateInvoicesAction(
+            "test-project",
+            "test-region",
+            "staging_bucket",
+            "billing_bucket",
+            "REG-INV",
+            false,
+            YearMonth.of(2017, 10),
+            emailUtils,
+            cloudTasksUtils,
+            clock,
+            response,
+            dataflow);
+    action.run();
+    assertThat(response.getStatus()).isEqualTo(SC_INTERNAL_SERVER_ERROR);
+    assertThat(response.getPayload()).contains("Pipeline launch failed");
+    assertThat(response.getPayload()).contains("BillingRecurrence expansion cursor");
+    verify(emailUtils)
+        .sendAlertEmail(
+            startsWith("Pipeline Launch failed due to BillingRecurrence expansion cursor"));
+    cloudTasksHelper.assertNoTasksEnqueued("beam-reporting");
+  }
 }

core/src/test/java/google/registry/tools/server/GenerateZoneFilesActionTest.java

@@ -23,6 +23,7 @@ import static google.registry.testing.DatabaseHelper.persistResource;
 import static google.registry.testing.TestDataHelper.loadFile;
 import static google.registry.util.DateTimeUtils.plusMinutes;
 import static java.nio.charset.StandardCharsets.UTF_8;
+import static org.junit.jupiter.api.Assertions.assertThrows;
 
 import com.google.cloud.storage.BlobId;
 import com.google.cloud.storage.contrib.nio.testing.LocalStorageHelper;
@@ -56,6 +57,20 @@ class GenerateZoneFilesActionTest {
 
   private final GcsUtils gcsUtils = new GcsUtils(LocalStorageHelper.getOptions());
 
+  @Test
+  void testGenerate_nonexistentTld_throwsException() {
+    GenerateZoneFilesAction action = new GenerateZoneFilesAction();
+    IllegalArgumentException thrown =
+        assertThrows(
+            IllegalArgumentException.class,
+            () ->
+                action.handleJsonRequest(
+                    ImmutableMap.<String, Object>of(
+                        "tlds", ImmutableList.of("nonexistent-tld"),
+                        "exportTime", Instant.parse("2024-03-27T00:00:00Z"))));
+    assertThat(thrown).hasMessageThat().contains("TLDs do not exist: nonexistent-tld");
+  }
+
   @Test
   void testGenerate_defaultTtls() throws Exception {
     createTlds("tld", "com");
@@ -167,4 +182,34 @@ class GenerateZoneFilesActionTest {
     // The remaining lines can be in any order.
     assertThat(generatedFileLines).containsExactlyElementsIn(goldenFileLines);
   }
+
+  @Test
+  void testGenerate_deletedNameserver_ignoresAndLogsSevere() throws Exception {
+    createTlds("tld");
+    Instant now = Instant.parse("2024-03-27T00:00:00Z");
+    Host deletedHost = persistResource(newHost("ns.deleted.tld"));
+    // Delete the host before exportTime
+    persistResource(deletedHost.asBuilder().setDeletionTime(now.minusSeconds(3600)).build());
+
+    persistResource(
+        DatabaseHelper.newDomain("example.tld")
+            .asBuilder()
+            .addNameservers(ImmutableSet.of(deletedHost.createVKey()))
+            .build());
+
+    GenerateZoneFilesAction action = new GenerateZoneFilesAction();
+    action.bucket = "zonefiles-bucket";
+    action.gcsUtils = gcsUtils;
+    action.databaseRetention = Duration.ofDays(29);
+    action.dnsDefaultATtl = Duration.ofSeconds(11);
+    action.dnsDefaultNsTtl = Duration.ofSeconds(222);
+    action.dnsDefaultDsTtl = Duration.ofSeconds(3333);
+    action.clock = new FakeClock(plusMinutes(now, 2));
+
+    Map<String, Object> response =
+        action.handleJsonRequest(
+            ImmutableMap.<String, Object>of("tlds", ImmutableList.of("tld"), "exportTime", now));
+    assertThat(response)
+        .containsEntry("filenames", ImmutableList.of("gs://zonefiles-bucket/tld-" + now + ".zone"));
+  }
 }

core/src/test/java/google/registry/ui/server/console/ConsoleUsersActionTest.java

@@ -424,6 +424,51 @@ class ConsoleUsersActionTest extends ConsoleActionBaseTestCase {
         .isEqualTo(RegistrarRole.TECH_CONTACT);
   }
 
+  @Test
+  void testFailure_appendUser_crossTenantNoPermission() throws IOException {
+    User callingUser = DatabaseHelper.loadByKey(VKey.create(User.class, "test1@test.com"));
+    AuthResult authResult = AuthResult.createUser(callingUser);
+    ConsoleUsersAction action =
+        createAction(
+            Optional.of(ConsoleApiParamsUtils.createFake(authResult)),
+            Optional.of("POST"),
+            Optional.of(
+                new UserData("test3@test.com", null, RegistrarRole.TECH_CONTACT.name(), null)));
+    action.run();
+    assertThat(response.getStatus()).isEqualTo(SC_FORBIDDEN);
+  }
+
+  @Test
+  void testSuccess_appendUser_crossTenantWithPermission() throws IOException {
+    User callingUser =
+        DatabaseHelper.persistResource(
+            new User.Builder()
+                .setEmailAddress("multitenant@test.com")
+                .setUserRoles(
+                    new UserRoles()
+                        .asBuilder()
+                        .setRegistrarRoles(
+                            ImmutableMap.of(
+                                "TheRegistrar",
+                                RegistrarRole.PRIMARY_CONTACT,
+                                "NewRegistrar",
+                                RegistrarRole.PRIMARY_CONTACT))
+                        .build())
+                .build());
+    AuthResult authResult = AuthResult.createUser(callingUser);
+    ConsoleUsersAction action =
+        createAction(
+            Optional.of(ConsoleApiParamsUtils.createFake(authResult)),
+            Optional.of("POST"),
+            Optional.of(
+                new UserData("test3@test.com", null, RegistrarRole.TECH_CONTACT.name(), null)));
+    action.run();
+    assertThat(response.getStatus()).isEqualTo(SC_OK);
+    User appendedUser = DatabaseHelper.loadByKey(VKey.create(User.class, "test3@test.com"));
+    assertThat(appendedUser.getUserRoles().getRegistrarRoles().get("TheRegistrar"))
+        .isEqualTo(RegistrarRole.TECH_CONTACT);
+  }
+
   @Test
   void testFailure_appendUser_globalAdmin() throws IOException {
     User user = DatabaseHelper.createAdminUser("email@email.com");
@@ -504,6 +549,62 @@ class ConsoleUsersActionTest extends ConsoleActionBaseTestCase {
         .contains("Cannot delete a global administrator or user with a global role");
   }
 
+  @Test
+  void testFailure_updateUser_globalAdmin() throws IOException {
+    User user = DatabaseHelper.createAdminUser("email@email.com");
+    AuthResult authResult = AuthResult.createUser(user);
+    // Historically associated global admin
+    DatabaseHelper.persistResource(
+        new User.Builder()
+            .setEmailAddress("globaladmin@test.com")
+            .setUserRoles(
+                new UserRoles.Builder()
+                    .setIsAdmin(true)
+                    .setGlobalRole(GlobalRole.NONE)
+                    .setRegistrarRoles(
+                        ImmutableMap.of("TheRegistrar", RegistrarRole.PRIMARY_CONTACT))
+                    .build())
+            .build());
+
+    ConsoleUsersAction action =
+        createAction(
+            Optional.of(ConsoleApiParamsUtils.createFake(authResult)),
+            Optional.of("PUT"),
+            Optional.of(
+                new UserData(
+                    "globaladmin@test.com", null, RegistrarRole.ACCOUNT_MANAGER.toString(), null)));
+    action.run();
+    assertThat(response.getStatus()).isEqualTo(SC_BAD_REQUEST);
+    assertThat(response.getPayload())
+        .contains("Cannot update a global administrator or user with a global role");
+  }
+
+  @Test
+  void testFailure_updateUser_globalRole() throws IOException {
+    User user = DatabaseHelper.createAdminUser("email@email.com");
+    AuthResult authResult = AuthResult.createUser(user);
+    // Historically associated user with global role
+    DatabaseHelper.persistResource(
+        new User.Builder()
+            .setEmailAddress("support@test.com")
+            .setUserRoles(
+                new UserRoles.Builder()
+                    .setIsAdmin(false)
+                    .setGlobalRole(GlobalRole.SUPPORT_AGENT)
+                    .build())
+            .build());
+
+    ConsoleUsersAction action =
+        createAction(
+            Optional.of(ConsoleApiParamsUtils.createFake(authResult)),
+            Optional.of("PUT"),
+            Optional.of(
+                new UserData(
+                    "support@test.com", null, RegistrarRole.ACCOUNT_MANAGER.toString(), null)));
+    action.run();
+    assertThat(response.getStatus()).isEqualTo(SC_FORBIDDEN);
+  }
+
   private ConsoleUsersAction createAction(
       Optional<ConsoleApiParams> maybeConsoleApiParams,
       Optional<String> method,

core/src/test/java/google/registry/ui/server/console/settings/SecurityActionTest.java

@@ -113,6 +113,31 @@ class SecurityActionTest extends ConsoleActionBaseTestCase {
     assertThat(history.getDescription()).hasValue("registrarId|IP_CHANGE,PRIMARY_SSL_CERT_CHANGE");
   }
 
+  @Test
+  void testSuccess_postRegistrarInfo_nullIpAllowList() throws IOException {
+    CertificateChecker lenientChecker =
+        new CertificateChecker(
+            ImmutableSortedMap.of(START_INSTANT, 20825, Instant.parse("2020-09-01T00:00:00Z"), 398),
+            30,
+            15,
+            2048,
+            ImmutableSet.of("secp256r1", "secp384r1"),
+            clock);
+
+    clock.setTo(Instant.parse("2020-11-01T00:00:00Z"));
+    String jsonWithNullIp =
+        String.format(
+            "{\"registrarId\": \"registrarId\", \"clientCertificate\": \"%s\","
+                + " \"ipAddressAllowList\": null}",
+            SAMPLE_CERT2);
+    SecurityAction action =
+        createAction(testRegistrar.getRegistrarId(), jsonWithNullIp, lenientChecker);
+    action.run();
+    assertThat(((FakeResponse) consoleApiParams.response()).getStatus()).isEqualTo(SC_OK);
+    Registrar r = loadRegistrar(testRegistrar.getRegistrarId());
+    assertThat(r.getIpAddressAllowList()).isEmpty();
+  }
+
   @Test
   void testFailure_validityPeriodTooLong_returnsSpecificError() throws IOException {
     CertificateChecker strictChecker =

db/src/main/resources/sql/er_diagram/brief_er_diagram.html

@@ -261,11 +261,11 @@ td.section {
     </tr>
     <tr>
      <td class="property_name">generated on</td>
-     <td class="property_value">2026-06-23 01:40:35</td>
+     <td class="property_value">2026-07-06 17:57:29</td>
     </tr>
     <tr>
      <td class="property_name">last flyway file</td>
-     <td id="lastFlywayFile" class="property_value">V223__tld_change_xap_enabled_to_transitions.sql</td>
+     <td id="lastFlywayFile" class="property_value">V224__add_registrar_expiry_access_period_enabled.sql</td>
     </tr>
    </tbody>
   </table>
@@ -273,7 +273,7 @@ td.section {
   <p>&nbsp;</p>
   <svg viewBox="0.00 0.00 4783.00 3613.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" id="erDiagram" style="overflow: hidden; width: 100%; height: 800px">
    <g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 3608.5)">
-    <title>SchemaCrawler_Diagram</title> <polygon fill="white" stroke="none" points="-4,4 -4,-3608.5 4778.75,-3608.5 4778.75,4 -4,4" /> <text xml:space="preserve" text-anchor="start" x="4535.5" y="-29.2" font-family="Helvetica,sans-Serif" font-size="14.00">generated by</text> <text xml:space="preserve" text-anchor="start" x="4618.25" y="-29.2" font-family="Helvetica,sans-Serif" font-size="14.00">SchemaCrawler 17.11.1</text> <text xml:space="preserve" text-anchor="start" x="4534.75" y="-9.45" font-family="Helvetica,sans-Serif" font-size="14.00">generated on</text> <text xml:space="preserve" text-anchor="start" x="4618.25" y="-9.45" font-family="Helvetica,sans-Serif" font-size="14.00">2026-06-23 01:40:35</text> <polygon fill="none" stroke="#888888" points="4531.75,-4 4531.75,-45.5 4766.75,-45.5 4766.75,-4 4531.75,-4" /> <!-- allocationtoken_a08ccbef -->
+    <title>SchemaCrawler_Diagram</title> <polygon fill="white" stroke="none" points="-4,4 -4,-3608.5 4778.75,-3608.5 4778.75,4 -4,4" /> <text xml:space="preserve" text-anchor="start" x="4535.5" y="-29.2" font-family="Helvetica,sans-Serif" font-size="14.00">generated by</text> <text xml:space="preserve" text-anchor="start" x="4618.25" y="-29.2" font-family="Helvetica,sans-Serif" font-size="14.00">SchemaCrawler 17.11.1</text> <text xml:space="preserve" text-anchor="start" x="4534.75" y="-9.45" font-family="Helvetica,sans-Serif" font-size="14.00">generated on</text> <text xml:space="preserve" text-anchor="start" x="4618.25" y="-9.45" font-family="Helvetica,sans-Serif" font-size="14.00">2026-07-06 17:57:29</text> <polygon fill="none" stroke="#888888" points="4531.75,-4 4531.75,-45.5 4766.75,-45.5 4766.75,-4 4531.75,-4" /> <!-- allocationtoken_a08ccbef -->
     <g id="node1" class="node">
      <title>allocationtoken_a08ccbef</title> <polygon fill="#e9c2f2" stroke="none" points="479.25,-1017.62 479.25,-1037.38 664.25,-1037.38 664.25,-1017.62 479.25,-1017.62" /> <text xml:space="preserve" text-anchor="start" x="481.25" y="-1023.08" font-family="Helvetica,sans-Serif" font-weight="bold" font-style="italic" font-size="14.00">public."AllocationToken"</text> <polygon fill="#e9c2f2" stroke="none" points="664.25,-1017.62 664.25,-1037.38 737.25,-1037.38 737.25,-1017.62 664.25,-1017.62" /> <text xml:space="preserve" text-anchor="start" x="698.5" y="-1022.08" font-family="Helvetica,sans-Serif" font-size="14.00">[table]</text> <text xml:space="preserve" text-anchor="start" x="481.25" y="-1003.33" font-family="Helvetica,sans-Serif" font-weight="bold" font-style="italic" font-size="14.00">token</text> <text xml:space="preserve" text-anchor="start" x="658.5" y="-1002.33" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text xml:space="preserve" text-anchor="start" x="666.25" y="-1002.33" font-family="Helvetica,sans-Serif" font-size="14.00">text not null</text> <text xml:space="preserve" text-anchor="start" x="481.25" y="-982.58" font-family="Helvetica,sans-Serif" font-size="14.00">domain_name</text> <text xml:space="preserve" text-anchor="start" x="658.5" y="-982.58" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text xml:space="preserve" text-anchor="start" x="666.25" y="-982.58" font-family="Helvetica,sans-Serif" font-size="14.00">text</text> <text xml:space="preserve" text-anchor="start" x="481.25" y="-962.83" font-family="Helvetica,sans-Serif" font-size="14.00">redemption_domain_repo_id</text> <text xml:space="preserve" text-anchor="start" x="658.5" y="-962.83" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text xml:space="preserve" text-anchor="start" x="666.25" y="-962.83" font-family="Helvetica,sans-Serif" font-size="14.00">text</text> <text xml:space="preserve" text-anchor="start" x="481.25" y="-943.08" font-family="Helvetica,sans-Serif" font-size="14.00">token_type</text> <text xml:space="preserve" text-anchor="start" x="658.5" y="-943.08" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text xml:space="preserve" text-anchor="start" x="666.25" y="-943.08" font-family="Helvetica,sans-Serif" font-size="14.00">text</text> <polygon fill="none" stroke="#888888" points="478.25,-937.62 478.25,-1038.38 738.25,-1038.38 738.25,-937.62 478.25,-937.62" />
     </g>

db/src/main/resources/sql/flyway.txt

@@ -221,3 +221,4 @@ V220__domain_package_token_idx.sql
 V221__remove_contact_history.sql
 V222__remove_contact.sql
 V223__tld_change_xap_enabled_to_transitions.sql
+V224__add_registrar_expiry_access_period_enabled.sql

db/src/main/resources/sql/flyway/V224__add_registrar_expiry_access_period_enabled.sql

@@ -0,0 +1,20 @@
+-- Copyright 2026 The Nomulus Authors. All Rights Reserved.
+--
+-- Licensed under the Apache License, Version 2.0 (the "License");
+-- you may not use this file except in compliance with the License.
+-- You may obtain a copy of the License at
+--
+--     http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+
+-- Add the XAP opt-in column to Registrar, defaulting to false for all existing registrars.
+-- To ensure backward compatibility with running servers (old Java code) during
+-- the transition phase of the deployment, we set DEFAULT false NOT NULL.
+-- TODO(mcilwain): Drop this DEFAULT constraint in a subsequent schema release once the Java code has been fully deployed.
+ALTER TABLE "Registrar" ADD COLUMN expiry_access_period_enabled boolean
+    DEFAULT false NOT NULL;

db/src/main/resources/sql/schema/nomulus.golden.sql

@@ -855,7 +855,8 @@ CREATE TABLE public."Registrar" (
     whois_server text,
     last_expiring_cert_notification_sent_date timestamp with time zone,
     last_expiring_failover_cert_notification_sent_date timestamp with time zone,
-    last_poc_verification_date timestamp with time zone
+    last_poc_verification_date timestamp with time zone,
+    expiry_access_period_enabled boolean DEFAULT false NOT NULL
 );
 
 

prober/.gitignore

@@ -1,2 +1,2 @@
 out/
-src/main/resources/google/registry/monitoring/blackbox/modules/secrets/
+src/main/resources/google/registry/monitoring/blackbox/module/secrets/