Require EPP validation for complex OT&E stats (#3107 )

Previously, if there were no XML bytes somehow, it'd increment a bunch of counts for requirements that shouldn't be incremented. We should only increment these values if we have the data to do so. Note: this doesn't happen and shouldn't happen, so there's no issues currently occurring, but this code is more correct anyway.
Add loadAllOfSorted to TransactionManager (#3136 )
2026-07-08 00:56:53 +00:00 · 2026-07-07 20:05:01 +00:00 · 2026-07-07 19:44:56 +00:00 · 2026-07-07 19:33:10 +00:00 · 2026-07-07 18:37:58 +00:00 · 2026-07-07 18:05:48 +00:00
57 changed files with 967 additions and 244 deletions
@@ -44,6 +44,11 @@ This document outlines foundational mandates, architectural patterns, and projec
        - **Test Helpers & Timestamps:** If a static test helper method (like in `DatabaseHelper`) needs the database transaction time but might be called from outside a transaction, using `tm().reTransact(tm()::getTxTime)` is acceptable. However, NEVER wrap it redundantly like `tm().transact(() -> tm().reTransact(tm()::getTxTime))`. If you are just setting an arbitrary timestamp in a test where the exact DB transaction time isn't strictly required, prefer `Instant.now()` or `clock.now()` to avoid creating unnecessary database transactions.
        - **Production Code:** In production code, if a flow fails because it is calling `getTxTime()` outside of a transaction, you must wrap the *caller* in a transaction instead of adding an unnecessary `reTransact()` around `getTxTime()`.
    - **Transactional Time:** Ensure code that relies on `tm().getTransactionTime()` (or `tm().getTxTime()`) is executed within a transaction context.
+- **Database Schema Migrations & 2-PR Split Mandate:**
+    - **Mandatory Consultation of `db/README.md`:** Before planning, drafting, or executing any database schema modifications (e.g., adding/altering columns, creating Flyway `.sql` migration scripts, or modifying JPA entity `@Column` mappings), you **MUST read and strictly adhere to `db/README.md`**.
+    - **Strict 2-PR Deployment Split:** Never propose or submit combining Flyway SQL scripts (`db/src/main/resources/sql/flyway/V*.sql`) and Java ORM changes (`.java` entity files + `db-schema.sql.generated`) into a single PR for submission to `master`. Because live servers during a rolling deployment will fail if Java code attempts to access unmigrated database columns/constraints, all schema additions must be split into two sequential PRs per `db/README.md`:
+        1. **PR #1 (Database Schema Only):** Contains *only* the new Flyway `.sql` script, the `flyway.txt` index update (`:db:generateFlywayIndex`), the `nomulus.golden.sql` dump (`:nom:generate_golden_file`), and ER diagrams (`er_diagram/`). Must contain **zero `.java` files or `db-schema.sql.generated` changes**.
+        2. **PR #2 (Java ORM, EPP Flows & Generated Schema Map):** Submitted *only after* PR #1 is deployed to production. Contains all `.java` entity/flow modifications, tests, and the regenerated `db-schema.sql.generated` (`generateSqlSchema`).

 ### 5. Testing Best Practices
 - **Mandatory Proactive Testing:** You MUST automatically write and update tests alongside your code changes WITHOUT waiting for the user to prompt you. If you add a new feature, fix a bug, or change core logic, you are explicitly required to identify the corresponding `*Test.java` file and implement comprehensive test coverage for your changes.
@@ -51,6 +56,7 @@ This document outlines foundational mandates, architectural patterns, and projec
 - **Empirical Reproduction:** Before fixing a bug, always create a test case that reproduces the failure.
 - **Base Classes:** Leverage `CommandTestCase`, `EppToolCommandTestCase`, etc., to reduce boilerplate and ensure consistent setup (e.g., clock initialization).
 - **Gradle Test Patterns:** When running tests to investigate fixes in the "core" directory, try to first use the "standardTest" Gradle task. It is faster than the "test" task, which includes the "fragileTest" task. Only run the full "test" task after "standardTest" succeeds.
+- **Mandatory SQL Integration Verification:** Whenever you modify any database schema, Flyway script (`.sql`), or JPA entity class, you MUST explicitly run both `./gradlew :db:test` and `./gradlew :core:sqlIntegrationTest` **in addition to** the standard test suites (`./gradlew standardTest` or `./gradlew test`) before finalizing the task or declaring completion. Do not rely solely on `standardTest` when database schemas or ORM mappings are touched; all three test suites (`standardTest`, `:db:test`, and `:core:sqlIntegrationTest`) are mandatory.

 ### 6. Project Dependencies
 - **Common Module:** When using `Clock` or other core utilities in a new or separate module (like `load-testing`), ensure `implementation project(':common')` is added to the module's `build.gradle`.
@@ -171,3 +177,4 @@ This protocol defines the standard for interacting with GitHub repositories and
 - **One Commit Per PR:** Ensure all changes are squashed into a single, clean commit. Use `git commit --amend --no-edit` for follow-up fixes.
 - **Clean Workspace:** Always run `git status` and verify the repository state before declaring a task complete.
 - **Package Lock:** The Gradle build automatically modifies `console-webapp/package-lock.json` via the `npmInstallDeps` task. ALWAYS revert this file (`git checkout console-webapp/package-lock.json`) before staging changes or finalizing a commit unless you explicitly modified NPM dependencies.
+- **PR Description Synchronization:** Whenever you amend or update a commit's description, you **MUST** check whether the corresponding GitHub PR description (if one exists) matches the previous commit description. If the PR description was not manually customized (i.e., it simply reflects the older commit description), you must automatically update it via `gh pr edit` to keep it in sync with your newly updated commit description—**strictly after the updated commit has been pushed to the remote GitHub PR branch**. Do not update the remote PR description when changes are only committed locally. **CRITICAL:** When updating the PR description, you must strictly preserve any automated review links or footer blocks (such as Reviewable link blocks: `<!-- Reviewable:start -->...<!-- Reviewable:end -->`) at the bottom of the description.
@@ -21,27 +21,23 @@ clean {

 task npmInstallDeps(type: Exec) {
  workingDir "${consoleDir}/"
-  executable 'npm'
-  args 'i', '--no-audit', '--no-fund', '--loglevel=error'
+  commandLine 'sh', '-c', 'npm i --no-audit --no-fund --loglevel=error'
 }

 task runConsoleWebappLocally(type: Exec) {
  workingDir "${consoleDir}/"
-  executable 'npm'
-  args 'run', 'start:dev'
+  commandLine 'sh', '-c', 'npm run start:dev'
 }

 task runConsoleWebappUnitTests(type: Exec) {
  workingDir "${consoleDir}/"
-  executable 'npm'
-  args 'run', 'test'
+  commandLine 'sh', '-c', 'npm run test'
 }

 task buildConsoleWebapp(type: Exec) {
  workingDir "${consoleDir}/"
-  executable 'npx'
  def configuration = project.getProperty('configuration')
-  args 'ng', 'build', '--base-href=/console/', "--configuration=${configuration}", "--output-path=staged/dist"
+  commandLine 'sh', '-c', "npx ng build --base-href=/console/ --configuration=${configuration} --output-path=staged/dist"
  doFirst {
    println "Building console for environment: ${configuration}"
  }
@@ -52,8 +48,7 @@ task buildConsoleForAll() {}
 def createConsoleTask = { env ->
  project.tasks.register("buildConsoleFor${env.capitalize()}", Exec) {
    workingDir "${consoleDir}/"
-    executable 'npx'
-    args 'ng', 'build', '--base-href=/console/', "--configuration=${env}"
+    commandLine 'sh', '-c', "npx ng build --base-href=/console/ --configuration=${env}"
    doFirst {
      println "Building console for environment: ${env}"
    }
@@ -91,14 +86,12 @@ tasks.buildConsoleWebapp.mustRunAfter(tasks.buildConsoleForProduction)

 task applyFormatting(type: Exec) {
  workingDir "${consoleDir}/"
-  executable 'npm'
-  args 'run', 'prettify'
+  commandLine 'sh', '-c', 'npm run prettify'
 }

 task checkFormatting(type: Exec) {
  workingDir "${consoleDir}/"
-  executable 'npm'
-  args 'run', 'prettify:check'
+  commandLine 'sh', '-c', 'npm run prettify:check'
 }

 tasks.buildConsoleWebapp.dependsOn(tasks.npmInstallDeps)
@@ -75,7 +75,7 @@ export class RegistrarDetailsComponent implements OnInit {
  }

  checkOteStatus() {
-    this.router.navigate(['ote-status/', this.registrarInEdit.registrarId], {
+    this.router.navigate(['ote-status', this.registrarInEdit.registrarId], {
      queryParamsHandling: 'merge',
    });
  }
@@ -226,18 +226,17 @@ public class SafeBrowsingTransforms {
    private void processResponse(
        CloseableHttpResponse response,
        ImmutableSet.Builder<KV<DomainNameInfo, ThreatMatch>> resultBuilder)
-        throws JSONException, IOException {
+        throws IOException {
      int statusCode = response.getStatusLine().getStatusCode();
      if (statusCode != SC_OK) {
-        logger.atWarning().log("Got unexpected status code %s from response.", statusCode);
-      } else {
-        // Unpack the response body
-        JSONObject responseBody =
-            new JSONObject(
-                CharStreams.toString(
-                    new InputStreamReader(response.getEntity().getContent(), UTF_8)));
-        logger.atInfo().log("Got response: %s", responseBody);
-        if (responseBody.length() == 0) {
+        throw new IOException(
+            String.format("Got unexpected status code %s from response.", statusCode));
+      }
+      // Unpack the response body
+      try (InputStreamReader reader =
+          new InputStreamReader(response.getEntity().getContent(), UTF_8)) {
+        JSONObject responseBody = new JSONObject(CharStreams.toString(reader));
+        if (responseBody.isEmpty()) {
          logger.atInfo().log("Response was empty, no threats detected.");
        } else {
          // Emit all DomainNameInfos with their API results.
@@ -14,12 +14,15 @@

 package google.registry.gcs;

+import static com.google.common.base.Preconditions.checkState;
 import static com.google.common.collect.ImmutableList.toImmutableList;
 import static com.google.common.collect.Iterables.getLast;

 import com.google.cloud.storage.Blob;
 import com.google.cloud.storage.BlobId;
 import com.google.cloud.storage.BlobInfo;
+import com.google.cloud.storage.Bucket;
+import com.google.cloud.storage.BucketInfo;
 import com.google.cloud.storage.Storage;
 import com.google.cloud.storage.Storage.BlobListOption;
 import com.google.cloud.storage.StorageException;
@@ -33,12 +36,14 @@ import com.google.common.flogger.FluentLogger;
 import com.google.common.net.MediaType;
 import google.registry.config.CredentialModule.ApplicationDefaultCredential;
 import google.registry.util.GoogleCredentialsBundle;
+import google.registry.util.RegistryEnvironment;
 import jakarta.inject.Inject;
 import java.io.IOException;
 import java.io.InputStream;
 import java.io.OutputStream;
 import java.io.Serializable;
 import java.nio.channels.Channels;
+import java.util.concurrent.ConcurrentHashMap;
 import javax.annotation.CheckReturnValue;

 /**
@@ -50,6 +55,8 @@ public class GcsUtils implements Serializable {

  private static final FluentLogger logger = FluentLogger.forEnclosingClass();

+  private static final ConcurrentHashMap<String, Boolean> PAP_CACHE = new ConcurrentHashMap<>();
+
  private static final ImmutableMap<String, MediaType> EXTENSIONS =
      new ImmutableMap.Builder<String, MediaType>()
          .put("ghostryde", MediaType.APPLICATION_BINARY)
@@ -85,6 +92,7 @@ public class GcsUtils implements Serializable {
  /** Opens a GCS file for writing as an {@link OutputStream}, overwriting existing files. */
  @CheckReturnValue
  public OutputStream openOutputStream(BlobId blobId) {
+    verifyPublicAccessPrevention(blobId.getBucket());
    return Channels.newOutputStream(storage().writer(createBlobInfo(blobId)));
  }

@@ -94,6 +102,7 @@ public class GcsUtils implements Serializable {
   */
  @CheckReturnValue
  public OutputStream openOutputStream(BlobId blobId, ImmutableMap<String, String> metadata) {
+    verifyPublicAccessPrevention(blobId.getBucket());
    return Channels.newOutputStream(
        storage().writer(BlobInfo.newBuilder(blobId).setMetadata(metadata).build()));
  }
@@ -105,6 +114,7 @@ public class GcsUtils implements Serializable {

  /** Creates a GCS file with the given byte contents and metadata, overwriting existing files. */
  public void createFromBytes(BlobInfo blobInfo, byte[] bytes) throws StorageException {
+    verifyPublicAccessPrevention(blobInfo.getBucket());
    storage().create(blobInfo, bytes);
  }

@@ -120,6 +130,7 @@ public class GcsUtils implements Serializable {

  /** Update file content type on existing GCS file */
  public void updateContentType(BlobId blobId, String contentType) throws StorageException {
+    verifyPublicAccessPrevention(blobId.getBucket());
    if (existsAndNotEmpty(blobId)) {
      Blob blob = storage().get(blobId);
      blob.toBuilder().setContentType(contentType).build().update();
@@ -154,12 +165,6 @@ public class GcsUtils implements Serializable {
    }
  }

-  /** Returns the user defined metadata of a GCS file if the file exists, or an empty map. */
-  public ImmutableMap<String, String> getMetadata(BlobId blobId) throws StorageException {
-    Blob blob = storage().get(blobId);
-    return blob == null ? ImmutableMap.of() : ImmutableMap.copyOf(blob.getMetadata());
-  }
-
  /**
   * Returns the {@link BlobInfo} of the given GCS file.
   *
@@ -179,6 +184,37 @@ public class GcsUtils implements Serializable {
    return builder.build();
  }

+  /**
+   * Asserts that Public Access Prevention (PAP) is enforced on the GCS bucket.
+   *
+   * @throws IllegalStateException if PAP is not ENFORCED.
+   */
+  @VisibleForTesting
+  void verifyPublicAccessPrevention(String bucketName) {
+    if (RegistryEnvironment.get() != RegistryEnvironment.PRODUCTION) {
+      return;
+    }
+    PAP_CACHE.computeIfAbsent(
+        bucketName,
+        name -> {
+          Bucket bucket = storage().get(name);
+          checkState(bucket != null, "Bucket %s does not exist", name);
+          BucketInfo.PublicAccessPrevention pap =
+              bucket.getIamConfiguration().getPublicAccessPrevention();
+          checkState(
+              pap == BucketInfo.PublicAccessPrevention.ENFORCED,
+              "Public Access Prevention is not enforced on bucket %s. Current state: %s",
+              name,
+              pap);
+          return true;
+        });
+  }
+
+  @VisibleForTesting
+  static void clearPapCache() {
+    PAP_CACHE.clear();
+  }
+
  // These two methods are needed to check whether serialization is done correctly in tests.
  @Override
  public boolean equals(Object obj) {
@@ -176,11 +176,12 @@ public class OteStats {
     * Check if the {@link HistoryEntry} type matches as well as the {@link EppInput} if supplied.
     */
    private boolean matches(HistoryEntry.Type historyType, Optional<EppInput> eppInput) {
-      if (eppInputFilter.isPresent() && eppInput.isPresent()) {
-        return typeFilter.test(historyType) && eppInputFilter.get().test(eppInput.get());
-      } else {
-        return typeFilter.test(historyType);
+      if (!typeFilter.test(historyType)) {
+        return false;
      }
+      return eppInputFilter
+          .map(filter -> eppInput.isPresent() && filter.test(eppInput.get()))
+          .orElse(true);
    }
  }

@@ -158,12 +158,12 @@ public class DomainCommand {
        throws InvalidReferencesException, ParameterValuePolicyErrorException {
      Create clone = clone(this);
      clone.nameservers = linkHosts(nullSafeImmutableCopy(clone.nameserverHostNames), now);
-      if (registrantContactId != null) {
-        throw new RegistrantProhibitedException();
-      }
      if (!isNullOrEmpty(foreignKeyedDesignatedContacts)) {
        throw new ContactsProhibitedException();
      }
+      if (registrantContactId != null) {
+        throw new RegistrantProhibitedException();
+      }
      return clone;
    }

@@ -1030,6 +1030,11 @@ public class Registrar extends UpdateAutoTimestampEntity implements Buildable, J
    return tm().transact(() -> tm().loadAllOf(Registrar.class));
  }

+  /** Loads all registrar entities directly from the database, sorted by the given field names. */
+  public static Iterable<Registrar> loadAllSorted(String... sortFields) {
+    return tm().transact(() -> tm().loadAllOfSorted(Registrar.class, sortFields));
+  }
+
  /** Loads all registrar entities using an in-memory cache. */
  public static Iterable<Registrar> loadAllCached() {
    return CACHE_BY_REGISTRAR_ID.get().values();
@@ -71,6 +71,8 @@ import google.registry.model.domain.token.VKeyConverter_AllocationToken;
 import google.registry.model.tld.label.PremiumList;
 import google.registry.model.tld.label.ReservedList;
 import google.registry.persistence.EntityCallbacksListener.RecursivePostPersist;
+import google.registry.persistence.EntityCallbacksListener.RecursivePostRemove;
+import google.registry.persistence.EntityCallbacksListener.RecursivePostUpdate;
 import google.registry.persistence.VKey;
 import google.registry.persistence.converter.AllocationTokenVkeyListUserType;
 import google.registry.persistence.converter.BillingCostTransitionUserType;
@@ -219,9 +221,12 @@ public class Tld extends ImmutableObject implements Buildable, UnsafeSerializabl
   * Invalidates the cache entry.
   *
   * <p>This is called automatically when the tld is saved. One should also call it when a tld is
-   * deleted.
+   * deleted. This only affects the pod-local cache so most pods won't catch it, but it's still the
+   * right thing to do.
   */
  @RecursivePostPersist
+  @RecursivePostRemove
+  @RecursivePostUpdate
  public void invalidateInCache() {
    CACHE.invalidate(tldStr);
  }
@@ -169,6 +169,14 @@ public final class PremiumList extends BaseDomainLabelList<BigDecimal, PremiumEn
        getInstance().revisionId = revisionId;
        return this;
      }
+
+      @Override
+      public PremiumEntry build() {
+        checkArgument(getInstance().price != null, "Price must not be null");
+        checkArgument(
+            getInstance().price.compareTo(BigDecimal.ZERO) >= 0, "Price must not be negative");
+        return super.build();
+      }
    }
  }

@@ -164,7 +164,7 @@ import google.registry.ui.server.console.settings.SecurityAction;
      ToolsServerModule.class,
      WhiteboxModule.class
    })
-interface RequestComponent {
+public interface RequestComponent {
  FlowComponent.Builder flowComponentBuilder();

  BrdaCopyAction brdaCopyAction();
@@ -257,11 +257,21 @@ public class DelegatingReplicaJpaTransactionManager implements JpaTransactionMan
    return getReplica().loadAllOf(clazz);
  }

+  @Override
+  public <T> ImmutableList<T> loadAllOfSorted(Class<T> clazz, String... sortFields) {
+    return getReplica().loadAllOfSorted(clazz, sortFields);
+  }
+
  @Override
  public <T> Stream<T> loadAllOfStream(Class<T> clazz) {
    return getReplica().loadAllOfStream(clazz);
  }

+  @Override
+  public <T> Stream<T> loadAllOfSortedStream(Class<T> clazz, String... sortFields) {
+    return getReplica().loadAllOfSortedStream(clazz, sortFields);
+  }
+
  @Override
  public <T> Optional<T> loadSingleton(Class<T> clazz) {
    return getReplica().loadSingleton(clazz);
@@ -76,6 +76,7 @@ import java.util.concurrent.TimeUnit;
 import java.util.concurrent.atomic.AtomicLong;
 import java.util.function.Supplier;
 import java.util.function.UnaryOperator;
+import java.util.regex.Pattern;
 import java.util.stream.Stream;
 import java.util.stream.StreamSupport;
 import javax.annotation.Nullable;
@@ -88,6 +89,16 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {

  private static final FluentLogger logger = FluentLogger.forEnclosingClass();
  private static final Retrier retrier = new Retrier(new SystemSleeper(), 6);
+
+  /**
+   * Strict allowlist regex for property/field names in dynamic JPQL ORDER BY clauses.
+   *
+   * <p>JPA and database engines forbid bind parameters (e.g. ? or :param) for schema identifiers or
+   * property names in ORDER BY clauses. To prevent JPQL/SQL injection when dynamically constructing
+   * sort queries, every sort field MUST be validated against this pattern before concatenation.
+   */
+  private static final Pattern VALID_SORT_FIELD_PATTERN = Pattern.compile("^[a-zA-Z0-9_.]+$");
+
  private static final String NESTED_TRANSACTION_MESSAGE =
      "Nested transaction detected. Try refactoring to avoid nested transactions. If unachievable,"
          + " use reTransact() in nested transactions";
@@ -528,15 +539,38 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {

  @Override
  public <T> ImmutableList<T> loadAllOf(Class<T> clazz) {
-    return loadAllOfStream(clazz).collect(toImmutableList());
+    return loadAllOfSorted(clazz);
  }

  @Override
  public <T> Stream<T> loadAllOfStream(Class<T> clazz) {
+    return loadAllOfSortedStream(clazz);
+  }
+
+  @Override
+  public <T> ImmutableList<T> loadAllOfSorted(Class<T> clazz, String... sortFields) {
+    return loadAllOfSortedStream(clazz, sortFields).collect(toImmutableList());
+  }
+
+  @Override
+  public <T> Stream<T> loadAllOfSortedStream(Class<T> clazz, String... sortFields) {
    checkArgumentNotNull(clazz, "clazz must be specified");
+    checkArgumentNotNull(sortFields, "sortFields must not be null");
    assertInTransaction();
+    StringBuilder queryString =
+        new StringBuilder(String.format("FROM %s", getEntityType(clazz).getName()));
+    if (sortFields.length > 0) {
+      for (String field : sortFields) {
+        checkArgument(
+            VALID_SORT_FIELD_PATTERN.matcher(field).matches(),
+            "Invalid sort field name: %s",
+            field);
+      }
+      queryString.append(" ORDER BY ");
+      queryString.append(String.join(", ", sortFields));
+    }
    return getEntityManager()
-        .createQuery(String.format("FROM %s", getEntityType(clazz).getName()), clazz)
+        .createQuery(queryString.toString(), clazz)
        .getResultStream()
        .map(this::detach);
  }
@@ -219,6 +219,14 @@ public interface TransactionManager {
   */
  <T> ImmutableList<T> loadAllOf(Class<T> clazz);

+  /**
+   * Returns a list of all entities of the given type that exist in the database, ordered by the
+   * specified field names in ascending order.
+   *
+   * <p>The resulting list is empty if there are no entities of this type.
+   */
+  <T> ImmutableList<T> loadAllOfSorted(Class<T> clazz, String... sortFields);
+
  /**
   * Returns a stream of all entities of the given type that exist in the database.
   *
@@ -226,6 +234,14 @@ public interface TransactionManager {
   */
  <T> Stream<T> loadAllOfStream(Class<T> clazz);

+  /**
+   * Returns a stream of all entities of the given type that exist in the database, ordered by the
+   * specified field names in ascending order.
+   *
+   * <p>The resulting stream is empty if there are no entities of this type.
+   */
+  <T> Stream<T> loadAllOfSortedStream(Class<T> clazz, String... sortFields);
+
  /**
   * Loads the only instance of this particular class, or empty if none exists.
   *
@@ -14,10 +14,14 @@

 package google.registry.reporting.billing;

+import static com.google.common.base.Preconditions.checkState;
 import static google.registry.beam.BeamUtils.createJobName;
+import static google.registry.model.common.Cursor.CursorType.RECURRING_BILLING;
+import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
 import static google.registry.request.Action.Method.POST;
 import static jakarta.servlet.http.HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
 import static jakarta.servlet.http.HttpServletResponse.SC_OK;
+import static java.time.ZoneOffset.UTC;

 import com.google.api.services.dataflow.Dataflow;
 import com.google.api.services.dataflow.model.LaunchFlexTemplateParameter;
@@ -29,6 +33,7 @@ import com.google.common.flogger.FluentLogger;
 import com.google.common.net.MediaType;
 import google.registry.batch.CloudTasksUtils;
 import google.registry.config.RegistryConfig.Config;
+import google.registry.model.common.Cursor;
 import google.registry.persistence.PersistenceModule;
 import google.registry.reporting.ReportingModule;
 import google.registry.request.Action;
@@ -40,7 +45,9 @@ import google.registry.util.RegistryEnvironment;
 import jakarta.inject.Inject;
 import java.io.IOException;
 import java.time.Duration;
+import java.time.Instant;
 import java.time.YearMonth;
+import java.util.Optional;

 /**
 * Invokes the {@code InvoicingPipeline} beam template via the REST api, and enqueues the {@link
@@ -107,6 +114,7 @@ public class GenerateInvoicesAction implements Runnable {
    response.setContentType(MediaType.PLAIN_TEXT_UTF_8);
    logger.atInfo().log("Launching invoicing pipeline for %s.", yearMonth);
    try {
+      checkBillingRecurrenceCursor();
      LaunchFlexTemplateParameter parameter =
          new LaunchFlexTemplateParameter()
              .setJobName(createJobName("invoicing", clock))
@@ -156,4 +164,20 @@ public class GenerateInvoicesAction implements Runnable {
      response.setPayload(String.format("Pipeline launch failed: %s", e.getMessage()));
    }
  }
+
+  private void checkBillingRecurrenceCursor() {
+    Optional<Cursor> previousCursor =
+        tm().transact(() -> tm().loadByKeyIfPresent(Cursor.createGlobalVKey(RECURRING_BILLING)));
+    checkState(
+        previousCursor.isPresent(),
+        "BillingRecurrence expansion cursor is not present. Run ExpandBillingRecurrencesAction.");
+    Instant startOfNextMonth = yearMonth.plusMonths(1).atDay(1).atStartOfDay(UTC).toInstant();
+    Instant previousCursorTime = previousCursor.get().getCursorTime();
+    checkState(
+        !previousCursorTime.isBefore(startOfNextMonth),
+        "BillingRecurrence expansion cursor (%s) is before the start of the next month (%s). "
+            + "Run ExpandBillingRecurrencesAction.",
+        previousCursorTime,
+        startOfNextMonth);
+  }
 }
@@ -14,11 +14,9 @@

 package google.registry.request;

-import static com.google.common.collect.ImmutableList.toImmutableList;
 import static java.util.stream.Collectors.joining;

 import com.google.common.base.Joiner;
-import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableMap;
 import com.google.common.collect.Streams;
 import java.util.Comparator;
@@ -64,18 +62,6 @@ public class RouterDisplayHelper {
    return formatRoutes(Router.extractRoutesFromComponent(componentClass).values());
  }

-  public static ImmutableList<String> extractHumanReadableRoutesWithWrongService(
-      Class<?> componentClass, Action.Service expectedService) {
-    return Router.extractRoutesFromComponent(componentClass).values().stream()
-        .filter(route -> route.action().service() != expectedService)
-        .map(
-            route ->
-                String.format(
-                    "%s (%s%s)",
-                    route.actionClass(), route.action().service(), route.action().path()))
-        .collect(toImmutableList());
-  }
-
  private static String getFormatString(Map<String, Integer> columnWidths) {
    return String.format(
        FORMAT,
@@ -14,12 +14,14 @@

 package google.registry.tmch;

+import static com.google.common.base.Preconditions.checkArgument;
 import static google.registry.request.UrlConnectionUtils.getResponseBytes;
 import static jakarta.servlet.http.HttpServletResponse.SC_NO_CONTENT;
 import static jakarta.servlet.http.HttpServletResponse.SC_OK;
 import static java.nio.charset.StandardCharsets.UTF_8;

 import com.google.common.annotations.VisibleForTesting;
+import com.google.common.base.Ascii;
 import com.google.common.flogger.FluentLogger;
 import com.google.common.io.ByteSource;
 import google.registry.request.Action;
@@ -62,6 +64,8 @@ public final class NordnVerifyAction implements Runnable {
  static final String NORDN_URL_PARAM = "nordnUrl";
  static final String NORDN_LOG_ID_PARAM = "nordnLogId";

+  private static final String MARKSDB_URL_BEGINNING = "ry.marksdb.org";
+
  private static final FluentLogger logger = FluentLogger.forEnclosingClass();

  @Inject LordnRequestInitializer lordnRequestInitializer;
@@ -104,6 +108,12 @@ public final class NordnVerifyAction implements Runnable {
   */
  @VisibleForTesting
  LordnLog verify() throws IOException, GeneralSecurityException {
+    String host = Ascii.toLowerCase(url.getHost());
+    checkArgument(
+        host.startsWith(MARKSDB_URL_BEGINNING),
+        "URL %s must start with %s",
+        url,
+        MARKSDB_URL_BEGINNING);
    logger.atInfo().log("LORDN verify task %s: Sending request to URL %s", actionLogId, url);
    HttpURLConnection connection = urlConnectionService.createConnection(url);
    lordnRequestInitializer.initialize(connection, tld);
@@ -25,6 +25,7 @@ import google.registry.request.HttpException.BadRequestException;
 import google.registry.request.Parameter;
 import jakarta.servlet.http.HttpServletRequest;
 import java.net.MalformedURLException;
+import java.net.URI;
 import java.net.URL;
 import org.bouncycastle.openpgp.PGPPublicKey;

@@ -53,7 +54,7 @@ public final class TmchModule {
  @Parameter(NordnVerifyAction.NORDN_URL_PARAM)
  static URL provideNordnUrl(HttpServletRequest req) {
    try {
-      return new URL(extractRequiredParameter(req, NordnVerifyAction.NORDN_URL_PARAM));
+      return URI.create(extractRequiredParameter(req, NordnVerifyAction.NORDN_URL_PARAM)).toURL();
    } catch (MalformedURLException e) {
      throw new BadRequestException("Bad URL: " + NordnVerifyAction.NORDN_URL_PARAM);
    }
@@ -14,27 +14,17 @@

 package google.registry.tools;

-import com.beust.jcommander.Parameter;
 import com.beust.jcommander.Parameters;
+import google.registry.module.RequestComponent;
 import google.registry.request.RouterDisplayHelper;

 /** Generates the routing map file used for unit testing. */
@Parameters(commandDescription = "Generate a routing map file")
 final class GetRoutingMapCommand implements Command {

-  @Parameter(
-    names = {"-c", "--class"},
-    description =
-        "Request component class (e.g. google.registry.module.backend.BackendRequestComponent)"
-        + " for which routing map should be generated",
-    required = true
-  )
-  private String serviceClassName;
-
  @Override
  public void run() throws Exception {
    System.out.println(
-        RouterDisplayHelper.extractHumanReadableRoutesFromComponent(
-            Class.forName(serviceClassName)));
+        RouterDisplayHelper.extractHumanReadableRoutesFromComponent(RequestComponent.class));
  }
 }
@@ -33,6 +33,7 @@ import google.registry.model.domain.Domain;
 import google.registry.model.domain.secdns.DomainDsData;
 import google.registry.model.host.Host;
 import google.registry.model.tld.Tld;
+import google.registry.model.tld.Tlds;
 import google.registry.request.Action;
 import google.registry.request.HttpException.BadRequestException;
 import google.registry.request.JsonActionRunner;
@@ -119,6 +120,7 @@ public class GenerateZoneFilesAction implements Runnable, JsonActionRunner.JsonA
  public Map<String, Object> handleJsonRequest(Map<String, ?> json) {
    @SuppressWarnings("unchecked")
    ImmutableSet<String> tlds = ImmutableSet.copyOf((List<String>) json.get("tlds"));
+    Tlds.assertTldsExist(tlds);
    Instant exportTime = Instant.parse(json.get("exportTime").toString());
    // We disallow exporting within the past 2 minutes because there might be outstanding writes.
    // We can only reliably call loadAtPointInTime at times that are UTC midnight and >
@@ -105,6 +105,7 @@ public class RefreshDnsForAllDomainsAction implements Runnable {

  @Override
  public void run() {
+    checkArgument(!tlds.isEmpty(), "Must specify TLDs to refresh");
    assertTldsExist(tlds);
    checkArgument(batchSize > 0, "Must specify a positive number for batch size");
    logger.atInfo().log("Enqueueing DNS refresh tasks for TLDs %s.", tlds);
@@ -37,6 +37,7 @@ import com.google.gson.annotations.Expose;
 import google.registry.config.RegistryConfig.Config;
 import google.registry.model.console.ConsolePermission;
 import google.registry.model.console.ConsoleUpdateHistory;
+import google.registry.model.console.GlobalRole;
 import google.registry.model.console.RegistrarRole;
 import google.registry.model.console.User;
 import google.registry.model.console.UserRoles;
@@ -149,10 +150,15 @@ public class ConsoleUsersAction extends ConsoleApiAction {
      throw new BadRequestException("Total users amount per registrar is limited to 4");
    }

+    User userToAppend = verifyUserExists(this.userData.get().emailAddress);
+    if (userToAppend.getUserRoles().isAdmin()
+        || !userToAppend.getUserRoles().getGlobalRole().equals(GlobalRole.NONE)) {
+      throw new BadRequestException(
+          "Cannot append a global administrator or user with a global role to a registrar");
+    }
+
    updateUserRegistrarRoles(
-        this.userData.get().emailAddress,
-        registrarId,
-        requestRoleToAllowedRoles(this.userData.get().role));
+        userToAppend, registrarId, requestRoleToAllowedRoles(this.userData.get().role));

    sendConfirmationEmail(registrarId, this.userData.get().emailAddress, "Added existing user");
    consoleApiParams.response().setStatus(SC_OK);
@@ -164,7 +170,14 @@ public class ConsoleUsersAction extends ConsoleApiAction {
    }

    String email = this.userData.get().emailAddress;
-    User updatedUser = updateUserRegistrarRoles(email, registrarId, null);
+    User userToDelete = verifyUserExists(email);
+    if (userToDelete.getUserRoles().isAdmin()
+        || !userToDelete.getUserRoles().getGlobalRole().equals(GlobalRole.NONE)) {
+      throw new BadRequestException(
+          "Cannot delete a global administrator or user with a global role");
+    }
+
+    User updatedUser = updateUserRegistrarRoles(userToDelete, registrarId, null);

    // User has no registrars assigned
    if (updatedUser.getUserRoles().getRegistrarRoles().isEmpty()) {
@@ -251,7 +264,7 @@ public class ConsoleUsersAction extends ConsoleApiAction {
    }

    updateUserRegistrarRoles(
-        this.userData.get().emailAddress,
+        verifyUserExists(this.userData.get().emailAddress),
        registrarId,
        requestRoleToAllowedRoles(this.userData.get().role));

@@ -297,9 +310,8 @@ public class ConsoleUsersAction extends ConsoleApiAction {
    return false;
  }

-  private User updateUserRegistrarRoles(String email, String registrarId, RegistrarRole newRole) {
+  private User updateUserRegistrarRoles(User user, String registrarId, RegistrarRole newRole) {
    Map<String, RegistrarRole> updatedRegistrarRoles;
-    User user = verifyUserExists(email);
    if (newRole == null) {
      updatedRegistrarRoles =
          user.getUserRoles().getRegistrarRoles().entrySet().stream()
@@ -59,6 +59,7 @@ public class RegistrarsAction extends ConsoleApiAction {
      """
            SELECT * FROM "Registrar"
            WHERE registrar_id in :registrarIds
+            ORDER BY registrar_name ASC, registrar_id ASC
      """;
  static final String PATH = "/console-api/registrars";
  private final Optional<Registrar> registrar;
@@ -83,7 +84,7 @@ public class RegistrarsAction extends ConsoleApiAction {
      ImmutableSet<Registrar.Type> allowedRegistrarTypes =
          user.getUserRoles().isAdmin() ? TYPES_ALLOWED_FOR_ADMINS : TYPES_ALLOWED_FOR_USERS;
      ImmutableList<Registrar> registrars =
-          Streams.stream(Registrar.loadAll())
+          Streams.stream(Registrar.loadAllSorted("registrarName", "registrarId"))
              .filter(r -> allowedRegistrarTypes.contains(r.getType()))
              .collect(ImmutableList.toImmutableList());
      consoleApiParams.response().setPayload(consoleApiParams.gson().toJson(registrars));
@@ -115,6 +115,10 @@ public class XmlTransformer {
    // Prevent XXE attacks.
    xmlInputFactory.setProperty(XMLInputFactory.IS_SUPPORTING_EXTERNAL_ENTITIES, false);
    xmlInputFactory.setProperty(XMLInputFactory.SUPPORT_DTD, false);
+    xmlInputFactory.setXMLResolver(
+        (publicID, systemID, baseURI, namespace) -> {
+          throw new XMLStreamException("Entity resolution disabled.");
+        });
    return xmlInputFactory;
  }

@@ -47,6 +47,8 @@ Child elements of the <create> command.
    minOccurs="0"/>
   <element name="registrant" type="eppcom:clIDType"
    minOccurs="0"/>
+   <element name="contact" type="domain:contactType"
+    minOccurs="0" maxOccurs="unbounded"/>
   <element name="authInfo" type="domain:authInfoType"/>
 </sequence>
 </complexType>
@@ -98,6 +100,22 @@ If attributes, addresses are optional and follow the
 structure defined in the host mapping.
 -->

+<complexType name="contactType">
+ <simpleContent>
+   <extension base="eppcom:clIDType">
+     <attribute name="type" type="domain:contactAttrType"/>
+   </extension>
+ </simpleContent>
+</complexType>
+
+<simpleType name="contactAttrType">
+ <restriction base="token">
+   <enumeration value="admin"/>
+   <enumeration value="billing"/>
+   <enumeration value="tech"/>
+ </restriction>
+</simpleType>
+
 <complexType name="authInfoType">
 <choice>
   <element name="pw" type="eppcom:pwAuthInfoType"/>
@@ -198,6 +216,8 @@ Data elements that can be added or removed.
 <sequence>
   <element name="ns" type="domain:nsType"
    minOccurs="0"/>
+   <element name="contact" type="domain:contactType"
+    minOccurs="0" maxOccurs="unbounded"/>
   <element name="status" type="domain:statusType"
    minOccurs="0" maxOccurs="11"/>
 </sequence>
@@ -299,6 +319,8 @@ Child response elements.
    minOccurs="0" maxOccurs="11"/>
   <element name="registrant" type="eppcom:clIDType"
    minOccurs="0"/>
+   <element name="contact" type="domain:contactType"
+    minOccurs="0" maxOccurs="unbounded"/>
   <element name="ns" type="domain:nsType"
    minOccurs="0"/>
   <element name="host" type="eppcom:labelType"
@@ -60,6 +60,9 @@
            maxOccurs="unbounded"/>
          <element name="registrant"
            type="eppcom:clIDType" minOccurs="0"/>
+          <element name="contact"
+            type="domain:contactType"
+            minOccurs="0" maxOccurs="unbounded"/>
          <element name="ns"
            type="domain:nsType" minOccurs="0"/>
          <element name="clID"
@@ -30,7 +30,7 @@ JOIN Registrar r ON b.clientId = r.registrarId
 JOIN Domain d ON b.domainRepoId = d.repoId
 JOIN Tld t ON t.tldStr = d.tld
 LEFT JOIN BillingCancellation c ON b.id = c.billingEvent
-LEFT JOIN BillingCancellation cr ON b.cancellationMatchingBillingEvent = cr.billingRecurrence
+LEFT JOIN BillingCancellation cr ON b.cancellationMatchingBillingEvent = cr.billingRecurrence AND b.billingTime = cr.billingTime
 WHERE r.billingAccountMap IS NOT NULL
 AND r.type = 'REAL'
 AND t.invoicingEnabled IS TRUE
@@ -229,7 +229,22 @@ class InvoicingPipelineTest {
              3,
              "USD",
              20.5,
-              ""));
+              ""),
+          google.registry.beam.billing.BillingEvent.create(
+              17,
+              Instant.parse("2017-10-04T00:00:00Z"),
+              Instant.parse("2017-10-04T00:00:00Z"),
+              "theRegistrar",
+              "234",
+              "",
+              "test",
+              "RENEW",
+              "recurrence-collision.test",
+              "REPO-ID",
+              3,
+              "USD",
+              20.5,
+              "SYNTHETIC"));

  private static final ImmutableMap<String, ImmutableList<String>> EXPECTED_DETAILED_REPORT_MAP =
      ImmutableMap.of(
@@ -239,6 +254,8 @@ class InvoicingPipelineTest {
                  + "test,RENEW,mydomain2.test,REPO-ID,3,USD,20.50,",
              "1,2017-10-04 00:00:00 UTC,2017-10-04 00:00:00 UTC,theRegistrar,234,,"
                  + "test,RENEW,mydomain.test,REPO-ID,3,USD,20.50,",
+              "17,2017-10-04 00:00:00 UTC,2017-10-04 00:00:00 UTC,theRegistrar,234,,"
+                  + "test,RENEW,recurrence-collision.test,REPO-ID,3,USD,20.50,",
              "7,2017-10-04 00:00:00 UTC,2017-10-04 00:00:00 UTC,theRegistrar,234,,"
                  + "test,SERVER_STATUS,update-prohibited.test,REPO-ID,0,USD,20.00,",
              "6,2017-10-04 00:00:00 UTC,2017-10-04 00:00:00 UTC,theRegistrar,234,,"
@@ -264,7 +281,7 @@ class InvoicingPipelineTest {

  private static final ImmutableList<String> EXPECTED_INVOICE_OUTPUT =
      ImmutableList.of(
-          "2017-10-01,2020-09-30,234,61.50,USD,10125,1,PURCHASE,,3,"
+          "2017-10-01,2020-09-30,234,82.00,USD,10125,1,PURCHASE,,4,"
              + "RENEW | TLD: test | TERM: 3-year,20.50,USD,",
          "2017-10-01,2022-09-30,234,70.00,JPY,10125,1,PURCHASE,,1,"
              + "CREATE | TLD: hello | TERM: 5-year,70.00,JPY,",
@@ -398,7 +415,8 @@ JOIN Registrar r ON b.clientId = r.registrarId
 JOIN Domain d ON b.domainRepoId = d.repoId
 JOIN Tld t ON t.tldStr = d.tld
 LEFT JOIN BillingCancellation c ON b.id = c.billingEvent
-LEFT JOIN BillingCancellation cr ON b.cancellationMatchingBillingEvent = cr.billingRecurrence
+LEFT JOIN BillingCancellation cr ON b.cancellationMatchingBillingEvent = cr.billingRecurrence AND \
+b.billingTime = cr.billingTime
 WHERE r.billingAccountMap IS NOT NULL
 AND r.type = 'REAL'
 AND t.invoicingEnabled IS TRUE
@@ -607,6 +625,51 @@ AND cr.id IS NULL
        Instant.parse("2017-10-04T00:00:00.0Z"),
        Instant.parse("2017-10-02T00:00:00.0Z"));
    persistBillingEvent(16, domain15, registrar11, Reason.RENEW, 3, Money.of(USD, 20.5));
+
+    // Add a billing event in Year 1 and a cancellation in Year 2 for the same recurrence.
+    // The Year 1 event should NOT be cancelled.
+    Domain domain17 = persistActiveDomain("recurrence-collision.test");
+    DomainHistory domainHistoryCollision = persistDomainHistory(domain17, registrar1);
+
+    BillingRecurrence billingRecurrenceCollision =
+        new BillingRecurrence()
+            .asBuilder()
+            .setRegistrarId(registrar1.getRegistrarId())
+            .setRecurrenceEndTime(END_INSTANT)
+            .setId(100)
+            .setDomainHistory(domainHistoryCollision)
+            .setTargetId(domain17.getDomainName())
+            .setEventTime(Instant.parse("2017-10-04T00:00:00.0Z"))
+            .setReason(Reason.RENEW)
+            .build();
+    persistResource(billingRecurrenceCollision);
+
+    // Year 1 Billing Event (October 2017)
+    BillingEvent billingEventYear1 =
+        persistBillingEvent(17, domain17, registrar1, Reason.RENEW, 3, Money.of(USD, 20.5));
+    billingEventYear1 =
+        billingEventYear1
+            .asBuilder()
+            .setCancellationMatchingBillingEvent(billingRecurrenceCollision)
+            .setFlags(ImmutableSet.of(Flag.SYNTHETIC))
+            .setSyntheticCreationTime(Instant.parse("2017-10-03T00:00:00.0Z"))
+            .build();
+    persistResource(billingEventYear1);
+
+    // Year 2 Billing Cancellation (October 2018)
+    BillingCancellation cancellationYear2 =
+        new BillingCancellation()
+            .asBuilder()
+            .setId(101)
+            .setRegistrarId(registrar1.getRegistrarId())
+            .setEventTime(Instant.parse("2018-10-05T00:00:00.0Z"))
+            .setBillingTime(Instant.parse("2018-10-04T00:00:00.0Z"))
+            .setBillingRecurrence(billingRecurrenceCollision.createVKey())
+            .setTargetId(domain17.getDomainName())
+            .setReason(Reason.RENEW)
+            .setDomainHistory(domainHistoryCollision)
+            .build();
+    persistResource(cancellationYear2);
  }

  private static DomainHistory persistDomainHistory(Domain domain, Registrar registrar) {
@@ -116,4 +116,13 @@ class EppXmlSanitizerTest {
    String sanitizedXml = sanitizeEppXml(inputXml.getBytes(UTF_16LE));
    assertThat(sanitizedXml).isEqualTo(inputXml);
  }
+
+  @Test
+  void testSanitize_withDtd_returnsBase64() {
+    String inputXml = "<!DOCTYPE foo [<!ENTITY xxe SYSTEM \"file:///etc/passwd\">]><pw>&xxe;</pw>";
+    byte[] inputXmlBytes = inputXml.getBytes(UTF_8);
+    // Since DTDs are disabled, parsing should fail and fallback to base64 encoding of input.
+    String expectedBase64 = Base64.getMimeEncoder().encodeToString(inputXmlBytes);
+    assertThat(sanitizeEppXml(inputXmlBytes).trim()).isEqualTo(expectedBase64.trim());
+  }
 }
@@ -81,7 +81,6 @@ import google.registry.flows.EppException;
 import google.registry.flows.EppException.UnimplementedExtensionException;
 import google.registry.flows.EppRequestSource;
 import google.registry.flows.ExtensionManager.UndeclaredServiceExtensionException;
-import google.registry.flows.FlowUtils;
 import google.registry.flows.FlowUtils.NotLoggedInException;
 import google.registry.flows.FlowUtils.UnknownCurrencyEppException;
 import google.registry.flows.ResourceFlowTestCase;
@@ -144,6 +143,7 @@ import google.registry.flows.domain.token.AllocationTokenFlowUtils.AllocationTok
 import google.registry.flows.domain.token.AllocationTokenFlowUtils.AllocationTokenNotValidForRegistrarException;
 import google.registry.flows.domain.token.AllocationTokenFlowUtils.AlreadyRedeemedAllocationTokenException;
 import google.registry.flows.domain.token.AllocationTokenFlowUtils.NonexistentAllocationTokenException;
+import google.registry.flows.exceptions.ContactsProhibitedException;
 import google.registry.flows.exceptions.OnlyToolCanPassMetadataException;
 import google.registry.flows.exceptions.ResourceCreateContentionException;
 import google.registry.model.billing.BillingBase;
@@ -1937,8 +1937,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
  void testFailure_minimumDataset_noRegistrantButSomeOtherContactTypes() throws Exception {
    setEppInput("domain_create_other_contact_types.xml");
    persistHosts();
-    EppException thrown =
-        assertThrows(FlowUtils.GenericXmlSyntaxErrorException.class, this::runFlow);
+    EppException thrown = assertThrows(ContactsProhibitedException.class, this::runFlow);
    assertAboutEppExceptions().that(thrown).marshalsToXml();
  }

@@ -67,7 +67,6 @@ import google.registry.config.RegistryConfig;
 import google.registry.flows.EppException;
 import google.registry.flows.EppException.UnimplementedExtensionException;
 import google.registry.flows.EppRequestSource;
-import google.registry.flows.FlowUtils;
 import google.registry.flows.FlowUtils.NotLoggedInException;
 import google.registry.flows.ResourceFlowTestCase;
 import google.registry.flows.ResourceFlowUtils.AddExistingValueException;
@@ -90,6 +89,7 @@ import google.registry.flows.domain.DomainFlowUtils.SecDnsAllUsageException;
 import google.registry.flows.domain.DomainFlowUtils.TooManyDsRecordsException;
 import google.registry.flows.domain.DomainFlowUtils.TooManyNameserversException;
 import google.registry.flows.domain.DomainFlowUtils.UrgentAttributeNotSupportedException;
+import google.registry.flows.exceptions.ContactsProhibitedException;
 import google.registry.flows.exceptions.OnlyToolCanPassMetadataException;
 import google.registry.flows.exceptions.ResourceHasClientUpdateProhibitedException;
 import google.registry.flows.exceptions.ResourceStatusProhibitsOperationException;
@@ -283,8 +283,10 @@ class DomainUpdateFlowTest extends ResourceFlowTestCase<DomainUpdateFlow, Domain
    // This EPP adds a new technical contact mak21 that wasn't already present.
    setEppInput("domain_update_empty_registrant.xml");
    persistReferencedEntities();
+    persistDomain();
    // Fails because the update adds some new contacts, although the registrant has been removed.
-    assertThrows(FlowUtils.GenericXmlSyntaxErrorException.class, this::persistDomain);
+    EppException thrown = assertThrows(ContactsProhibitedException.class, this::runFlow);
+    assertAboutEppExceptions().that(thrown).marshalsToXml();
  }

  private void modifyDomainToHave13Nameservers() throws Exception {
@@ -15,27 +15,40 @@
 package google.registry.gcs;

 import static com.google.common.truth.Truth.assertThat;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;

 import com.google.cloud.storage.BlobId;
 import com.google.cloud.storage.BlobInfo;
+import com.google.cloud.storage.Bucket;
+import com.google.cloud.storage.BucketInfo;
+import com.google.cloud.storage.Storage;
 import com.google.cloud.storage.StorageOptions;
 import com.google.cloud.storage.contrib.nio.testing.LocalStorageHelper;
 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableMap;
 import com.google.common.io.ByteStreams;
 import com.google.common.net.MediaType;
+import google.registry.testing.SystemPropertyExtension;
+import google.registry.util.RegistryEnvironment;
 import java.io.ByteArrayInputStream;
 import java.io.ByteArrayOutputStream;
 import java.io.IOException;
 import java.io.ObjectInputStream;
 import java.io.ObjectOutputStream;
 import java.io.OutputStream;
+import org.junit.jupiter.api.AfterEach;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.RegisterExtension;

 /** Unit tests for {@link GcsUtilsTest}. */
 class GcsUtilsTest {

+  @RegisterExtension
+  final SystemPropertyExtension systemPropertyExtension = new SystemPropertyExtension();
+
  private GcsUtils gcsUtils = new GcsUtils(LocalStorageHelper.getOptions());

  private String bucket = "my-bucket";
@@ -43,9 +56,18 @@ class GcsUtilsTest {
  private BlobId blobId = BlobId.of(bucket, filename);
  private ImmutableMap<String, String> metadata = ImmutableMap.of("key1", "val1", "Key2", "val2");
  private final byte[] bytes = new byte[] {'a', 'b', 'c'};
+  private RegistryEnvironment previousEnvironment;

  @BeforeEach
-  void beforeEach() {}
+  void beforeEach() {
+    previousEnvironment = RegistryEnvironment.get();
+    GcsUtils.clearPapCache();
+  }
+
+  @AfterEach
+  void afterEach() {
+    previousEnvironment.setup();
+  }

  @Test
  void testSerialization_testStorage() throws Exception {
@@ -111,6 +133,88 @@ class GcsUtilsTest {
    assertThat(gcsUtils.existsAndNotEmpty(blobId)).isFalse();
  }

+  @Test
+  void testVerifyPublicAccessPrevention_enforced() {
+    RegistryEnvironment.PRODUCTION.setup(systemPropertyExtension);
+    Storage mockStorage = mock(Storage.class);
+    StorageOptions mockOptions = mock(StorageOptions.class);
+    when(mockOptions.getService()).thenReturn(mockStorage);
+
+    Bucket mockBucket = mock(Bucket.class);
+    when(mockStorage.get("my-bucket")).thenReturn(mockBucket);
+
+    BucketInfo.IamConfiguration mockIamConfig = mock(BucketInfo.IamConfiguration.class);
+    when(mockBucket.getIamConfiguration()).thenReturn(mockIamConfig);
+    when(mockIamConfig.getPublicAccessPrevention())
+        .thenReturn(BucketInfo.PublicAccessPrevention.ENFORCED);
+
+    GcsUtils utils = new GcsUtils(mockOptions);
+    utils.verifyPublicAccessPrevention("my-bucket");
+  }
+
+  @Test
+  void testVerifyPublicAccessPrevention_notCheckedInNonProd() {
+    RegistryEnvironment.SANDBOX.setup(systemPropertyExtension);
+    Storage mockStorage = mock(Storage.class);
+    StorageOptions mockOptions = mock(StorageOptions.class);
+    when(mockOptions.getService()).thenReturn(mockStorage);
+
+    Bucket mockBucket = mock(Bucket.class);
+    when(mockStorage.get("my-bucket")).thenReturn(mockBucket);
+
+    BucketInfo.IamConfiguration mockIamConfig = mock(BucketInfo.IamConfiguration.class);
+    when(mockBucket.getIamConfiguration()).thenReturn(mockIamConfig);
+    when(mockIamConfig.getPublicAccessPrevention())
+        .thenReturn(BucketInfo.PublicAccessPrevention.INHERITED);
+
+    GcsUtils utils = new GcsUtils(mockOptions);
+    // no exception thrown even though PAP isn't enforced
+    utils.verifyPublicAccessPrevention("my-bucket");
+  }
+
+  @Test
+  void testVerifyPublicAccessPrevention_notEnforced() {
+    RegistryEnvironment.PRODUCTION.setup(systemPropertyExtension);
+    Storage mockStorage = mock(Storage.class);
+    StorageOptions mockOptions = mock(StorageOptions.class);
+    when(mockOptions.getService()).thenReturn(mockStorage);
+
+    Bucket mockBucket = mock(Bucket.class);
+    when(mockStorage.get("my-bucket")).thenReturn(mockBucket);
+
+    BucketInfo.IamConfiguration mockIamConfig = mock(BucketInfo.IamConfiguration.class);
+    when(mockBucket.getIamConfiguration()).thenReturn(mockIamConfig);
+    when(mockIamConfig.getPublicAccessPrevention())
+        .thenReturn(BucketInfo.PublicAccessPrevention.INHERITED);
+
+    GcsUtils utils = new GcsUtils(mockOptions);
+
+    IllegalStateException thrown =
+        assertThrows(
+            IllegalStateException.class, () -> utils.verifyPublicAccessPrevention("my-bucket"));
+    assertThat(thrown)
+        .hasMessageThat()
+        .contains("Public Access Prevention is not enforced on bucket my-bucket");
+  }
+
+  @Test
+  void testVerifyPublicAccessPrevention_nonexistentBucket() {
+    RegistryEnvironment.PRODUCTION.setup(systemPropertyExtension);
+    Storage mockStorage = mock(Storage.class);
+    StorageOptions mockOptions = mock(StorageOptions.class);
+    when(mockOptions.getService()).thenReturn(mockStorage);
+
+    when(mockStorage.get("nonexistent-bucket")).thenReturn(null);
+
+    GcsUtils utils = new GcsUtils(mockOptions);
+
+    IllegalStateException thrown =
+        assertThrows(
+            IllegalStateException.class,
+            () -> utils.verifyPublicAccessPrevention("nonexistent-bucket"));
+    assertThat(thrown).hasMessageThat().contains("Bucket nonexistent-bucket does not exist");
+  }
+
  private static byte[] serialize(Object object) throws IOException {
    try (ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
      ObjectOutputStream oos = new ObjectOutputStream(baos);
@@ -16,10 +16,17 @@ package google.registry.model;

 import static com.google.common.truth.Truth.assertThat;
 import static google.registry.testing.DatabaseHelper.createTld;
+import static google.registry.testing.DatabaseHelper.persistActiveDomain;
+import static google.registry.testing.DatabaseHelper.persistPremiumList;
+import static google.registry.testing.DatabaseHelper.persistResource;
+import static org.joda.money.CurrencyUnit.USD;

 import google.registry.model.OteStats.StatType;
+import google.registry.model.domain.DomainHistory;
+import google.registry.model.reporting.HistoryEntry.Type;
 import google.registry.persistence.transaction.JpaTestExtensions;
 import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
+import java.time.Instant;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.RegisterExtension;
@@ -60,39 +67,40 @@ public final class OteStatsTest {
    OteStats stats = OteStats.getFromRegistrar("blobio");
    String expected =
        """
-            contact creates: 0
-            contact deletes: 0
-            contact transfer approves: 0
-            contact transfer cancels: 0
-            contact transfer rejects: 0
-            contact transfer requests: 0
-            contact updates: 0
-            domain autorenews: 0
-            domain creates: 5
-            domain creates ascii: 4
-            domain creates idn: 1
-            domain creates start date sunrise: 1
-            domain creates with claims notice: 1
-            domain creates with fee: 1
-            domain creates with sec dns: 1
-            domain creates without sec dns: 4
-            domain deletes: 1
-            domain renews: 0
-            domain restores: 1
-            domain transfer approves: 1
-            domain transfer cancels: 1
-            domain transfer rejects: 1
-            domain transfer requests: 1
-            domain updates: 1
-            domain updates with sec dns: 1
-            domain updates without sec dns: 0
-            host creates: 1
-            host creates external: 0
-            host creates subordinate: 1
-            host deletes: 1
-            host updates: 1
-            unclassified flows: 0
-            TOTAL: 30""";
+        contact creates: 0
+        contact deletes: 0
+        contact transfer approves: 0
+        contact transfer cancels: 0
+        contact transfer rejects: 0
+        contact transfer requests: 0
+        contact updates: 0
+        domain autorenews: 0
+        domain creates: 5
+        domain creates ascii: 4
+        domain creates idn: 1
+        domain creates start date sunrise: 1
+        domain creates with claims notice: 1
+        domain creates with fee: 1
+        domain creates with sec dns: 1
+        domain creates without sec dns: 4
+        domain deletes: 1
+        domain renews: 0
+        domain restores: 1
+        domain transfer approves: 1
+        domain transfer cancels: 1
+        domain transfer rejects: 1
+        domain transfer requests: 1
+        domain updates: 1
+        domain updates with sec dns: 1
+        domain updates without sec dns: 0
+        host creates: 1
+        host creates external: 0
+        host creates subordinate: 1
+        host deletes: 1
+        host updates: 1
+        unclassified flows: 0
+        TOTAL: 30\
+        """;
    assertThat(stats.toString()).isEqualTo(expected);
  }

@@ -102,39 +110,72 @@ public final class OteStatsTest {
    OteStats stats = OteStats.getFromRegistrar("blobio");
    String expected =
        """
-            contact creates: 0
-            contact deletes: 0
-            contact transfer approves: 0
-            contact transfer cancels: 0
-            contact transfer rejects: 0
-            contact transfer requests: 0
-            contact updates: 0
-            domain autorenews: 0
-            domain creates: 4
-            domain creates ascii: 4
-            domain creates idn: 0
-            domain creates start date sunrise: 1
-            domain creates with claims notice: 1
-            domain creates with fee: 1
-            domain creates with sec dns: 1
-            domain creates without sec dns: 3
-            domain deletes: 1
-            domain renews: 0
-            domain restores: 0
-            domain transfer approves: 1
-            domain transfer cancels: 1
-            domain transfer rejects: 1
-            domain transfer requests: 1
-            domain updates: 1
-            domain updates with sec dns: 1
-            domain updates without sec dns: 0
-            host creates: 1
-            host creates external: 0
-            host creates subordinate: 1
-            host deletes: 0
-            host updates: 10
-            unclassified flows: 0
-            TOTAL: 34""";
+        contact creates: 0
+        contact deletes: 0
+        contact transfer approves: 0
+        contact transfer cancels: 0
+        contact transfer rejects: 0
+        contact transfer requests: 0
+        contact updates: 0
+        domain autorenews: 0
+        domain creates: 4
+        domain creates ascii: 4
+        domain creates idn: 0
+        domain creates start date sunrise: 1
+        domain creates with claims notice: 1
+        domain creates with fee: 1
+        domain creates with sec dns: 1
+        domain creates without sec dns: 3
+        domain deletes: 1
+        domain renews: 0
+        domain restores: 0
+        domain transfer approves: 1
+        domain transfer cancels: 1
+        domain transfer rejects: 1
+        domain transfer requests: 1
+        domain updates: 1
+        domain updates with sec dns: 1
+        domain updates without sec dns: 0
+        host creates: 1
+        host creates external: 0
+        host creates subordinate: 1
+        host deletes: 0
+        host updates: 10
+        unclassified flows: 0
+        TOTAL: 34\
+        """;
    assertThat(stats.toString()).isEqualTo(expected);
  }
+
+  @Test
+  void testDomainCreateWithoutXmlBytes_doesNotSatisfyComplexRequirements() throws Exception {
+    persistPremiumList("default_sandbox_list", USD, "sandbox,USD 1000");
+    OteAccountBuilder.forRegistrarId("blobio").buildAndPersist();
+    String oteAccount1 = "blobio-1";
+    Instant now = Instant.parse("2026-06-25T10:00:00Z");
+
+    // Persist a DOMAIN_CREATE history entry without XML bytes
+    persistResource(
+        new DomainHistory.Builder()
+            .setDomain(persistActiveDomain("example.tld"))
+            .setRegistrarId(oteAccount1)
+            .setType(Type.DOMAIN_CREATE)
+            .setXmlBytes(null) // explicitly null
+            .setModificationTime(now)
+            .build());
+
+    OteStats stats = OteStats.getFromRegistrar("blobio");
+
+    // It should count towards the basic DOMAIN_CREATES
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES)).isEqualTo(1);
+
+    // It should NOT count towards any stat type that requires EPP input filtering
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_ASCII)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_IDN)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_START_DATE_SUNRISE)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_WITH_CLAIMS_NOTICE)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_WITH_FEE)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_WITH_SEC_DNS)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_WITHOUT_SEC_DNS)).isEqualTo(0);
+  }
 }
@@ -17,8 +17,8 @@ package google.registry.model.domain;
 import static google.registry.testing.DatabaseHelper.persistActiveHost;
 import static org.junit.jupiter.api.Assertions.assertThrows;

-import google.registry.flows.FlowUtils;
 import google.registry.flows.domain.DomainFlowUtils.RegistrantProhibitedException;
+import google.registry.flows.exceptions.ContactsProhibitedException;
 import google.registry.model.ResourceCommandTestCase;
 import google.registry.model.eppinput.EppInput;
 import google.registry.model.eppinput.EppInput.ResourceCommandWrapper;
@@ -88,9 +88,10 @@ class DomainCommandTest extends ResourceCommandTestCase {
  void testCreate_cloneAndLinkReferences_failsWithContacts() throws Exception {
    persistActiveHost("ns1.example.net");
    persistActiveHost("ns2.example.net");
+    DomainCommand.Create create =
+        (DomainCommand.Create) loadEppResourceCommand("domain_create_with_contacts.xml");
    assertThrows(
-        FlowUtils.GenericXmlSyntaxErrorException.class,
-        () -> loadEppResourceCommand("domain_create_with_contacts.xml"));
+        ContactsProhibitedException.class, () -> create.cloneAndLinkReferences(fakeClock.now()));
  }

  @Test
@@ -138,9 +139,10 @@ class DomainCommandTest extends ResourceCommandTestCase {
  void testUpdate_cloneAndLinkReferences_failsWithContacts() throws Exception {
    persistActiveHost("ns1.example.com");
    persistActiveHost("ns2.example.com");
+    DomainCommand.Update update =
+        (DomainCommand.Update) loadEppResourceCommand("domain_update_with_contacts.xml");
    assertThrows(
-        FlowUtils.GenericXmlSyntaxErrorException.class,
-        () -> loadEppResourceCommand("domain_update_with_contacts.xml"));
+        ContactsProhibitedException.class, () -> update.cloneAndLinkReferences(fakeClock.now()));
  }

  @Test
@@ -110,28 +110,44 @@ public class PremiumListTest {

  @Test
  void testValidation_labelMustBeLowercase() {
-    Exception e =
-        assertThrows(
-            IllegalArgumentException.class,
-            () ->
-                new PremiumEntry.Builder()
-                    .setPrice(BigDecimal.valueOf(399))
-                    .setLabel("UPPER.tld")
-                    .build());
-    assertThat(e).hasMessageThat().contains("must be in puny-coded, lower-case form");
+    assertThat(
+            assertThrows(
+                IllegalArgumentException.class,
+                () ->
+                    new PremiumEntry.Builder()
+                        .setPrice(BigDecimal.valueOf(399))
+                        .setLabel("UPPER.tld")
+                        .build()))
+        .hasMessageThat()
+        .contains("must be in puny-coded, lower-case form");
+  }
+
+  @Test
+  void testValidation_priceMustNotBeNegative() {
+    assertThat(
+            assertThrows(
+                IllegalArgumentException.class,
+                () ->
+                    new PremiumEntry.Builder()
+                        .setPrice(BigDecimal.valueOf(-100))
+                        .setLabel("anchor")
+                        .build()))
+        .hasMessageThat()
+        .isEqualTo("Price must not be negative");
  }

  @Test
  void testValidation_labelMustBePunyCoded() {
-    Exception e =
-        assertThrows(
-            IllegalArgumentException.class,
-            () ->
-                new PremiumEntry.Builder()
-                    .setPrice(BigDecimal.valueOf(399))
-                    .setLabel("lower.みんな")
-                    .build());
-    assertThat(e).hasMessageThat().contains("must be in puny-coded, lower-case form");
+    assertThat(
+            assertThrows(
+                IllegalArgumentException.class,
+                () ->
+                    new PremiumEntry.Builder()
+                        .setPrice(BigDecimal.valueOf(399))
+                        .setLabel("lower.みんな")
+                        .build()))
+        .hasMessageThat()
+        .contains("must be in puny-coded, lower-case form");
  }

  @Test
@@ -404,6 +404,39 @@ class JpaTransactionManagerImplTest {
        .containsExactlyElementsIn(moreEntities);
  }

+  @Test
+  void loadAllOfSorted() {
+    TestEntity entityB = new TestEntity("b_entity", "gamma");
+    TestEntity entityC = new TestEntity("c_entity", "alpha");
+    TestEntity entityA = new TestEntity("a_entity", "beta");
+    persistResources(ImmutableList.of(entityB, entityC, entityA));
+
+    assertThat(tm().transact(() -> tm().loadAllOfSorted(TestEntity.class, "name")))
+        .containsExactly(entityA, entityB, entityC)
+        .inOrder();
+
+    assertThat(tm().transact(() -> tm().loadAllOfSorted(TestEntity.class, "data")))
+        .containsExactly(entityC, entityA, entityB)
+        .inOrder();
+
+    assertThat(tm().transact(() -> tm().loadAllOfSorted(TestEntity.class, "data", "name")))
+        .containsExactly(entityC, entityA, entityB)
+        .inOrder();
+  }
+
+  @Test
+  void loadAllOfSorted_invalidFieldName_throwsException() {
+    IllegalArgumentException thrown =
+        assertThrows(
+            IllegalArgumentException.class,
+            () ->
+                tm().transact(
+                        () ->
+                            tm().loadAllOfSorted(
+                                    TestEntity.class, "name; DROP TABLE TestEntity;")));
+    assertThat(thrown).hasMessageThat().contains("Invalid sort field name");
+  }
+
  @Test
  void saveAllNew_rollsBackWhenFailure() {
    moreEntities.forEach(entity -> assertThat(tm().transact(() -> tm().exists(entity))).isFalse());
@@ -37,7 +37,7 @@ import google.registry.testing.FakeUrlConnectionService;
 import google.registry.util.UrlConnectionException;
 import java.io.ByteArrayInputStream;
 import java.net.HttpURLConnection;
-import java.net.URL;
+import java.net.URI;
 import java.nio.charset.StandardCharsets;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
@@ -88,7 +88,8 @@ public final class UpdateRegistrarRdapBaseUrlsActionTest {
  private void assertCorrectRequestSent() throws Exception {
    assertThat(urlConnectionService.getConnectedUrls())
        .containsExactly(
-            new URL("https://www.iana.org/assignments/registrar-ids/registrar-ids-1.csv"));
+            URI.create("https://www.iana.org/assignments/registrar-ids/registrar-ids-1.csv")
+                .toURL());
    verify(connection).setRequestProperty("Accept-Encoding", "gzip");
  }

@@ -15,8 +15,11 @@
 package google.registry.reporting.billing;

 import static com.google.common.truth.Truth.assertThat;
+import static google.registry.model.common.Cursor.CursorType.RECURRING_BILLING;
+import static google.registry.testing.DatabaseHelper.persistResource;
 import static jakarta.servlet.http.HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
 import static jakarta.servlet.http.HttpServletResponse.SC_OK;
+import static org.mockito.ArgumentMatchers.startsWith;
 import static org.mockito.Mockito.mock;
 import static org.mockito.Mockito.verify;
 import static org.mockito.Mockito.when;
@@ -25,6 +28,7 @@ import com.google.cloud.tasks.v2.HttpMethod;
 import com.google.common.net.MediaType;
 import google.registry.batch.CloudTasksUtils;
 import google.registry.beam.BeamActionTestBase;
+import google.registry.model.common.Cursor;
 import google.registry.persistence.transaction.JpaTestExtensions;
 import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
 import google.registry.reporting.ReportingModule;
@@ -33,6 +37,7 @@ import google.registry.testing.CloudTasksHelper.TaskMatcher;
 import google.registry.testing.FakeClock;
 import java.io.IOException;
 import java.time.Duration;
+import java.time.Instant;
 import java.time.YearMonth;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.RegisterExtension;
@@ -50,8 +55,13 @@ class GenerateInvoicesActionTest extends BeamActionTestBase {
  private CloudTasksUtils cloudTasksUtils = cloudTasksHelper.getTestCloudTasksUtils();
  private GenerateInvoicesAction action;

+  private void setCursor(Instant cursorTime) {
+    persistResource(Cursor.createGlobal(RECURRING_BILLING, cursorTime));
+  }
+
  @Test
  void testLaunchTemplateJob_withPublish() throws Exception {
+    setCursor(Instant.parse("2017-11-01T00:00:00Z"));
    action =
        new GenerateInvoicesAction(
            "test-project",
@@ -84,6 +94,7 @@ class GenerateInvoicesActionTest extends BeamActionTestBase {

  @Test
  void testLaunchTemplateJob_withoutPublish() throws Exception {
+    setCursor(Instant.parse("2017-11-01T00:00:00Z"));
    action =
        new GenerateInvoicesAction(
            "test-project",
@@ -107,6 +118,7 @@ class GenerateInvoicesActionTest extends BeamActionTestBase {

  @Test
  void testCaughtIOException() throws IOException {
+    setCursor(Instant.parse("2017-11-01T00:00:00Z"));
    when(launch.execute()).thenThrow(new IOException("Pipeline error"));
    action =
        new GenerateInvoicesAction(
@@ -128,4 +140,58 @@ class GenerateInvoicesActionTest extends BeamActionTestBase {
    verify(emailUtils).sendAlertEmail("Pipeline Launch failed due to Pipeline error");
    cloudTasksHelper.assertNoTasksEnqueued("beam-reporting");
  }
+
+  @Test
+  void testFailure_cursorLagging() {
+    setCursor(Instant.parse("2017-10-31T23:59:59.999Z"));
+    action =
+        new GenerateInvoicesAction(
+            "test-project",
+            "test-region",
+            "staging_bucket",
+            "billing_bucket",
+            "REG-INV",
+            false,
+            YearMonth.of(2017, 10),
+            emailUtils,
+            cloudTasksUtils,
+            clock,
+            response,
+            dataflow);
+    action.run();
+    assertThat(response.getStatus()).isEqualTo(SC_INTERNAL_SERVER_ERROR);
+    assertThat(response.getPayload()).contains("Pipeline launch failed");
+    assertThat(response.getPayload()).contains("BillingRecurrence expansion cursor");
+    verify(emailUtils)
+        .sendAlertEmail(
+            startsWith("Pipeline Launch failed due to BillingRecurrence expansion cursor"));
+    cloudTasksHelper.assertNoTasksEnqueued("beam-reporting");
+  }
+
+  @Test
+  void testFailure_cursorMissing() {
+    // Do not set cursor, should default to START_INSTANT (1970)
+    action =
+        new GenerateInvoicesAction(
+            "test-project",
+            "test-region",
+            "staging_bucket",
+            "billing_bucket",
+            "REG-INV",
+            false,
+            YearMonth.of(2017, 10),
+            emailUtils,
+            cloudTasksUtils,
+            clock,
+            response,
+            dataflow);
+    action.run();
+    assertThat(response.getStatus()).isEqualTo(SC_INTERNAL_SERVER_ERROR);
+    assertThat(response.getPayload()).contains("Pipeline launch failed");
+    assertThat(response.getPayload()).contains("BillingRecurrence expansion cursor");
+    verify(emailUtils)
+        .sendAlertEmail(
+            startsWith("Pipeline Launch failed due to BillingRecurrence expansion cursor"));
+    cloudTasksHelper.assertNoTasksEnqueued("beam-reporting");
+  }
 }
@@ -34,7 +34,7 @@ import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationT
 import google.registry.testing.FakeUrlConnectionService;
 import java.io.ByteArrayOutputStream;
 import java.net.HttpURLConnection;
-import java.net.URL;
+import java.net.URI;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.RegisterExtension;
@@ -74,7 +74,7 @@ class IcannHttpReporterTest {
    assertThat(reporter.send(FAKE_PAYLOAD, "test-transactions-201706.csv")).isTrue();

    assertThat(urlConnectionService.getConnectedUrls())
-        .containsExactly(new URL("https://fake-transactions.url/test/2017-06"));
+        .containsExactly(URI.create("https://fake-transactions.url/test/2017-06").toURL());
    String userPass = "test_ry:fakePass";
    String expectedAuth =
        String.format("Basic %s", BaseEncoding.base64().encode(StringUtils.getBytesUtf8(userPass)));
@@ -88,7 +88,7 @@ class IcannHttpReporterTest {
    assertThat(reporter.send(FAKE_PAYLOAD, "xn--abc123-transactions-201706.csv")).isTrue();

    assertThat(urlConnectionService.getConnectedUrls())
-        .containsExactly(new URL("https://fake-transactions.url/xn--abc123/2017-06"));
+        .containsExactly(URI.create("https://fake-transactions.url/xn--abc123/2017-06").toURL());
    String userPass = "xn--abc123_ry:fakePass";
    String expectedAuth =
        String.format("Basic %s", BaseEncoding.base64().encode(StringUtils.getBytesUtf8(userPass)));
@@ -55,7 +55,7 @@ public class GoldenFileTestHelper {

  public static GoldenFileTestHelper assertThatRoutesFromComponent(Class<?> component) {
    return assertThat(RouterDisplayHelper.extractHumanReadableRoutesFromComponent(component))
-        .createdByNomulusCommand("get_routing_map -c " + component.getName());
+        .createdByNomulusCommand("get_routing_map");
  }

  public GoldenFileTestHelper createdByNomulusCommand(String nomulusCommand) {
@@ -57,7 +57,7 @@ import google.registry.util.UrlConnectionException;
 import java.io.ByteArrayInputStream;
 import java.io.ByteArrayOutputStream;
 import java.net.HttpURLConnection;
-import java.net.URL;
+import java.net.URI;
 import java.security.SecureRandom;
 import java.time.Duration;
 import java.time.Instant;
@@ -249,7 +249,7 @@ class NordnUploadActionTest {
        .setRequestProperty(eq(CONTENT_TYPE), startsWith("multipart/form-data; boundary="));
    verify(httpUrlConnection).setRequestMethod("POST");
    assertThat(httpUrlConnection.getURL())
-        .isEqualTo(new URL("http://127.0.0.1/LORDN/tld/" + phase));
+        .isEqualTo(URI.create("http://127.0.0.1/LORDN/tld/" + phase).toURL());
    assertThat(connectionOutputStream.toString(UTF_8)).contains(csv);
    verifyColumnCleared(domain1);
    verifyColumnCleared(domain2);
@@ -38,7 +38,7 @@ import google.registry.testing.FakeResponse;
 import google.registry.testing.FakeUrlConnectionService;
 import java.io.ByteArrayInputStream;
 import java.net.HttpURLConnection;
-import java.net.URL;
+import java.net.URI;
 import java.util.Optional;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
@@ -49,33 +49,35 @@ class NordnVerifyActionTest {

  private static final String LOG_ACCEPTED =
      """
-          1,2012-08-16T02:15:00.0Z,2012-08-16T00:00:00.0Z,0000000000000478Nzs+3VMkR8ckuUynOLmyeqTmZQSbzDuf/R50n2n5QX4=,accepted,no-warnings,1
-          roid,result-code
-          SH8013-REP,2000""";
+      1,2012-08-16T02:15:00.0Z,2012-08-16T00:00:00.0Z,0000000000000478Nzs+3VMkR8ckuUynOLmyeqTmZQSbzDuf/R50n2n5QX4=,accepted,no-warnings,1
+      roid,result-code
+      SH8013-REP,2000\
+      """;

  private static final String LOG_REJECTED =
      """
-          1,2012-08-16T02:15:00.0Z,2012-08-16T00:00:00.0Z,0000000000000478Nzs+3VMkR8ckuUynOLmyeqTmZQSbzDuf/R50n2n5QX4=,rejected,no-warnings,1
-          roid,result-code
-          SH8013-REP,2001""";
+      1,2012-08-16T02:15:00.0Z,2012-08-16T00:00:00.0Z,0000000000000478Nzs+3VMkR8ckuUynOLmyeqTmZQSbzDuf/R50n2n5QX4=,rejected,no-warnings,1
+      roid,result-code
+      SH8013-REP,2001\
+      """;

  private static final String LOG_WARNINGS =
      """
-          1,2012-08-16T02:15:00.0Z,2012-08-16T00:00:00.0Z,0000000000000478Nzs+3VMkR8ckuUynOLmyeqTmZQSbzDuf/R50n2n5QX4=,accepted,warnings-present,3
-          roid,result-code
-          SH8013-REP,2001
-          lulz-roid,3609
-          sabokitty-roid,3610
-          """;
+      1,2012-08-16T02:15:00.0Z,2012-08-16T00:00:00.0Z,0000000000000478Nzs+3VMkR8ckuUynOLmyeqTmZQSbzDuf/R50n2n5QX4=,accepted,warnings-present,3
+      roid,result-code
+      SH8013-REP,2001
+      lulz-roid,3609
+      sabokitty-roid,3610
+      """;

  private static final String LOG_ERRORS =
      """
-          1,2012-08-16T02:15:00.0Z,2012-08-16T00:00:00.0Z,0000000000000478Nzs+3VMkR8ckuUynOLmyeqTmZQSbzDuf/R50n2n5QX4=,accepted,warnings-present,3
-          roid,result-code
-          SH8013-REP,2000
-          lulz-roid,4601
-          bogpog,4611
-          """;
+      1,2012-08-16T02:15:00.0Z,2012-08-16T00:00:00.0Z,0000000000000478Nzs+3VMkR8ckuUynOLmyeqTmZQSbzDuf/R50n2n5QX4=,accepted,warnings-present,3
+      roid,result-code
+      SH8013-REP,2000
+      lulz-roid,4601
+      bogpog,4611
+      """;

  @RegisterExtension
  final JpaIntegrationTestExtension jpa =
@@ -101,14 +103,15 @@ class NordnVerifyActionTest {
        .thenReturn(new ByteArrayInputStream(LOG_ACCEPTED.getBytes(UTF_8)));
    action.lordnRequestInitializer = lordnRequestInitializer;
    action.response = response;
-    action.url = new URL("http://127.0.0.1/blobio");
+    action.url = URI.create("http://ry.marksdb.org/blobio").toURL();
  }

  @Test
  @SuppressWarnings("DirectInvocationOnMock")
  void testSuccess_sendHttpRequest_urlIsCorrect() throws Exception {
    action.run();
-    assertThat(httpUrlConnection.getURL()).isEqualTo(new URL("http://127.0.0.1/blobio"));
+    assertThat(httpUrlConnection.getURL())
+        .isEqualTo(URI.create("http://ry.marksdb.org/blobio").toURL());
  }

  @Test
@@ -162,4 +165,22 @@ class NordnVerifyActionTest {
    ConflictException thrown = assertThrows(ConflictException.class, action::run);
    assertThat(thrown).hasMessageThat().contains("Not ready");
  }
+
+  @Test
+  void testFailure_badUrl() throws Exception {
+    action.url = URI.create("http://example.com/blobio").toURL();
+    IllegalArgumentException thrown = assertThrows(IllegalArgumentException.class, action::run);
+    assertThat(thrown)
+        .hasMessageThat()
+        .isEqualTo("URL http://example.com/blobio must start with ry.marksdb.org");
+  }
+
+  @Test
+  @SuppressWarnings("DirectInvocationOnMock")
+  void testSuccess_uppercaseUrl() throws Exception {
+    action.url = URI.create("http://RY.MARKSDB.ORG/blobio").toURL();
+    action.run();
+    assertThat(httpUrlConnection.getURL())
+        .isEqualTo(URI.create("http://RY.MARKSDB.ORG/blobio").toURL());
+  }
 }
@@ -24,7 +24,7 @@ import static org.mockito.Mockito.when;
 import google.registry.config.RegistryConfig.ConfigModule.TmchCaMode;
 import java.io.ByteArrayInputStream;
 import java.net.MalformedURLException;
-import java.net.URL;
+import java.net.URI;
 import java.security.SignatureException;
 import java.security.cert.CRLException;
 import java.security.cert.CertificateNotYetValidException;
@@ -39,7 +39,7 @@ class TmchCrlActionTest extends TmchActionTestCase {
    TmchCrlAction action = new TmchCrlAction();
    action.marksdb = marksdb;
    action.tmchCertificateAuthority = new TmchCertificateAuthority(tmchCaMode, clock);
-    action.tmchCrlUrl = new URL("https://sloth.lol/tmch.crl");
+    action.tmchCrlUrl = URI.create("https://sloth.lol/tmch.crl").toURL();
    return action;
  }

@@ -57,7 +57,7 @@ class TmchCrlActionTest extends TmchActionTestCase {
    newTmchCrlAction(TmchCaMode.PILOT).run();
    verify(httpUrlConnection).getInputStream();
    assertThat(urlConnectionService.getConnectedUrls())
-        .containsExactly(new URL("https://sloth.lol/tmch.crl"));
+        .containsExactly(URI.create("https://sloth.lol/tmch.crl").toURL());
  }

  @Test
@@ -23,6 +23,7 @@ import static google.registry.testing.DatabaseHelper.persistResource;
 import static google.registry.testing.TestDataHelper.loadFile;
 import static google.registry.util.DateTimeUtils.plusMinutes;
 import static java.nio.charset.StandardCharsets.UTF_8;
+import static org.junit.jupiter.api.Assertions.assertThrows;

 import com.google.cloud.storage.BlobId;
 import com.google.cloud.storage.contrib.nio.testing.LocalStorageHelper;
@@ -56,6 +57,20 @@ class GenerateZoneFilesActionTest {

  private final GcsUtils gcsUtils = new GcsUtils(LocalStorageHelper.getOptions());

+  @Test
+  void testGenerate_nonexistentTld_throwsException() {
+    GenerateZoneFilesAction action = new GenerateZoneFilesAction();
+    IllegalArgumentException thrown =
+        assertThrows(
+            IllegalArgumentException.class,
+            () ->
+                action.handleJsonRequest(
+                    ImmutableMap.<String, Object>of(
+                        "tlds", ImmutableList.of("nonexistent-tld"),
+                        "exportTime", Instant.parse("2024-03-27T00:00:00Z"))));
+    assertThat(thrown).hasMessageThat().contains("TLDs do not exist: nonexistent-tld");
+  }
+
  @Test
  void testGenerate_defaultTtls() throws Exception {
    createTlds("tld", "com");
@@ -24,6 +24,7 @@ import static google.registry.testing.DatabaseHelper.createTld;
 import static google.registry.testing.DatabaseHelper.persistActiveDomain;
 import static google.registry.testing.DatabaseHelper.persistDeletedDomain;
 import static google.registry.util.DateTimeUtils.minusYears;
+import static org.junit.jupiter.api.Assertions.assertThrows;

 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableSet;
@@ -151,4 +152,18 @@ public class RefreshDnsForAllDomainsActionTest {
    action.run();
    assertDnsRequestsWithRequestTime(clock.now(), 11);
  }
+
+  @Test
+  void test_runAction_emptyTlds_throwsException() {
+    action =
+        new RefreshDnsForAllDomainsAction(
+            response,
+            ImmutableSet.of(),
+            Optional.of(10),
+            Optional.empty(),
+            Optional.empty(),
+            new Random());
+    IllegalArgumentException thrown = assertThrows(IllegalArgumentException.class, action::run);
+    assertThat(thrown).hasMessageThat().isEqualTo("Must specify TLDs to refresh");
+  }
 }
@@ -29,13 +29,20 @@ import static org.mockito.Mockito.verify;
 import static org.mockito.Mockito.when;

 import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
 import com.google.common.collect.ImmutableSetMultimap;
 import google.registry.flows.PasswordOnlyTransportCredentials;
 import google.registry.model.console.ConsoleUpdateHistory;
+import google.registry.model.console.RegistrarRole;
+import google.registry.model.console.User;
+import google.registry.model.console.UserRoles;
 import google.registry.model.registrar.Registrar;
 import google.registry.request.Action;
 import google.registry.request.RequestModule;
+import google.registry.request.auth.AuthResult;
 import google.registry.request.auth.AuthenticatedRegistrarAccessor;
+import google.registry.testing.ConsoleApiParamsUtils;
+import google.registry.testing.FakeResponse;
 import google.registry.ui.server.console.ConsoleEppPasswordAction.EppPasswordData;
 import google.registry.util.EmailMessage;
 import jakarta.mail.internet.AddressException;
@@ -123,6 +130,36 @@ class ConsoleEppPasswordActionTest extends ConsoleActionBaseTestCase {
    assertThat(history.getDescription()).hasValue("TheRegistrar");
  }

+  @Test
+  void testFailure_noPermission() throws IOException {
+    User user =
+        persistResource(
+            new User.Builder()
+                .setEmailAddress("no.permission@example.tld")
+                .setUserRoles(
+                    new UserRoles.Builder()
+                        .setRegistrarRoles(
+                            ImmutableMap.of("TheRegistrar", RegistrarRole.ACCOUNT_MANAGER))
+                        .build())
+                .build());
+    ConsoleEppPasswordAction action =
+        createAction(user, "TheRegistrar", "foobar", "randomPassword", "randomPassword");
+    action.run();
+    assertThat(response.getStatus()).isEqualTo(SC_FORBIDDEN);
+  }
+
+  private ConsoleEppPasswordAction createAction(
+      User user,
+      String registrarId,
+      String oldPassword,
+      String newPassword,
+      String newPasswordRepeat)
+      throws IOException {
+    consoleApiParams = ConsoleApiParamsUtils.createFake(AuthResult.createUser(user));
+    response = (FakeResponse) consoleApiParams.response();
+    return createAction(registrarId, oldPassword, newPassword, newPasswordRepeat);
+  }
+
  private ConsoleEppPasswordAction createAction(
      String registrarId, String oldPassword, String newPassword, String newPasswordRepeat)
      throws IOException {
@@ -407,6 +407,103 @@ class ConsoleUsersActionTest extends ConsoleActionBaseTestCase {
        .contains("Can't update user not associated with registrarId TheRegistrar");
  }

+  @Test
+  void testSuccess_appendUser() throws IOException {
+    User user = DatabaseHelper.createAdminUser("email@email.com");
+    AuthResult authResult = AuthResult.createUser(user);
+    ConsoleUsersAction action =
+        createAction(
+            Optional.of(ConsoleApiParamsUtils.createFake(authResult)),
+            Optional.of("POST"),
+            Optional.of(
+                new UserData("test3@test.com", null, RegistrarRole.TECH_CONTACT.name(), null)));
+    action.run();
+    assertThat(response.getStatus()).isEqualTo(SC_OK);
+    User appendedUser = DatabaseHelper.loadByKey(VKey.create(User.class, "test3@test.com"));
+    assertThat(appendedUser.getUserRoles().getRegistrarRoles().get("TheRegistrar"))
+        .isEqualTo(RegistrarRole.TECH_CONTACT);
+  }
+
+  @Test
+  void testFailure_appendUser_globalAdmin() throws IOException {
+    User user = DatabaseHelper.createAdminUser("email@email.com");
+    AuthResult authResult = AuthResult.createUser(user);
+    DatabaseHelper.persistResource(
+        new User.Builder()
+            .setEmailAddress("globaladmin@test.com")
+            .setUserRoles(
+                new UserRoles.Builder().setIsAdmin(true).setGlobalRole(GlobalRole.NONE).build())
+            .build());
+
+    ConsoleUsersAction action =
+        createAction(
+            Optional.of(ConsoleApiParamsUtils.createFake(authResult)),
+            Optional.of("POST"),
+            Optional.of(
+                new UserData(
+                    "globaladmin@test.com", null, RegistrarRole.TECH_CONTACT.name(), null)));
+    action.run();
+    assertThat(response.getStatus()).isEqualTo(SC_BAD_REQUEST);
+    assertThat(response.getPayload())
+        .contains("Cannot append a global administrator or user with a global role");
+  }
+
+  @Test
+  void testFailure_appendUser_globalRole() throws IOException {
+    User user = DatabaseHelper.createAdminUser("email@email.com");
+    AuthResult authResult = AuthResult.createUser(user);
+    DatabaseHelper.persistResource(
+        new User.Builder()
+            .setEmailAddress("support@test.com")
+            .setUserRoles(
+                new UserRoles.Builder()
+                    .setIsAdmin(false)
+                    .setGlobalRole(GlobalRole.SUPPORT_AGENT)
+                    .build())
+            .build());
+
+    ConsoleUsersAction action =
+        createAction(
+            Optional.of(ConsoleApiParamsUtils.createFake(authResult)),
+            Optional.of("POST"),
+            Optional.of(
+                new UserData("support@test.com", null, RegistrarRole.TECH_CONTACT.name(), null)));
+    action.run();
+    assertThat(response.getStatus()).isEqualTo(SC_BAD_REQUEST);
+    assertThat(response.getPayload())
+        .contains("Cannot append a global administrator or user with a global role");
+  }
+
+  @Test
+  void testFailure_deleteUser_globalAdmin() throws IOException {
+    User user = DatabaseHelper.createAdminUser("email@email.com");
+    AuthResult authResult = AuthResult.createUser(user);
+    // Historically associated global admin
+    DatabaseHelper.persistResource(
+        new User.Builder()
+            .setEmailAddress("globaladmin@test.com")
+            .setUserRoles(
+                new UserRoles.Builder()
+                    .setIsAdmin(true)
+                    .setGlobalRole(GlobalRole.NONE)
+                    .setRegistrarRoles(
+                        ImmutableMap.of("TheRegistrar", RegistrarRole.PRIMARY_CONTACT))
+                    .build())
+            .build());
+
+    ConsoleUsersAction action =
+        createAction(
+            Optional.of(ConsoleApiParamsUtils.createFake(authResult)),
+            Optional.of("DELETE"),
+            Optional.of(
+                new UserData(
+                    "globaladmin@test.com", null, RegistrarRole.ACCOUNT_MANAGER.toString(), null)));
+    action.run();
+    assertThat(response.getStatus()).isEqualTo(SC_BAD_REQUEST);
+    assertThat(response.getPayload())
+        .contains("Cannot delete a global administrator or user with a global role");
+  }
+
  private ConsoleUsersAction createAction(
      Optional<ConsoleApiParams> maybeConsoleApiParams,
      Optional<String> method,
@@ -18,6 +18,7 @@ import static com.google.common.base.Preconditions.checkNotNull;

 import google.registry.util.UrlChecker;
 import java.net.MalformedURLException;
+import java.net.URI;
 import java.net.URL;
 import java.time.Duration;
 import java.time.temporal.ChronoUnit;
@@ -55,11 +56,12 @@ class DockerWebDriverExtension implements BeforeAllCallback, AfterAllCallback {
    URL url;
    try {
      url =
-          new URL(
-              String.format(
-                  "http://%s:%d",
-                  container.getContainerIpAddress(),
-                  container.getMappedPort(CHROME_DRIVER_SERVICE_PORT)));
+          URI.create(
+                  String.format(
+                      "http://%s:%d",
+                      container.getContainerIpAddress(),
+                      container.getMappedPort(CHROME_DRIVER_SERVICE_PORT)))
+              .toURL();
    } catch (MalformedURLException e) {
      throw new IllegalArgumentException(e);
    }
@@ -261,11 +261,11 @@ td.section {
    </tr>
    <tr>
     <td class="property_name">generated on</td>
-     <td class="property_value">2026-06-23 01:40:35</td>
+     <td class="property_value">2026-07-06 17:57:29</td>
    </tr>
    <tr>
     <td class="property_name">last flyway file</td>
-     <td id="lastFlywayFile" class="property_value">V223__tld_change_xap_enabled_to_transitions.sql</td>
+     <td id="lastFlywayFile" class="property_value">V224__add_registrar_expiry_access_period_enabled.sql</td>
    </tr>
   </tbody>
  </table>
@@ -273,7 +273,7 @@ td.section {
  <p>&nbsp;</p>
  <svg viewBox="0.00 0.00 4783.00 3613.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" id="erDiagram" style="overflow: hidden; width: 100%; height: 800px">
   <g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 3608.5)">
-    <title>SchemaCrawler_Diagram</title> <polygon fill="white" stroke="none" points="-4,4 -4,-3608.5 4778.75,-3608.5 4778.75,4 -4,4" /> <text xml:space="preserve" text-anchor="start" x="4535.5" y="-29.2" font-family="Helvetica,sans-Serif" font-size="14.00">generated by</text> <text xml:space="preserve" text-anchor="start" x="4618.25" y="-29.2" font-family="Helvetica,sans-Serif" font-size="14.00">SchemaCrawler 17.11.1</text> <text xml:space="preserve" text-anchor="start" x="4534.75" y="-9.45" font-family="Helvetica,sans-Serif" font-size="14.00">generated on</text> <text xml:space="preserve" text-anchor="start" x="4618.25" y="-9.45" font-family="Helvetica,sans-Serif" font-size="14.00">2026-06-23 01:40:35</text> <polygon fill="none" stroke="#888888" points="4531.75,-4 4531.75,-45.5 4766.75,-45.5 4766.75,-4 4531.75,-4" /> <!-- allocationtoken_a08ccbef -->
+    <title>SchemaCrawler_Diagram</title> <polygon fill="white" stroke="none" points="-4,4 -4,-3608.5 4778.75,-3608.5 4778.75,4 -4,4" /> <text xml:space="preserve" text-anchor="start" x="4535.5" y="-29.2" font-family="Helvetica,sans-Serif" font-size="14.00">generated by</text> <text xml:space="preserve" text-anchor="start" x="4618.25" y="-29.2" font-family="Helvetica,sans-Serif" font-size="14.00">SchemaCrawler 17.11.1</text> <text xml:space="preserve" text-anchor="start" x="4534.75" y="-9.45" font-family="Helvetica,sans-Serif" font-size="14.00">generated on</text> <text xml:space="preserve" text-anchor="start" x="4618.25" y="-9.45" font-family="Helvetica,sans-Serif" font-size="14.00">2026-07-06 17:57:29</text> <polygon fill="none" stroke="#888888" points="4531.75,-4 4531.75,-45.5 4766.75,-45.5 4766.75,-4 4531.75,-4" /> <!-- allocationtoken_a08ccbef -->
    <g id="node1" class="node">
     <title>allocationtoken_a08ccbef</title> <polygon fill="#e9c2f2" stroke="none" points="479.25,-1017.62 479.25,-1037.38 664.25,-1037.38 664.25,-1017.62 479.25,-1017.62" /> <text xml:space="preserve" text-anchor="start" x="481.25" y="-1023.08" font-family="Helvetica,sans-Serif" font-weight="bold" font-style="italic" font-size="14.00">public."AllocationToken"</text> <polygon fill="#e9c2f2" stroke="none" points="664.25,-1017.62 664.25,-1037.38 737.25,-1037.38 737.25,-1017.62 664.25,-1017.62" /> <text xml:space="preserve" text-anchor="start" x="698.5" y="-1022.08" font-family="Helvetica,sans-Serif" font-size="14.00">[table]</text> <text xml:space="preserve" text-anchor="start" x="481.25" y="-1003.33" font-family="Helvetica,sans-Serif" font-weight="bold" font-style="italic" font-size="14.00">token</text> <text xml:space="preserve" text-anchor="start" x="658.5" y="-1002.33" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text xml:space="preserve" text-anchor="start" x="666.25" y="-1002.33" font-family="Helvetica,sans-Serif" font-size="14.00">text not null</text> <text xml:space="preserve" text-anchor="start" x="481.25" y="-982.58" font-family="Helvetica,sans-Serif" font-size="14.00">domain_name</text> <text xml:space="preserve" text-anchor="start" x="658.5" y="-982.58" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text xml:space="preserve" text-anchor="start" x="666.25" y="-982.58" font-family="Helvetica,sans-Serif" font-size="14.00">text</text> <text xml:space="preserve" text-anchor="start" x="481.25" y="-962.83" font-family="Helvetica,sans-Serif" font-size="14.00">redemption_domain_repo_id</text> <text xml:space="preserve" text-anchor="start" x="658.5" y="-962.83" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text xml:space="preserve" text-anchor="start" x="666.25" y="-962.83" font-family="Helvetica,sans-Serif" font-size="14.00">text</text> <text xml:space="preserve" text-anchor="start" x="481.25" y="-943.08" font-family="Helvetica,sans-Serif" font-size="14.00">token_type</text> <text xml:space="preserve" text-anchor="start" x="658.5" y="-943.08" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text xml:space="preserve" text-anchor="start" x="666.25" y="-943.08" font-family="Helvetica,sans-Serif" font-size="14.00">text</text> <polygon fill="none" stroke="#888888" points="478.25,-937.62 478.25,-1038.38 738.25,-1038.38 738.25,-937.62 478.25,-937.62" />
    </g>
@@ -221,3 +221,4 @@ V220__domain_package_token_idx.sql
 V221__remove_contact_history.sql
 V222__remove_contact.sql
 V223__tld_change_xap_enabled_to_transitions.sql
+V224__add_registrar_expiry_access_period_enabled.sql
@@ -0,0 +1,20 @@
+-- Copyright 2026 The Nomulus Authors. All Rights Reserved.
+--
+-- Licensed under the Apache License, Version 2.0 (the "License");
+-- you may not use this file except in compliance with the License.
+-- You may obtain a copy of the License at
+--
+--     http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+
+-- Add the XAP opt-in column to Registrar, defaulting to false for all existing registrars.
+-- To ensure backward compatibility with running servers (old Java code) during
+-- the transition phase of the deployment, we set DEFAULT false NOT NULL.
+-- TODO(mcilwain): Drop this DEFAULT constraint in a subsequent schema release once the Java code has been fully deployed.
+ALTER TABLE "Registrar" ADD COLUMN expiry_access_period_enabled boolean
+    DEFAULT false NOT NULL;
@@ -855,7 +855,8 @@ CREATE TABLE public."Registrar" (
    whois_server text,
    last_expiring_cert_notification_sent_date timestamp with time zone,
    last_expiring_failover_cert_notification_sent_date timestamp with time zone,
-    last_poc_verification_date timestamp with time zone
+    last_poc_verification_date timestamp with time zone,
+    expiry_access_period_enabled boolean DEFAULT false NOT NULL
 );


@@ -1,2 +1,2 @@
 out/
-src/main/resources/google/registry/monitoring/blackbox/modules/secrets/
+src/main/resources/google/registry/monitoring/blackbox/module/secrets/
@@ -9,8 +9,6 @@
          <domain:hostObj>ns.fake-domain.tld</domain:hostObj>
        </domain:ns>
        <domain:registrant>google-mon</domain:registrant>
-        <domain:contact type="admin">google-mon</domain:contact>
-        <domain:contact type="tech">google-mon</domain:contact>
        <domain:authInfo>
          <domain:pw>insecure</domain:pw>
        </domain:authInfo>
@@ -185,7 +185,7 @@ public class TokenStore {
          if (Duration.between(availableTokens.timestamp(), clock.now())
                  .compareTo(config.getRefreshPeriod())
              >= 0) {
-            tokensMap.remove(user);
+            tokensMap.remove(user, availableTokens);
          }
        });
  }
Author	SHA1	Message	Date
gbrodman	0bf0b4fc66	Require EPP validation for complex OT&E stats (#3107 ) Previously, if there were no XML bytes somehow, it'd increment a bunch of counts for requirements that shouldn't be incremented. We should only increment these values if we have the data to do so. Note: this doesn't happen and shouldn't happen, so there's no issues currently occurring, but this code is more correct anyway.	2026-07-07 20:05:01 +00:00
Ben McIlwain	2ce91e3477	Add loadAllOfSorted to TransactionManager (#3136 ) During our implementation of Expiry Access Period (XAP), adding a field to the Registrar entity altered database heap scan order when loadAllOf() was called without an explicit ORDER BY clause. This caused non-deterministic row shifting in the Angular console registrar table (/console/registrars), breaking golden image comparisons in ConsoleScreenshotTest. To permanently fix this root cause and guarantee deterministic UI rendering and test stability across schema evolutions, this commit introduces sorted database loading across our persistence layer. Specifically, this commit: - Adds loadAllOfSorted and loadAllOfSortedStream to TransactionManager, JpaTransactionManagerImpl, and DelegatingReplicaJpaTransactionManager. - Enforces strict allowlist regex validation (^[a-zA-Z0-9_.]+$) on property names in JpaTransactionManagerImpl before appending dynamic ORDER BY clauses, preventing JPQL/SQL injection since bind parameters cannot be used for schema identifiers. - Adds Registrar.loadAllSorted and updates RegistrarsAction to explicitly sort registrars alphabetically by registrarName and registrarId. - Updates golden screenshot ConsoleScreenshotTest_globalRole_registrars_registrarsPage.png to reflect the newly enforced deterministic alphabetical registrar ordering. - Adds comprehensive unit test coverage in JpaTransactionManagerImplTest verifying exact sorted entity retrieval across multiple fields and regex rejection of invalid field names. TAG=agy CONV=f2488c74-8b4a-43f1-9c22-d1dddbdbb4e0 BUG=http://b/531889856	2026-07-07 19:44:56 +00:00
gbrodman	4d6b5a82df	Forbid negative premium prices (#3108 ) This hasn't been an issue, but it seems like a good idea. Note that I'm allowing prices of 0 here just in case there's some shenanigans at some point in time where we want a domain to technically be premium without having a cost.	2026-07-07 19:33:10 +00:00
gbrodman	1fe1043306	Enforce GCS Public Access Prevention (#3133 ) Implement programmatic verification of Public Access Prevention (PAP) on all GCS buckets before performing write operations via GcsUtils. - Add verifyPublicAccessPrevention(String bucketName) to GcsUtils.java, querying the bucket's IAM configuration and asserting that PAP is ENFORCED. - Call verifyPublicAccessPrevention in all output stream and byte creation write paths in GcsUtils.	2026-07-07 18:37:58 +00:00
gbrodman	b1e42cfd5e	Verify billing recurrence expansion is caught up before invoicing (#3121 )	2026-07-07 18:05:48 +00:00
gbrodman	0fa82e30bb	Fix typo in .gitignore (#3130 )	2026-07-07 16:08:26 +00:00
Ben McIlwain	6608ee282d	Add XAP registrar opt-in database schema (#3134 ) This commit implements the first stage of the Two-PR database schema deployment for registrar opt-in to the Expiry Access Period (XAP), along with updating GEMINI.md to mandate strict adherence to db/README.md for all future database schema modifications. Specifically, it: - Adds Flyway migration V224__add_registrar_expiry_access_period_enabled.sql, which adds the expiry_access_period_enabled boolean column (DEFAULT false NOT NULL) to the Registrar table. - Updates flyway.txt, nomulus.golden.sql, and ER diagrams to reflect the schema changes. - Updates GEMINI.md with explicit instructions for AI agents to always consult db/README.md, follow the mandatory Two-PR deployment split when altering database schemas, and run both :db:test and :core:sqlIntegrationTest whenever database schema or ORM mappings are touched. - Updates golden screenshot ConsoleScreenshotTest_globalRole_registrars_registrarsPage.png to reflect the new default database ordering after adding the column. (Note: A separate pull request, PR #3136 / http://go/r3pr/3136, permanently fixes this underlying sort non-determinism across our persistence layer by enforcing deterministic alphabetical ordering in Registrar.loadAllSorted). TAG=agy CONV=f2488c74-8b4a-43f1-9c22-d1dddbdbb4e0 BUG=http://b/437398822	2026-07-07 03:44:37 +00:00
gbrodman	22c867f3f2	Add TLD verification to GenerateZoneFilesAction (#3132 ) 1. this is a good idea in general 2. it can prevent GCS path traversal if someone somehow bypasses auth to add a weird path	2026-07-06 19:26:42 +00:00
gbrodman	160abef731	Throw on invalid responses from SafeBrowsing (#3111 ) this allows us to 1. use the built-in retrier to retry 2. fail loudly if it keeps actually throwing an exception if we're doing something invalid	2026-07-06 18:52:13 +00:00
gbrodman	a6e4017971	Join cancellations on billing time as well (#3124 ) This is a small edge case where a cancellation for year 2 could cancel out events in year 1 if we re-ran billing for year 1 for some reason. I don't think we would ever re-run one year later so this is kind of moot, but it's probably still a good thing to do.	2026-07-06 18:51:51 +00:00
gbrodman	d9a857133a	Remove class name option in GetRoutingMapCommand (#3101 ) we only have the one component now, and in general it's a good idea to remove class loading based on user input, even if that user is one of us	2026-07-01 18:55:26 +00:00
gbrodman	c7a27061d8	Remove entries from token store atomically (#3112 ) just in case some other thread comes in while we're iterating	2026-07-01 18:42:59 +00:00
Weimin Yu	7766db36a7	Support custom npm location in local environment (#3127 ) Make sure Gradle can find the npm executable at custom locations, e.g., when installed by nvm.	2026-07-01 18:39:29 +00:00
Ben McIlwain	eda0f7ad7c	Harden EppXmlSanitizer against XXE and entity expansion (#3117 ) Configure a throwing XMLResolver on XMLInputFactory in XmlTransformer to act as an additional defense-in-depth security layer against XML External Entity (XXE) and entity expansion/resolution attacks. This prevents resolution of external and internal entities in StAX parsing pipelines, ensuring that malformed payloads containing entity definitions are safely blocked and rejected. Also add a unit test to EppXmlSanitizerTest verifying that EPP messages containing DTD and external entities fail parsing and fall back safely to their Base64 representation. TAG=agy CONV=610c2358-a99f-4605-94cd-ff0d4ee08176 BUG= http://b/529387728	2026-06-30 18:19:50 +00:00
gbrodman	67527f1560	Invalidate TLD cache on any type of update (#3123 )	2026-06-30 18:15:45 +00:00
Ben McIlwain	4aeba6e3f7	Validate non-empty tlds in refresh DNS action (#3114 ) Add validation to RefreshDnsForAllDomainsAction to ensure the tlds parameter is not empty when invoked. Previously, invoking the action without specifying tlds would result in a no-op that logged an empty list and exited without error. This change makes missing tlds an explicit error, throwing an IllegalArgumentException so users running the command via nomulus curl receive clear feedback that the parameter is required.	2026-06-30 16:34:15 +00:00
Ben McIlwain	d6f1f5894b	Restore RFC schemas, prober XML, and OT&E route (#3120 ) In commit `17b851de42` (#2979), <element name="contact"> and related types were incorrectly removed from core/src/main/java/google/registry/xml/xsd/domain.xsd and rde-domain.xsd. Because those XSD schemas are published standards (RFC 5731 and RFC 5831), altering them caused JAXB unmarshaling to reject historical database records (HistoryEntry) containing <domain:contact> elements during OT&E status checks. Restore <element name="contact" minOccurs="0" maxOccurs="unbounded"/> to domain.xsd and rde-domain.xsd so all XSD schemas conform strictly to published EPP RFCs and historical contact elements unmarshal cleanly without throwing cvc-complex-type.2.4.a schema validation errors. Remove deprecated <domain:contact> elements from prober creation payload (prober/.../create.xml) to match current flow rules where contact associations are forbidden. Remove trailing slash from ote-status route parameter in RegistrarDetailsComponent to prevent double-slash (/ote-status//<id>) URL generation when checking OT&E status. BUG= http://b/529391412	2026-06-30 16:07:31 +00:00
gbrodman	47ad569cb0	Verify that marksdb verify URL starts with ry.marksdb.org (#3115 )	2026-06-30 16:02:00 +00:00
gbrodman	06934daf94	Don't allow registrar-based ops on global users (#3100 )	2026-06-29 19:13:20 +00:00
gbrodman	9a032e4bb9	Add extra perm check for EPP password changes (#3099 )	2026-06-29 18:30:52 +00:00