Require EPP validation for complex OT&E stats (#3107)

Previously, if there were no XML bytes somehow, it'd increment a bunch
of counts for requirements that shouldn't be incremented. We should only
increment these values if we have the data to do so.

Note: this doesn't happen and shouldn't happen, so there's no issues
currently occurring, but this code is more correct anyway.

GEMINI.md

@@ -44,6 +44,11 @@ This document outlines foundational mandates, architectural patterns, and projec
         - **Test Helpers & Timestamps:** If a static test helper method (like in `DatabaseHelper`) needs the database transaction time but might be called from outside a transaction, using `tm().reTransact(tm()::getTxTime)` is acceptable. However, NEVER wrap it redundantly like `tm().transact(() -> tm().reTransact(tm()::getTxTime))`. If you are just setting an arbitrary timestamp in a test where the exact DB transaction time isn't strictly required, prefer `Instant.now()` or `clock.now()` to avoid creating unnecessary database transactions.
         - **Production Code:** In production code, if a flow fails because it is calling `getTxTime()` outside of a transaction, you must wrap the *caller* in a transaction instead of adding an unnecessary `reTransact()` around `getTxTime()`.
     - **Transactional Time:** Ensure code that relies on `tm().getTransactionTime()` (or `tm().getTxTime()`) is executed within a transaction context.
+- **Database Schema Migrations & 2-PR Split Mandate:**
+    - **Mandatory Consultation of `db/README.md`:** Before planning, drafting, or executing any database schema modifications (e.g., adding/altering columns, creating Flyway `.sql` migration scripts, or modifying JPA entity `@Column` mappings), you **MUST read and strictly adhere to `db/README.md`**.
+    - **Strict 2-PR Deployment Split:** Never propose or submit combining Flyway SQL scripts (`db/src/main/resources/sql/flyway/V*.sql`) and Java ORM changes (`.java` entity files + `db-schema.sql.generated`) into a single PR for submission to `master`. Because live servers during a rolling deployment will fail if Java code attempts to access unmigrated database columns/constraints, all schema additions must be split into two sequential PRs per `db/README.md`:
+        1. **PR #1 (Database Schema Only):** Contains *only* the new Flyway `.sql` script, the `flyway.txt` index update (`:db:generateFlywayIndex`), the `nomulus.golden.sql` dump (`:nom:generate_golden_file`), and ER diagrams (`er_diagram/`). Must contain **zero `.java` files or `db-schema.sql.generated` changes**.
+        2. **PR #2 (Java ORM, EPP Flows & Generated Schema Map):** Submitted *only after* PR #1 is deployed to production. Contains all `.java` entity/flow modifications, tests, and the regenerated `db-schema.sql.generated` (`generateSqlSchema`).
 
 ### 5. Testing Best Practices
 - **Mandatory Proactive Testing:** You MUST automatically write and update tests alongside your code changes WITHOUT waiting for the user to prompt you. If you add a new feature, fix a bug, or change core logic, you are explicitly required to identify the corresponding `*Test.java` file and implement comprehensive test coverage for your changes.
@@ -51,6 +56,7 @@ This document outlines foundational mandates, architectural patterns, and projec
 - **Empirical Reproduction:** Before fixing a bug, always create a test case that reproduces the failure.
 - **Base Classes:** Leverage `CommandTestCase`, `EppToolCommandTestCase`, etc., to reduce boilerplate and ensure consistent setup (e.g., clock initialization).
 - **Gradle Test Patterns:** When running tests to investigate fixes in the "core" directory, try to first use the "standardTest" Gradle task. It is faster than the "test" task, which includes the "fragileTest" task. Only run the full "test" task after "standardTest" succeeds.
+- **Mandatory SQL Integration Verification:** Whenever you modify any database schema, Flyway script (`.sql`), or JPA entity class, you MUST explicitly run both `./gradlew :db:test` and `./gradlew :core:sqlIntegrationTest` **in addition to** the standard test suites (`./gradlew standardTest` or `./gradlew test`) before finalizing the task or declaring completion. Do not rely solely on `standardTest` when database schemas or ORM mappings are touched; all three test suites (`standardTest`, `:db:test`, and `:core:sqlIntegrationTest`) are mandatory.
 
 ### 6. Project Dependencies
 - **Common Module:** When using `Clock` or other core utilities in a new or separate module (like `load-testing`), ensure `implementation project(':common')` is added to the module's `build.gradle`.
@@ -171,3 +177,4 @@ This protocol defines the standard for interacting with GitHub repositories and
 - **One Commit Per PR:** Ensure all changes are squashed into a single, clean commit. Use `git commit --amend --no-edit` for follow-up fixes.
 - **Clean Workspace:** Always run `git status` and verify the repository state before declaring a task complete.
 - **Package Lock:** The Gradle build automatically modifies `console-webapp/package-lock.json` via the `npmInstallDeps` task. ALWAYS revert this file (`git checkout console-webapp/package-lock.json`) before staging changes or finalizing a commit unless you explicitly modified NPM dependencies.
+- **PR Description Synchronization:** Whenever you amend or update a commit's description, you **MUST** check whether the corresponding GitHub PR description (if one exists) matches the previous commit description. If the PR description was not manually customized (i.e., it simply reflects the older commit description), you must automatically update it via `gh pr edit` to keep it in sync with your newly updated commit description—**strictly after the updated commit has been pushed to the remote GitHub PR branch**. Do not update the remote PR description when changes are only committed locally. **CRITICAL:** When updating the PR description, you must strictly preserve any automated review links or footer blocks (such as Reviewable link blocks: `<!-- Reviewable:start -->...<!-- Reviewable:end -->`) at the bottom of the description.

core/src/main/java/google/registry/beam/spec11/SafeBrowsingTransforms.java

@@ -226,18 +226,17 @@ public class SafeBrowsingTransforms {
     private void processResponse(
         CloseableHttpResponse response,
         ImmutableSet.Builder<KV<DomainNameInfo, ThreatMatch>> resultBuilder)
-        throws JSONException, IOException {
+        throws IOException {
       int statusCode = response.getStatusLine().getStatusCode();
       if (statusCode != SC_OK) {
-        logger.atWarning().log("Got unexpected status code %s from response.", statusCode);
-      } else {
-        // Unpack the response body
-        JSONObject responseBody =
-            new JSONObject(
-                CharStreams.toString(
-                    new InputStreamReader(response.getEntity().getContent(), UTF_8)));
-        logger.atInfo().log("Got response: %s", responseBody);
-        if (responseBody.length() == 0) {
+        throw new IOException(
+            String.format("Got unexpected status code %s from response.", statusCode));
+      }
+      // Unpack the response body
+      try (InputStreamReader reader =
+          new InputStreamReader(response.getEntity().getContent(), UTF_8)) {
+        JSONObject responseBody = new JSONObject(CharStreams.toString(reader));
+        if (responseBody.isEmpty()) {
           logger.atInfo().log("Response was empty, no threats detected.");
         } else {
           // Emit all DomainNameInfos with their API results.

core/src/main/java/google/registry/gcs/GcsUtils.java

@@ -14,12 +14,15 @@
 
 package google.registry.gcs;
 
+import static com.google.common.base.Preconditions.checkState;
 import static com.google.common.collect.ImmutableList.toImmutableList;
 import static com.google.common.collect.Iterables.getLast;
 
 import com.google.cloud.storage.Blob;
 import com.google.cloud.storage.BlobId;
 import com.google.cloud.storage.BlobInfo;
+import com.google.cloud.storage.Bucket;
+import com.google.cloud.storage.BucketInfo;
 import com.google.cloud.storage.Storage;
 import com.google.cloud.storage.Storage.BlobListOption;
 import com.google.cloud.storage.StorageException;
@@ -33,12 +36,14 @@ import com.google.common.flogger.FluentLogger;
 import com.google.common.net.MediaType;
 import google.registry.config.CredentialModule.ApplicationDefaultCredential;
 import google.registry.util.GoogleCredentialsBundle;
+import google.registry.util.RegistryEnvironment;
 import jakarta.inject.Inject;
 import java.io.IOException;
 import java.io.InputStream;
 import java.io.OutputStream;
 import java.io.Serializable;
 import java.nio.channels.Channels;
+import java.util.concurrent.ConcurrentHashMap;
 import javax.annotation.CheckReturnValue;
 
 /**
@@ -50,6 +55,8 @@ public class GcsUtils implements Serializable {
 
   private static final FluentLogger logger = FluentLogger.forEnclosingClass();
 
+  private static final ConcurrentHashMap<String, Boolean> PAP_CACHE = new ConcurrentHashMap<>();
+
   private static final ImmutableMap<String, MediaType> EXTENSIONS =
       new ImmutableMap.Builder<String, MediaType>()
           .put("ghostryde", MediaType.APPLICATION_BINARY)
@@ -85,6 +92,7 @@ public class GcsUtils implements Serializable {
   /** Opens a GCS file for writing as an {@link OutputStream}, overwriting existing files. */
   @CheckReturnValue
   public OutputStream openOutputStream(BlobId blobId) {
+    verifyPublicAccessPrevention(blobId.getBucket());
     return Channels.newOutputStream(storage().writer(createBlobInfo(blobId)));
   }
 
@@ -94,6 +102,7 @@ public class GcsUtils implements Serializable {
    */
   @CheckReturnValue
   public OutputStream openOutputStream(BlobId blobId, ImmutableMap<String, String> metadata) {
+    verifyPublicAccessPrevention(blobId.getBucket());
     return Channels.newOutputStream(
         storage().writer(BlobInfo.newBuilder(blobId).setMetadata(metadata).build()));
   }
@@ -105,6 +114,7 @@ public class GcsUtils implements Serializable {
 
   /** Creates a GCS file with the given byte contents and metadata, overwriting existing files. */
   public void createFromBytes(BlobInfo blobInfo, byte[] bytes) throws StorageException {
+    verifyPublicAccessPrevention(blobInfo.getBucket());
     storage().create(blobInfo, bytes);
   }
 
@@ -120,6 +130,7 @@ public class GcsUtils implements Serializable {
 
   /** Update file content type on existing GCS file */
   public void updateContentType(BlobId blobId, String contentType) throws StorageException {
+    verifyPublicAccessPrevention(blobId.getBucket());
     if (existsAndNotEmpty(blobId)) {
       Blob blob = storage().get(blobId);
       blob.toBuilder().setContentType(contentType).build().update();
@@ -154,12 +165,6 @@ public class GcsUtils implements Serializable {
     }
   }
 
-  /** Returns the user defined metadata of a GCS file if the file exists, or an empty map. */
-  public ImmutableMap<String, String> getMetadata(BlobId blobId) throws StorageException {
-    Blob blob = storage().get(blobId);
-    return blob == null ? ImmutableMap.of() : ImmutableMap.copyOf(blob.getMetadata());
-  }
-
   /**
    * Returns the {@link BlobInfo} of the given GCS file.
    *
@@ -179,6 +184,37 @@ public class GcsUtils implements Serializable {
     return builder.build();
   }
 
+  /**
+   * Asserts that Public Access Prevention (PAP) is enforced on the GCS bucket.
+   *
+   * @throws IllegalStateException if PAP is not ENFORCED.
+   */
+  @VisibleForTesting
+  void verifyPublicAccessPrevention(String bucketName) {
+    if (RegistryEnvironment.get() != RegistryEnvironment.PRODUCTION) {
+      return;
+    }
+    PAP_CACHE.computeIfAbsent(
+        bucketName,
+        name -> {
+          Bucket bucket = storage().get(name);
+          checkState(bucket != null, "Bucket %s does not exist", name);
+          BucketInfo.PublicAccessPrevention pap =
+              bucket.getIamConfiguration().getPublicAccessPrevention();
+          checkState(
+              pap == BucketInfo.PublicAccessPrevention.ENFORCED,
+              "Public Access Prevention is not enforced on bucket %s. Current state: %s",
+              name,
+              pap);
+          return true;
+        });
+  }
+
+  @VisibleForTesting
+  static void clearPapCache() {
+    PAP_CACHE.clear();
+  }
+
   // These two methods are needed to check whether serialization is done correctly in tests.
   @Override
   public boolean equals(Object obj) {

core/src/main/java/google/registry/model/OteStats.java

@@ -176,11 +176,12 @@ public class OteStats {
      * Check if the {@link HistoryEntry} type matches as well as the {@link EppInput} if supplied.
      */
     private boolean matches(HistoryEntry.Type historyType, Optional<EppInput> eppInput) {
-      if (eppInputFilter.isPresent() && eppInput.isPresent()) {
-        return typeFilter.test(historyType) && eppInputFilter.get().test(eppInput.get());
-      } else {
-        return typeFilter.test(historyType);
+      if (!typeFilter.test(historyType)) {
+        return false;
       }
+      return eppInputFilter
+          .map(filter -> eppInput.isPresent() && filter.test(eppInput.get()))
+          .orElse(true);
     }
   }
 

core/src/main/java/google/registry/model/registrar/Registrar.java

@@ -1030,6 +1030,11 @@ public class Registrar extends UpdateAutoTimestampEntity implements Buildable, J
     return tm().transact(() -> tm().loadAllOf(Registrar.class));
   }
 
+  /** Loads all registrar entities directly from the database, sorted by the given field names. */
+  public static Iterable<Registrar> loadAllSorted(String... sortFields) {
+    return tm().transact(() -> tm().loadAllOfSorted(Registrar.class, sortFields));
+  }
+
   /** Loads all registrar entities using an in-memory cache. */
   public static Iterable<Registrar> loadAllCached() {
     return CACHE_BY_REGISTRAR_ID.get().values();

core/src/main/java/google/registry/model/tld/label/PremiumList.java

@@ -169,6 +169,14 @@ public final class PremiumList extends BaseDomainLabelList<BigDecimal, PremiumEn
         getInstance().revisionId = revisionId;
         return this;
       }
+
+      @Override
+      public PremiumEntry build() {
+        checkArgument(getInstance().price != null, "Price must not be null");
+        checkArgument(
+            getInstance().price.compareTo(BigDecimal.ZERO) >= 0, "Price must not be negative");
+        return super.build();
+      }
     }
   }
 

core/src/main/java/google/registry/persistence/transaction/DelegatingReplicaJpaTransactionManager.java

@@ -257,11 +257,21 @@ public class DelegatingReplicaJpaTransactionManager implements JpaTransactionMan
     return getReplica().loadAllOf(clazz);
   }
 
+  @Override
+  public <T> ImmutableList<T> loadAllOfSorted(Class<T> clazz, String... sortFields) {
+    return getReplica().loadAllOfSorted(clazz, sortFields);
+  }
+
   @Override
   public <T> Stream<T> loadAllOfStream(Class<T> clazz) {
     return getReplica().loadAllOfStream(clazz);
   }
 
+  @Override
+  public <T> Stream<T> loadAllOfSortedStream(Class<T> clazz, String... sortFields) {
+    return getReplica().loadAllOfSortedStream(clazz, sortFields);
+  }
+
   @Override
   public <T> Optional<T> loadSingleton(Class<T> clazz) {
     return getReplica().loadSingleton(clazz);

core/src/main/java/google/registry/persistence/transaction/JpaTransactionManagerImpl.java

@@ -76,6 +76,7 @@ import java.util.concurrent.TimeUnit;
 import java.util.concurrent.atomic.AtomicLong;
 import java.util.function.Supplier;
 import java.util.function.UnaryOperator;
+import java.util.regex.Pattern;
 import java.util.stream.Stream;
 import java.util.stream.StreamSupport;
 import javax.annotation.Nullable;
@@ -88,6 +89,16 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
 
   private static final FluentLogger logger = FluentLogger.forEnclosingClass();
   private static final Retrier retrier = new Retrier(new SystemSleeper(), 6);
+
+  /**
+   * Strict allowlist regex for property/field names in dynamic JPQL ORDER BY clauses.
+   *
+   * <p>JPA and database engines forbid bind parameters (e.g. ? or :param) for schema identifiers or
+   * property names in ORDER BY clauses. To prevent JPQL/SQL injection when dynamically constructing
+   * sort queries, every sort field MUST be validated against this pattern before concatenation.
+   */
+  private static final Pattern VALID_SORT_FIELD_PATTERN = Pattern.compile("^[a-zA-Z0-9_.]+$");
+
   private static final String NESTED_TRANSACTION_MESSAGE =
       "Nested transaction detected. Try refactoring to avoid nested transactions. If unachievable,"
           + " use reTransact() in nested transactions";
@@ -528,15 +539,38 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
 
   @Override
   public <T> ImmutableList<T> loadAllOf(Class<T> clazz) {
-    return loadAllOfStream(clazz).collect(toImmutableList());
+    return loadAllOfSorted(clazz);
   }
 
   @Override
   public <T> Stream<T> loadAllOfStream(Class<T> clazz) {
+    return loadAllOfSortedStream(clazz);
+  }
+
+  @Override
+  public <T> ImmutableList<T> loadAllOfSorted(Class<T> clazz, String... sortFields) {
+    return loadAllOfSortedStream(clazz, sortFields).collect(toImmutableList());
+  }
+
+  @Override
+  public <T> Stream<T> loadAllOfSortedStream(Class<T> clazz, String... sortFields) {
     checkArgumentNotNull(clazz, "clazz must be specified");
+    checkArgumentNotNull(sortFields, "sortFields must not be null");
     assertInTransaction();
+    StringBuilder queryString =
+        new StringBuilder(String.format("FROM %s", getEntityType(clazz).getName()));
+    if (sortFields.length > 0) {
+      for (String field : sortFields) {
+        checkArgument(
+            VALID_SORT_FIELD_PATTERN.matcher(field).matches(),
+            "Invalid sort field name: %s",
+            field);
+      }
+      queryString.append(" ORDER BY ");
+      queryString.append(String.join(", ", sortFields));
+    }
     return getEntityManager()
-        .createQuery(String.format("FROM %s", getEntityType(clazz).getName()), clazz)
+        .createQuery(queryString.toString(), clazz)
         .getResultStream()
         .map(this::detach);
   }

core/src/main/java/google/registry/persistence/transaction/TransactionManager.java

@@ -219,6 +219,14 @@ public interface TransactionManager {
    */
   <T> ImmutableList<T> loadAllOf(Class<T> clazz);
 
+  /**
+   * Returns a list of all entities of the given type that exist in the database, ordered by the
+   * specified field names in ascending order.
+   *
+   * <p>The resulting list is empty if there are no entities of this type.
+   */
+  <T> ImmutableList<T> loadAllOfSorted(Class<T> clazz, String... sortFields);
+
   /**
    * Returns a stream of all entities of the given type that exist in the database.
    *
@@ -226,6 +234,14 @@ public interface TransactionManager {
    */
   <T> Stream<T> loadAllOfStream(Class<T> clazz);
 
+  /**
+   * Returns a stream of all entities of the given type that exist in the database, ordered by the
+   * specified field names in ascending order.
+   *
+   * <p>The resulting stream is empty if there are no entities of this type.
+   */
+  <T> Stream<T> loadAllOfSortedStream(Class<T> clazz, String... sortFields);
+
   /**
    * Loads the only instance of this particular class, or empty if none exists.
    *

core/src/main/java/google/registry/reporting/billing/GenerateInvoicesAction.java

@@ -14,10 +14,14 @@
 
 package google.registry.reporting.billing;
 
+import static com.google.common.base.Preconditions.checkState;
 import static google.registry.beam.BeamUtils.createJobName;
+import static google.registry.model.common.Cursor.CursorType.RECURRING_BILLING;
+import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
 import static google.registry.request.Action.Method.POST;
 import static jakarta.servlet.http.HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
 import static jakarta.servlet.http.HttpServletResponse.SC_OK;
+import static java.time.ZoneOffset.UTC;
 
 import com.google.api.services.dataflow.Dataflow;
 import com.google.api.services.dataflow.model.LaunchFlexTemplateParameter;
@@ -29,6 +33,7 @@ import com.google.common.flogger.FluentLogger;
 import com.google.common.net.MediaType;
 import google.registry.batch.CloudTasksUtils;
 import google.registry.config.RegistryConfig.Config;
+import google.registry.model.common.Cursor;
 import google.registry.persistence.PersistenceModule;
 import google.registry.reporting.ReportingModule;
 import google.registry.request.Action;
@@ -40,7 +45,9 @@ import google.registry.util.RegistryEnvironment;
 import jakarta.inject.Inject;
 import java.io.IOException;
 import java.time.Duration;
+import java.time.Instant;
 import java.time.YearMonth;
+import java.util.Optional;
 
 /**
  * Invokes the {@code InvoicingPipeline} beam template via the REST api, and enqueues the {@link
@@ -107,6 +114,7 @@ public class GenerateInvoicesAction implements Runnable {
     response.setContentType(MediaType.PLAIN_TEXT_UTF_8);
     logger.atInfo().log("Launching invoicing pipeline for %s.", yearMonth);
     try {
+      checkBillingRecurrenceCursor();
       LaunchFlexTemplateParameter parameter =
           new LaunchFlexTemplateParameter()
               .setJobName(createJobName("invoicing", clock))
@@ -156,4 +164,20 @@ public class GenerateInvoicesAction implements Runnable {
       response.setPayload(String.format("Pipeline launch failed: %s", e.getMessage()));
     }
   }
+
+  private void checkBillingRecurrenceCursor() {
+    Optional<Cursor> previousCursor =
+        tm().transact(() -> tm().loadByKeyIfPresent(Cursor.createGlobalVKey(RECURRING_BILLING)));
+    checkState(
+        previousCursor.isPresent(),
+        "BillingRecurrence expansion cursor is not present. Run ExpandBillingRecurrencesAction.");
+    Instant startOfNextMonth = yearMonth.plusMonths(1).atDay(1).atStartOfDay(UTC).toInstant();
+    Instant previousCursorTime = previousCursor.get().getCursorTime();
+    checkState(
+        !previousCursorTime.isBefore(startOfNextMonth),
+        "BillingRecurrence expansion cursor (%s) is before the start of the next month (%s). "
+            + "Run ExpandBillingRecurrencesAction.",
+        previousCursorTime,
+        startOfNextMonth);
+  }
 }

core/src/main/java/google/registry/tools/server/GenerateZoneFilesAction.java

@@ -33,6 +33,7 @@ import google.registry.model.domain.Domain;
 import google.registry.model.domain.secdns.DomainDsData;
 import google.registry.model.host.Host;
 import google.registry.model.tld.Tld;
+import google.registry.model.tld.Tlds;
 import google.registry.request.Action;
 import google.registry.request.HttpException.BadRequestException;
 import google.registry.request.JsonActionRunner;
@@ -119,6 +120,7 @@ public class GenerateZoneFilesAction implements Runnable, JsonActionRunner.JsonA
   public Map<String, Object> handleJsonRequest(Map<String, ?> json) {
     @SuppressWarnings("unchecked")
     ImmutableSet<String> tlds = ImmutableSet.copyOf((List<String>) json.get("tlds"));
+    Tlds.assertTldsExist(tlds);
     Instant exportTime = Instant.parse(json.get("exportTime").toString());
     // We disallow exporting within the past 2 minutes because there might be outstanding writes.
     // We can only reliably call loadAtPointInTime at times that are UTC midnight and >

core/src/main/java/google/registry/ui/server/console/RegistrarsAction.java

@@ -59,6 +59,7 @@ public class RegistrarsAction extends ConsoleApiAction {
       """
             SELECT * FROM "Registrar"
             WHERE registrar_id in :registrarIds
+            ORDER BY registrar_name ASC, registrar_id ASC
       """;
   static final String PATH = "/console-api/registrars";
   private final Optional<Registrar> registrar;
@@ -83,7 +84,7 @@ public class RegistrarsAction extends ConsoleApiAction {
       ImmutableSet<Registrar.Type> allowedRegistrarTypes =
           user.getUserRoles().isAdmin() ? TYPES_ALLOWED_FOR_ADMINS : TYPES_ALLOWED_FOR_USERS;
       ImmutableList<Registrar> registrars =
-          Streams.stream(Registrar.loadAll())
+          Streams.stream(Registrar.loadAllSorted("registrarName", "registrarId"))
               .filter(r -> allowedRegistrarTypes.contains(r.getType()))
               .collect(ImmutableList.toImmutableList());
       consoleApiParams.response().setPayload(consoleApiParams.gson().toJson(registrars));

core/src/main/resources/google/registry/beam/billing/sql/cloud_sql_billing_events.sql

@@ -30,7 +30,7 @@ JOIN Registrar r ON b.clientId = r.registrarId
 JOIN Domain d ON b.domainRepoId = d.repoId
 JOIN Tld t ON t.tldStr = d.tld
 LEFT JOIN BillingCancellation c ON b.id = c.billingEvent
-LEFT JOIN BillingCancellation cr ON b.cancellationMatchingBillingEvent = cr.billingRecurrence
+LEFT JOIN BillingCancellation cr ON b.cancellationMatchingBillingEvent = cr.billingRecurrence AND b.billingTime = cr.billingTime
 WHERE r.billingAccountMap IS NOT NULL
 AND r.type = 'REAL'
 AND t.invoicingEnabled IS TRUE

core/src/test/java/google/registry/beam/billing/InvoicingPipelineTest.java

@@ -229,7 +229,22 @@ class InvoicingPipelineTest {
               3,
               "USD",
               20.5,
-              ""));
+              ""),
+          google.registry.beam.billing.BillingEvent.create(
+              17,
+              Instant.parse("2017-10-04T00:00:00Z"),
+              Instant.parse("2017-10-04T00:00:00Z"),
+              "theRegistrar",
+              "234",
+              "",
+              "test",
+              "RENEW",
+              "recurrence-collision.test",
+              "REPO-ID",
+              3,
+              "USD",
+              20.5,
+              "SYNTHETIC"));
 
   private static final ImmutableMap<String, ImmutableList<String>> EXPECTED_DETAILED_REPORT_MAP =
       ImmutableMap.of(
@@ -239,6 +254,8 @@ class InvoicingPipelineTest {
                   + "test,RENEW,mydomain2.test,REPO-ID,3,USD,20.50,",
               "1,2017-10-04 00:00:00 UTC,2017-10-04 00:00:00 UTC,theRegistrar,234,,"
                   + "test,RENEW,mydomain.test,REPO-ID,3,USD,20.50,",
+              "17,2017-10-04 00:00:00 UTC,2017-10-04 00:00:00 UTC,theRegistrar,234,,"
+                  + "test,RENEW,recurrence-collision.test,REPO-ID,3,USD,20.50,",
               "7,2017-10-04 00:00:00 UTC,2017-10-04 00:00:00 UTC,theRegistrar,234,,"
                   + "test,SERVER_STATUS,update-prohibited.test,REPO-ID,0,USD,20.00,",
               "6,2017-10-04 00:00:00 UTC,2017-10-04 00:00:00 UTC,theRegistrar,234,,"
@@ -264,7 +281,7 @@ class InvoicingPipelineTest {
 
   private static final ImmutableList<String> EXPECTED_INVOICE_OUTPUT =
       ImmutableList.of(
-          "2017-10-01,2020-09-30,234,61.50,USD,10125,1,PURCHASE,,3,"
+          "2017-10-01,2020-09-30,234,82.00,USD,10125,1,PURCHASE,,4,"
               + "RENEW | TLD: test | TERM: 3-year,20.50,USD,",
           "2017-10-01,2022-09-30,234,70.00,JPY,10125,1,PURCHASE,,1,"
               + "CREATE | TLD: hello | TERM: 5-year,70.00,JPY,",
@@ -398,7 +415,8 @@ JOIN Registrar r ON b.clientId = r.registrarId
 JOIN Domain d ON b.domainRepoId = d.repoId
 JOIN Tld t ON t.tldStr = d.tld
 LEFT JOIN BillingCancellation c ON b.id = c.billingEvent
-LEFT JOIN BillingCancellation cr ON b.cancellationMatchingBillingEvent = cr.billingRecurrence
+LEFT JOIN BillingCancellation cr ON b.cancellationMatchingBillingEvent = cr.billingRecurrence AND \
+b.billingTime = cr.billingTime
 WHERE r.billingAccountMap IS NOT NULL
 AND r.type = 'REAL'
 AND t.invoicingEnabled IS TRUE
@@ -607,6 +625,51 @@ AND cr.id IS NULL
         Instant.parse("2017-10-04T00:00:00.0Z"),
         Instant.parse("2017-10-02T00:00:00.0Z"));
     persistBillingEvent(16, domain15, registrar11, Reason.RENEW, 3, Money.of(USD, 20.5));
+
+    // Add a billing event in Year 1 and a cancellation in Year 2 for the same recurrence.
+    // The Year 1 event should NOT be cancelled.
+    Domain domain17 = persistActiveDomain("recurrence-collision.test");
+    DomainHistory domainHistoryCollision = persistDomainHistory(domain17, registrar1);
+
+    BillingRecurrence billingRecurrenceCollision =
+        new BillingRecurrence()
+            .asBuilder()
+            .setRegistrarId(registrar1.getRegistrarId())
+            .setRecurrenceEndTime(END_INSTANT)
+            .setId(100)
+            .setDomainHistory(domainHistoryCollision)
+            .setTargetId(domain17.getDomainName())
+            .setEventTime(Instant.parse("2017-10-04T00:00:00.0Z"))
+            .setReason(Reason.RENEW)
+            .build();
+    persistResource(billingRecurrenceCollision);
+
+    // Year 1 Billing Event (October 2017)
+    BillingEvent billingEventYear1 =
+        persistBillingEvent(17, domain17, registrar1, Reason.RENEW, 3, Money.of(USD, 20.5));
+    billingEventYear1 =
+        billingEventYear1
+            .asBuilder()
+            .setCancellationMatchingBillingEvent(billingRecurrenceCollision)
+            .setFlags(ImmutableSet.of(Flag.SYNTHETIC))
+            .setSyntheticCreationTime(Instant.parse("2017-10-03T00:00:00.0Z"))
+            .build();
+    persistResource(billingEventYear1);
+
+    // Year 2 Billing Cancellation (October 2018)
+    BillingCancellation cancellationYear2 =
+        new BillingCancellation()
+            .asBuilder()
+            .setId(101)
+            .setRegistrarId(registrar1.getRegistrarId())
+            .setEventTime(Instant.parse("2018-10-05T00:00:00.0Z"))
+            .setBillingTime(Instant.parse("2018-10-04T00:00:00.0Z"))
+            .setBillingRecurrence(billingRecurrenceCollision.createVKey())
+            .setTargetId(domain17.getDomainName())
+            .setReason(Reason.RENEW)
+            .setDomainHistory(domainHistoryCollision)
+            .build();
+    persistResource(cancellationYear2);
   }
 
   private static DomainHistory persistDomainHistory(Domain domain, Registrar registrar) {

core/src/test/java/google/registry/gcs/GcsUtilsTest.java

@@ -15,27 +15,40 @@
 package google.registry.gcs;
 
 import static com.google.common.truth.Truth.assertThat;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
 
 import com.google.cloud.storage.BlobId;
 import com.google.cloud.storage.BlobInfo;
+import com.google.cloud.storage.Bucket;
+import com.google.cloud.storage.BucketInfo;
+import com.google.cloud.storage.Storage;
 import com.google.cloud.storage.StorageOptions;
 import com.google.cloud.storage.contrib.nio.testing.LocalStorageHelper;
 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableMap;
 import com.google.common.io.ByteStreams;
 import com.google.common.net.MediaType;
+import google.registry.testing.SystemPropertyExtension;
+import google.registry.util.RegistryEnvironment;
 import java.io.ByteArrayInputStream;
 import java.io.ByteArrayOutputStream;
 import java.io.IOException;
 import java.io.ObjectInputStream;
 import java.io.ObjectOutputStream;
 import java.io.OutputStream;
+import org.junit.jupiter.api.AfterEach;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.RegisterExtension;
 
 /** Unit tests for {@link GcsUtilsTest}. */
 class GcsUtilsTest {
 
+  @RegisterExtension
+  final SystemPropertyExtension systemPropertyExtension = new SystemPropertyExtension();
+
   private GcsUtils gcsUtils = new GcsUtils(LocalStorageHelper.getOptions());
 
   private String bucket = "my-bucket";
@@ -43,9 +56,18 @@ class GcsUtilsTest {
   private BlobId blobId = BlobId.of(bucket, filename);
   private ImmutableMap<String, String> metadata = ImmutableMap.of("key1", "val1", "Key2", "val2");
   private final byte[] bytes = new byte[] {'a', 'b', 'c'};
+  private RegistryEnvironment previousEnvironment;
 
   @BeforeEach
-  void beforeEach() {}
+  void beforeEach() {
+    previousEnvironment = RegistryEnvironment.get();
+    GcsUtils.clearPapCache();
+  }
+
+  @AfterEach
+  void afterEach() {
+    previousEnvironment.setup();
+  }
 
   @Test
   void testSerialization_testStorage() throws Exception {
@@ -111,6 +133,88 @@ class GcsUtilsTest {
     assertThat(gcsUtils.existsAndNotEmpty(blobId)).isFalse();
   }
 
+  @Test
+  void testVerifyPublicAccessPrevention_enforced() {
+    RegistryEnvironment.PRODUCTION.setup(systemPropertyExtension);
+    Storage mockStorage = mock(Storage.class);
+    StorageOptions mockOptions = mock(StorageOptions.class);
+    when(mockOptions.getService()).thenReturn(mockStorage);
+
+    Bucket mockBucket = mock(Bucket.class);
+    when(mockStorage.get("my-bucket")).thenReturn(mockBucket);
+
+    BucketInfo.IamConfiguration mockIamConfig = mock(BucketInfo.IamConfiguration.class);
+    when(mockBucket.getIamConfiguration()).thenReturn(mockIamConfig);
+    when(mockIamConfig.getPublicAccessPrevention())
+        .thenReturn(BucketInfo.PublicAccessPrevention.ENFORCED);
+
+    GcsUtils utils = new GcsUtils(mockOptions);
+    utils.verifyPublicAccessPrevention("my-bucket");
+  }
+
+  @Test
+  void testVerifyPublicAccessPrevention_notCheckedInNonProd() {
+    RegistryEnvironment.SANDBOX.setup(systemPropertyExtension);
+    Storage mockStorage = mock(Storage.class);
+    StorageOptions mockOptions = mock(StorageOptions.class);
+    when(mockOptions.getService()).thenReturn(mockStorage);
+
+    Bucket mockBucket = mock(Bucket.class);
+    when(mockStorage.get("my-bucket")).thenReturn(mockBucket);
+
+    BucketInfo.IamConfiguration mockIamConfig = mock(BucketInfo.IamConfiguration.class);
+    when(mockBucket.getIamConfiguration()).thenReturn(mockIamConfig);
+    when(mockIamConfig.getPublicAccessPrevention())
+        .thenReturn(BucketInfo.PublicAccessPrevention.INHERITED);
+
+    GcsUtils utils = new GcsUtils(mockOptions);
+    // no exception thrown even though PAP isn't enforced
+    utils.verifyPublicAccessPrevention("my-bucket");
+  }
+
+  @Test
+  void testVerifyPublicAccessPrevention_notEnforced() {
+    RegistryEnvironment.PRODUCTION.setup(systemPropertyExtension);
+    Storage mockStorage = mock(Storage.class);
+    StorageOptions mockOptions = mock(StorageOptions.class);
+    when(mockOptions.getService()).thenReturn(mockStorage);
+
+    Bucket mockBucket = mock(Bucket.class);
+    when(mockStorage.get("my-bucket")).thenReturn(mockBucket);
+
+    BucketInfo.IamConfiguration mockIamConfig = mock(BucketInfo.IamConfiguration.class);
+    when(mockBucket.getIamConfiguration()).thenReturn(mockIamConfig);
+    when(mockIamConfig.getPublicAccessPrevention())
+        .thenReturn(BucketInfo.PublicAccessPrevention.INHERITED);
+
+    GcsUtils utils = new GcsUtils(mockOptions);
+
+    IllegalStateException thrown =
+        assertThrows(
+            IllegalStateException.class, () -> utils.verifyPublicAccessPrevention("my-bucket"));
+    assertThat(thrown)
+        .hasMessageThat()
+        .contains("Public Access Prevention is not enforced on bucket my-bucket");
+  }
+
+  @Test
+  void testVerifyPublicAccessPrevention_nonexistentBucket() {
+    RegistryEnvironment.PRODUCTION.setup(systemPropertyExtension);
+    Storage mockStorage = mock(Storage.class);
+    StorageOptions mockOptions = mock(StorageOptions.class);
+    when(mockOptions.getService()).thenReturn(mockStorage);
+
+    when(mockStorage.get("nonexistent-bucket")).thenReturn(null);
+
+    GcsUtils utils = new GcsUtils(mockOptions);
+
+    IllegalStateException thrown =
+        assertThrows(
+            IllegalStateException.class,
+            () -> utils.verifyPublicAccessPrevention("nonexistent-bucket"));
+    assertThat(thrown).hasMessageThat().contains("Bucket nonexistent-bucket does not exist");
+  }
+
   private static byte[] serialize(Object object) throws IOException {
     try (ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
       ObjectOutputStream oos = new ObjectOutputStream(baos);

core/src/test/java/google/registry/model/OteStatsTest.java

@@ -16,10 +16,17 @@ package google.registry.model;
 
 import static com.google.common.truth.Truth.assertThat;
 import static google.registry.testing.DatabaseHelper.createTld;
+import static google.registry.testing.DatabaseHelper.persistActiveDomain;
+import static google.registry.testing.DatabaseHelper.persistPremiumList;
+import static google.registry.testing.DatabaseHelper.persistResource;
+import static org.joda.money.CurrencyUnit.USD;
 
 import google.registry.model.OteStats.StatType;
+import google.registry.model.domain.DomainHistory;
+import google.registry.model.reporting.HistoryEntry.Type;
 import google.registry.persistence.transaction.JpaTestExtensions;
 import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
+import java.time.Instant;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.RegisterExtension;
@@ -60,39 +67,40 @@ public final class OteStatsTest {
     OteStats stats = OteStats.getFromRegistrar("blobio");
     String expected =
         """
-            contact creates: 0
-            contact deletes: 0
-            contact transfer approves: 0
-            contact transfer cancels: 0
-            contact transfer rejects: 0
-            contact transfer requests: 0
-            contact updates: 0
-            domain autorenews: 0
-            domain creates: 5
-            domain creates ascii: 4
-            domain creates idn: 1
-            domain creates start date sunrise: 1
-            domain creates with claims notice: 1
-            domain creates with fee: 1
-            domain creates with sec dns: 1
-            domain creates without sec dns: 4
-            domain deletes: 1
-            domain renews: 0
-            domain restores: 1
-            domain transfer approves: 1
-            domain transfer cancels: 1
-            domain transfer rejects: 1
-            domain transfer requests: 1
-            domain updates: 1
-            domain updates with sec dns: 1
-            domain updates without sec dns: 0
-            host creates: 1
-            host creates external: 0
-            host creates subordinate: 1
-            host deletes: 1
-            host updates: 1
-            unclassified flows: 0
-            TOTAL: 30""";
+        contact creates: 0
+        contact deletes: 0
+        contact transfer approves: 0
+        contact transfer cancels: 0
+        contact transfer rejects: 0
+        contact transfer requests: 0
+        contact updates: 0
+        domain autorenews: 0
+        domain creates: 5
+        domain creates ascii: 4
+        domain creates idn: 1
+        domain creates start date sunrise: 1
+        domain creates with claims notice: 1
+        domain creates with fee: 1
+        domain creates with sec dns: 1
+        domain creates without sec dns: 4
+        domain deletes: 1
+        domain renews: 0
+        domain restores: 1
+        domain transfer approves: 1
+        domain transfer cancels: 1
+        domain transfer rejects: 1
+        domain transfer requests: 1
+        domain updates: 1
+        domain updates with sec dns: 1
+        domain updates without sec dns: 0
+        host creates: 1
+        host creates external: 0
+        host creates subordinate: 1
+        host deletes: 1
+        host updates: 1
+        unclassified flows: 0
+        TOTAL: 30\
+        """;
     assertThat(stats.toString()).isEqualTo(expected);
   }
 
@@ -102,39 +110,72 @@ public final class OteStatsTest {
     OteStats stats = OteStats.getFromRegistrar("blobio");
     String expected =
         """
-            contact creates: 0
-            contact deletes: 0
-            contact transfer approves: 0
-            contact transfer cancels: 0
-            contact transfer rejects: 0
-            contact transfer requests: 0
-            contact updates: 0
-            domain autorenews: 0
-            domain creates: 4
-            domain creates ascii: 4
-            domain creates idn: 0
-            domain creates start date sunrise: 1
-            domain creates with claims notice: 1
-            domain creates with fee: 1
-            domain creates with sec dns: 1
-            domain creates without sec dns: 3
-            domain deletes: 1
-            domain renews: 0
-            domain restores: 0
-            domain transfer approves: 1
-            domain transfer cancels: 1
-            domain transfer rejects: 1
-            domain transfer requests: 1
-            domain updates: 1
-            domain updates with sec dns: 1
-            domain updates without sec dns: 0
-            host creates: 1
-            host creates external: 0
-            host creates subordinate: 1
-            host deletes: 0
-            host updates: 10
-            unclassified flows: 0
-            TOTAL: 34""";
+        contact creates: 0
+        contact deletes: 0
+        contact transfer approves: 0
+        contact transfer cancels: 0
+        contact transfer rejects: 0
+        contact transfer requests: 0
+        contact updates: 0
+        domain autorenews: 0
+        domain creates: 4
+        domain creates ascii: 4
+        domain creates idn: 0
+        domain creates start date sunrise: 1
+        domain creates with claims notice: 1
+        domain creates with fee: 1
+        domain creates with sec dns: 1
+        domain creates without sec dns: 3
+        domain deletes: 1
+        domain renews: 0
+        domain restores: 0
+        domain transfer approves: 1
+        domain transfer cancels: 1
+        domain transfer rejects: 1
+        domain transfer requests: 1
+        domain updates: 1
+        domain updates with sec dns: 1
+        domain updates without sec dns: 0
+        host creates: 1
+        host creates external: 0
+        host creates subordinate: 1
+        host deletes: 0
+        host updates: 10
+        unclassified flows: 0
+        TOTAL: 34\
+        """;
     assertThat(stats.toString()).isEqualTo(expected);
   }
+
+  @Test
+  void testDomainCreateWithoutXmlBytes_doesNotSatisfyComplexRequirements() throws Exception {
+    persistPremiumList("default_sandbox_list", USD, "sandbox,USD 1000");
+    OteAccountBuilder.forRegistrarId("blobio").buildAndPersist();
+    String oteAccount1 = "blobio-1";
+    Instant now = Instant.parse("2026-06-25T10:00:00Z");
+
+    // Persist a DOMAIN_CREATE history entry without XML bytes
+    persistResource(
+        new DomainHistory.Builder()
+            .setDomain(persistActiveDomain("example.tld"))
+            .setRegistrarId(oteAccount1)
+            .setType(Type.DOMAIN_CREATE)
+            .setXmlBytes(null) // explicitly null
+            .setModificationTime(now)
+            .build());
+
+    OteStats stats = OteStats.getFromRegistrar("blobio");
+
+    // It should count towards the basic DOMAIN_CREATES
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES)).isEqualTo(1);
+
+    // It should NOT count towards any stat type that requires EPP input filtering
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_ASCII)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_IDN)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_START_DATE_SUNRISE)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_WITH_CLAIMS_NOTICE)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_WITH_FEE)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_WITH_SEC_DNS)).isEqualTo(0);
+    assertThat(stats.getCount(StatType.DOMAIN_CREATES_WITHOUT_SEC_DNS)).isEqualTo(0);
+  }
 }

core/src/test/java/google/registry/model/tld/label/PremiumListTest.java

@@ -110,28 +110,44 @@ public class PremiumListTest {
 
   @Test
   void testValidation_labelMustBeLowercase() {
-    Exception e =
-        assertThrows(
-            IllegalArgumentException.class,
-            () ->
-                new PremiumEntry.Builder()
-                    .setPrice(BigDecimal.valueOf(399))
-                    .setLabel("UPPER.tld")
-                    .build());
-    assertThat(e).hasMessageThat().contains("must be in puny-coded, lower-case form");
+    assertThat(
+            assertThrows(
+                IllegalArgumentException.class,
+                () ->
+                    new PremiumEntry.Builder()
+                        .setPrice(BigDecimal.valueOf(399))
+                        .setLabel("UPPER.tld")
+                        .build()))
+        .hasMessageThat()
+        .contains("must be in puny-coded, lower-case form");
+  }
+
+  @Test
+  void testValidation_priceMustNotBeNegative() {
+    assertThat(
+            assertThrows(
+                IllegalArgumentException.class,
+                () ->
+                    new PremiumEntry.Builder()
+                        .setPrice(BigDecimal.valueOf(-100))
+                        .setLabel("anchor")
+                        .build()))
+        .hasMessageThat()
+        .isEqualTo("Price must not be negative");
   }
 
   @Test
   void testValidation_labelMustBePunyCoded() {
-    Exception e =
-        assertThrows(
-            IllegalArgumentException.class,
-            () ->
-                new PremiumEntry.Builder()
-                    .setPrice(BigDecimal.valueOf(399))
-                    .setLabel("lower.みんな")
-                    .build());
-    assertThat(e).hasMessageThat().contains("must be in puny-coded, lower-case form");
+    assertThat(
+            assertThrows(
+                IllegalArgumentException.class,
+                () ->
+                    new PremiumEntry.Builder()
+                        .setPrice(BigDecimal.valueOf(399))
+                        .setLabel("lower.みんな")
+                        .build()))
+        .hasMessageThat()
+        .contains("must be in puny-coded, lower-case form");
   }
 
   @Test

core/src/test/java/google/registry/persistence/transaction/JpaTransactionManagerImplTest.java

@@ -404,6 +404,39 @@ class JpaTransactionManagerImplTest {
         .containsExactlyElementsIn(moreEntities);
   }
 
+  @Test
+  void loadAllOfSorted() {
+    TestEntity entityB = new TestEntity("b_entity", "gamma");
+    TestEntity entityC = new TestEntity("c_entity", "alpha");
+    TestEntity entityA = new TestEntity("a_entity", "beta");
+    persistResources(ImmutableList.of(entityB, entityC, entityA));
+
+    assertThat(tm().transact(() -> tm().loadAllOfSorted(TestEntity.class, "name")))
+        .containsExactly(entityA, entityB, entityC)
+        .inOrder();
+
+    assertThat(tm().transact(() -> tm().loadAllOfSorted(TestEntity.class, "data")))
+        .containsExactly(entityC, entityA, entityB)
+        .inOrder();
+
+    assertThat(tm().transact(() -> tm().loadAllOfSorted(TestEntity.class, "data", "name")))
+        .containsExactly(entityC, entityA, entityB)
+        .inOrder();
+  }
+
+  @Test
+  void loadAllOfSorted_invalidFieldName_throwsException() {
+    IllegalArgumentException thrown =
+        assertThrows(
+            IllegalArgumentException.class,
+            () ->
+                tm().transact(
+                        () ->
+                            tm().loadAllOfSorted(
+                                    TestEntity.class, "name; DROP TABLE TestEntity;")));
+    assertThat(thrown).hasMessageThat().contains("Invalid sort field name");
+  }
+
   @Test
   void saveAllNew_rollsBackWhenFailure() {
     moreEntities.forEach(entity -> assertThat(tm().transact(() -> tm().exists(entity))).isFalse());

core/src/test/java/google/registry/reporting/billing/GenerateInvoicesActionTest.java

@@ -15,8 +15,11 @@
 package google.registry.reporting.billing;
 
 import static com.google.common.truth.Truth.assertThat;
+import static google.registry.model.common.Cursor.CursorType.RECURRING_BILLING;
+import static google.registry.testing.DatabaseHelper.persistResource;
 import static jakarta.servlet.http.HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
 import static jakarta.servlet.http.HttpServletResponse.SC_OK;
+import static org.mockito.ArgumentMatchers.startsWith;
 import static org.mockito.Mockito.mock;
 import static org.mockito.Mockito.verify;
 import static org.mockito.Mockito.when;
@@ -25,6 +28,7 @@ import com.google.cloud.tasks.v2.HttpMethod;
 import com.google.common.net.MediaType;
 import google.registry.batch.CloudTasksUtils;
 import google.registry.beam.BeamActionTestBase;
+import google.registry.model.common.Cursor;
 import google.registry.persistence.transaction.JpaTestExtensions;
 import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
 import google.registry.reporting.ReportingModule;
@@ -33,6 +37,7 @@ import google.registry.testing.CloudTasksHelper.TaskMatcher;
 import google.registry.testing.FakeClock;
 import java.io.IOException;
 import java.time.Duration;
+import java.time.Instant;
 import java.time.YearMonth;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.RegisterExtension;
@@ -50,8 +55,13 @@ class GenerateInvoicesActionTest extends BeamActionTestBase {
   private CloudTasksUtils cloudTasksUtils = cloudTasksHelper.getTestCloudTasksUtils();
   private GenerateInvoicesAction action;
 
+  private void setCursor(Instant cursorTime) {
+    persistResource(Cursor.createGlobal(RECURRING_BILLING, cursorTime));
+  }
+
   @Test
   void testLaunchTemplateJob_withPublish() throws Exception {
+    setCursor(Instant.parse("2017-11-01T00:00:00Z"));
     action =
         new GenerateInvoicesAction(
             "test-project",
@@ -84,6 +94,7 @@ class GenerateInvoicesActionTest extends BeamActionTestBase {
 
   @Test
   void testLaunchTemplateJob_withoutPublish() throws Exception {
+    setCursor(Instant.parse("2017-11-01T00:00:00Z"));
     action =
         new GenerateInvoicesAction(
             "test-project",
@@ -107,6 +118,7 @@ class GenerateInvoicesActionTest extends BeamActionTestBase {
 
   @Test
   void testCaughtIOException() throws IOException {
+    setCursor(Instant.parse("2017-11-01T00:00:00Z"));
     when(launch.execute()).thenThrow(new IOException("Pipeline error"));
     action =
         new GenerateInvoicesAction(
@@ -128,4 +140,58 @@ class GenerateInvoicesActionTest extends BeamActionTestBase {
     verify(emailUtils).sendAlertEmail("Pipeline Launch failed due to Pipeline error");
     cloudTasksHelper.assertNoTasksEnqueued("beam-reporting");
   }
+
+  @Test
+  void testFailure_cursorLagging() {
+    setCursor(Instant.parse("2017-10-31T23:59:59.999Z"));
+    action =
+        new GenerateInvoicesAction(
+            "test-project",
+            "test-region",
+            "staging_bucket",
+            "billing_bucket",
+            "REG-INV",
+            false,
+            YearMonth.of(2017, 10),
+            emailUtils,
+            cloudTasksUtils,
+            clock,
+            response,
+            dataflow);
+    action.run();
+    assertThat(response.getStatus()).isEqualTo(SC_INTERNAL_SERVER_ERROR);
+    assertThat(response.getPayload()).contains("Pipeline launch failed");
+    assertThat(response.getPayload()).contains("BillingRecurrence expansion cursor");
+    verify(emailUtils)
+        .sendAlertEmail(
+            startsWith("Pipeline Launch failed due to BillingRecurrence expansion cursor"));
+    cloudTasksHelper.assertNoTasksEnqueued("beam-reporting");
+  }
+
+  @Test
+  void testFailure_cursorMissing() {
+    // Do not set cursor, should default to START_INSTANT (1970)
+    action =
+        new GenerateInvoicesAction(
+            "test-project",
+            "test-region",
+            "staging_bucket",
+            "billing_bucket",
+            "REG-INV",
+            false,
+            YearMonth.of(2017, 10),
+            emailUtils,
+            cloudTasksUtils,
+            clock,
+            response,
+            dataflow);
+    action.run();
+    assertThat(response.getStatus()).isEqualTo(SC_INTERNAL_SERVER_ERROR);
+    assertThat(response.getPayload()).contains("Pipeline launch failed");
+    assertThat(response.getPayload()).contains("BillingRecurrence expansion cursor");
+    verify(emailUtils)
+        .sendAlertEmail(
+            startsWith("Pipeline Launch failed due to BillingRecurrence expansion cursor"));
+    cloudTasksHelper.assertNoTasksEnqueued("beam-reporting");
+  }
 }

core/src/test/java/google/registry/tools/server/GenerateZoneFilesActionTest.java

@@ -23,6 +23,7 @@ import static google.registry.testing.DatabaseHelper.persistResource;
 import static google.registry.testing.TestDataHelper.loadFile;
 import static google.registry.util.DateTimeUtils.plusMinutes;
 import static java.nio.charset.StandardCharsets.UTF_8;
+import static org.junit.jupiter.api.Assertions.assertThrows;
 
 import com.google.cloud.storage.BlobId;
 import com.google.cloud.storage.contrib.nio.testing.LocalStorageHelper;
@@ -56,6 +57,20 @@ class GenerateZoneFilesActionTest {
 
   private final GcsUtils gcsUtils = new GcsUtils(LocalStorageHelper.getOptions());
 
+  @Test
+  void testGenerate_nonexistentTld_throwsException() {
+    GenerateZoneFilesAction action = new GenerateZoneFilesAction();
+    IllegalArgumentException thrown =
+        assertThrows(
+            IllegalArgumentException.class,
+            () ->
+                action.handleJsonRequest(
+                    ImmutableMap.<String, Object>of(
+                        "tlds", ImmutableList.of("nonexistent-tld"),
+                        "exportTime", Instant.parse("2024-03-27T00:00:00Z"))));
+    assertThat(thrown).hasMessageThat().contains("TLDs do not exist: nonexistent-tld");
+  }
+
   @Test
   void testGenerate_defaultTtls() throws Exception {
     createTlds("tld", "com");

db/src/main/resources/sql/er_diagram/brief_er_diagram.html

@@ -261,11 +261,11 @@ td.section {
     </tr>
     <tr>
      <td class="property_name">generated on</td>
-     <td class="property_value">2026-06-23 01:40:35</td>
+     <td class="property_value">2026-07-06 17:57:29</td>
     </tr>
     <tr>
      <td class="property_name">last flyway file</td>
-     <td id="lastFlywayFile" class="property_value">V223__tld_change_xap_enabled_to_transitions.sql</td>
+     <td id="lastFlywayFile" class="property_value">V224__add_registrar_expiry_access_period_enabled.sql</td>
     </tr>
    </tbody>
   </table>
@@ -273,7 +273,7 @@ td.section {
   <p>&nbsp;</p>
   <svg viewBox="0.00 0.00 4783.00 3613.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" id="erDiagram" style="overflow: hidden; width: 100%; height: 800px">
    <g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 3608.5)">
-    <title>SchemaCrawler_Diagram</title> <polygon fill="white" stroke="none" points="-4,4 -4,-3608.5 4778.75,-3608.5 4778.75,4 -4,4" /> <text xml:space="preserve" text-anchor="start" x="4535.5" y="-29.2" font-family="Helvetica,sans-Serif" font-size="14.00">generated by</text> <text xml:space="preserve" text-anchor="start" x="4618.25" y="-29.2" font-family="Helvetica,sans-Serif" font-size="14.00">SchemaCrawler 17.11.1</text> <text xml:space="preserve" text-anchor="start" x="4534.75" y="-9.45" font-family="Helvetica,sans-Serif" font-size="14.00">generated on</text> <text xml:space="preserve" text-anchor="start" x="4618.25" y="-9.45" font-family="Helvetica,sans-Serif" font-size="14.00">2026-06-23 01:40:35</text> <polygon fill="none" stroke="#888888" points="4531.75,-4 4531.75,-45.5 4766.75,-45.5 4766.75,-4 4531.75,-4" /> <!-- allocationtoken_a08ccbef -->
+    <title>SchemaCrawler_Diagram</title> <polygon fill="white" stroke="none" points="-4,4 -4,-3608.5 4778.75,-3608.5 4778.75,4 -4,4" /> <text xml:space="preserve" text-anchor="start" x="4535.5" y="-29.2" font-family="Helvetica,sans-Serif" font-size="14.00">generated by</text> <text xml:space="preserve" text-anchor="start" x="4618.25" y="-29.2" font-family="Helvetica,sans-Serif" font-size="14.00">SchemaCrawler 17.11.1</text> <text xml:space="preserve" text-anchor="start" x="4534.75" y="-9.45" font-family="Helvetica,sans-Serif" font-size="14.00">generated on</text> <text xml:space="preserve" text-anchor="start" x="4618.25" y="-9.45" font-family="Helvetica,sans-Serif" font-size="14.00">2026-07-06 17:57:29</text> <polygon fill="none" stroke="#888888" points="4531.75,-4 4531.75,-45.5 4766.75,-45.5 4766.75,-4 4531.75,-4" /> <!-- allocationtoken_a08ccbef -->
     <g id="node1" class="node">
      <title>allocationtoken_a08ccbef</title> <polygon fill="#e9c2f2" stroke="none" points="479.25,-1017.62 479.25,-1037.38 664.25,-1037.38 664.25,-1017.62 479.25,-1017.62" /> <text xml:space="preserve" text-anchor="start" x="481.25" y="-1023.08" font-family="Helvetica,sans-Serif" font-weight="bold" font-style="italic" font-size="14.00">public."AllocationToken"</text> <polygon fill="#e9c2f2" stroke="none" points="664.25,-1017.62 664.25,-1037.38 737.25,-1037.38 737.25,-1017.62 664.25,-1017.62" /> <text xml:space="preserve" text-anchor="start" x="698.5" y="-1022.08" font-family="Helvetica,sans-Serif" font-size="14.00">[table]</text> <text xml:space="preserve" text-anchor="start" x="481.25" y="-1003.33" font-family="Helvetica,sans-Serif" font-weight="bold" font-style="italic" font-size="14.00">token</text> <text xml:space="preserve" text-anchor="start" x="658.5" y="-1002.33" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text xml:space="preserve" text-anchor="start" x="666.25" y="-1002.33" font-family="Helvetica,sans-Serif" font-size="14.00">text not null</text> <text xml:space="preserve" text-anchor="start" x="481.25" y="-982.58" font-family="Helvetica,sans-Serif" font-size="14.00">domain_name</text> <text xml:space="preserve" text-anchor="start" x="658.5" y="-982.58" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text xml:space="preserve" text-anchor="start" x="666.25" y="-982.58" font-family="Helvetica,sans-Serif" font-size="14.00">text</text> <text xml:space="preserve" text-anchor="start" x="481.25" y="-962.83" font-family="Helvetica,sans-Serif" font-size="14.00">redemption_domain_repo_id</text> <text xml:space="preserve" text-anchor="start" x="658.5" y="-962.83" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text xml:space="preserve" text-anchor="start" x="666.25" y="-962.83" font-family="Helvetica,sans-Serif" font-size="14.00">text</text> <text xml:space="preserve" text-anchor="start" x="481.25" y="-943.08" font-family="Helvetica,sans-Serif" font-size="14.00">token_type</text> <text xml:space="preserve" text-anchor="start" x="658.5" y="-943.08" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text xml:space="preserve" text-anchor="start" x="666.25" y="-943.08" font-family="Helvetica,sans-Serif" font-size="14.00">text</text> <polygon fill="none" stroke="#888888" points="478.25,-937.62 478.25,-1038.38 738.25,-1038.38 738.25,-937.62 478.25,-937.62" />
     </g>

db/src/main/resources/sql/flyway.txt

@@ -221,3 +221,4 @@ V220__domain_package_token_idx.sql
 V221__remove_contact_history.sql
 V222__remove_contact.sql
 V223__tld_change_xap_enabled_to_transitions.sql
+V224__add_registrar_expiry_access_period_enabled.sql

db/src/main/resources/sql/flyway/V224__add_registrar_expiry_access_period_enabled.sql

@@ -0,0 +1,20 @@
+-- Copyright 2026 The Nomulus Authors. All Rights Reserved.
+--
+-- Licensed under the Apache License, Version 2.0 (the "License");
+-- you may not use this file except in compliance with the License.
+-- You may obtain a copy of the License at
+--
+--     http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+
+-- Add the XAP opt-in column to Registrar, defaulting to false for all existing registrars.
+-- To ensure backward compatibility with running servers (old Java code) during
+-- the transition phase of the deployment, we set DEFAULT false NOT NULL.
+-- TODO(mcilwain): Drop this DEFAULT constraint in a subsequent schema release once the Java code has been fully deployed.
+ALTER TABLE "Registrar" ADD COLUMN expiry_access_period_enabled boolean
+    DEFAULT false NOT NULL;

db/src/main/resources/sql/schema/nomulus.golden.sql

@@ -855,7 +855,8 @@ CREATE TABLE public."Registrar" (
     whois_server text,
     last_expiring_cert_notification_sent_date timestamp with time zone,
     last_expiring_failover_cert_notification_sent_date timestamp with time zone,
-    last_poc_verification_date timestamp with time zone
+    last_poc_verification_date timestamp with time zone,
+    expiry_access_period_enabled boolean DEFAULT false NOT NULL
 );
 
 

prober/.gitignore

@@ -1,2 +1,2 @@
 out/
-src/main/resources/google/registry/monitoring/blackbox/modules/secrets/
+src/main/resources/google/registry/monitoring/blackbox/module/secrets/