Reject loopback, link-local, site-local, wildcard, and multicast IP
addresses during host creation and update flows.
Glue records (A/AAAA records published in the parent zone for subordinate
name servers) must point to globally routable, public IP addresses to
ensure that recursive DNS resolvers on the public internet can reach the
authoritative name servers.
Using non-public or non-routable IP addresses in glue records is invalid
for the following reasons:
- Loopback (127.0.0.1, ::1) and Any-Local (0.0.0.0, ::) addresses point
back to the client or are unspecified, causing resolvers to query
themselves and fail.
- Private/Site-Local (e.g., 10.0.0.0/8, 192.168.0.0/16) and Link-Local
(169.254.0.0/16) addresses are not routable on the public internet,
rendering the delegated domain completely unreachable to external clients.
- Multicast addresses are designed for one-to-many delivery and cannot
be used for standard unicast DNS queries to a specific name server.
Rename LoopbackIpNotValidForHostException to IpAddressNotRoutableException
to reflect the broader set of forbidden non-routable IP addresses.
This commit addresses the following security vulnerabilities identified in the recent audit of the Console App and Backend APIs:
1. Angular XSS: Removed unsafe [innerHTML] bindings across all console-webapp templates (Contact, Registrars, Registrar Details, Users List) in favor of standard Angular interpolation.
2. Broken Access Control (IDOR): PasswordResetRequestAction and PasswordResetVerifyAction now explicitly verify that the target user's email belongs to the authorized registrarId.
3. Missing Permission Check: ConsoleEppPasswordAction now explicitly checks for CONFIGURE_EPP_CONNECTION permission before updating the EPP password.
4. Denial of Service (DoS): ConsoleBulkDomainAction now strictly limits the size of bulk domain lists (configurable, default 500) to prevent thread exhaustion.
5. Denial of Service (OOM): ConsoleHistoryDataAction now uses .setMaxResults() (configurable, default 500) on JPA native queries to prevent eager loading of the entire database into memory.
Makes the history query limit and bulk domain action limit configurable via RegistryConfig, allowing smaller limits to be used in tests to avoid heavy resource persistence.
Also removes an outdated Joda-Time migration reference from GEMINI.md.
This commit improves the database documentation in db/README.md by adding comprehensive guidelines for refactoring column types and managing two-PR schema deployments.
Key additions:
- Added a section on the "Expand and Contract" pattern for refactoring column types, explaining when it is safe to drop columns immediately vs. when a three-step release process is required.
- Added a section on writing safe NOT NULL migrations for timed transition properties, explaining the "Temporary Database Default" pattern to maintain backward compatibility with running servers during Two-PR deployments, and demonstrating the required explicit PostgreSQL `::hstore` casting syntax.
- Added a step-by-step "Recommended Git Workflow" section to help developers cleanly split their database and Java changes into chained PRs using Git.
TAG=agy
CONV=88271e71-e272-40e0-85f8-a075a423b7c2
This commit implements the database schema changes for the Expiry Access Period (XAP) launch configuration on TLDs. It represents the first step of a Two-PR deployment strategy, deploying the database schema changes in advance of the server logic.
Specifically, it replaces the `expiry_access_period_enabled` boolean column (originally introduced in PR #2804) with a new `expiry_access_period_transitions` hstore column.
Why we are making this change:
A basic boolean flag only allows an immediate, manual on/off toggle. To launch XAP on a TLD, registry operators would have to manually flip the flag at the exact launch time, which is operationally fragile and cannot be planned in advance. Refactoring this to an hstore-backed timed transition map (mapping Instant to ExpiryAccessPeriodMode) allows operators to schedule the XAP launch in advance via TLD YAML configurations. The registry will automatically transition the TLD to the ENABLED mode at the scheduled timestamp, aligning with how other scheduled TLD changes (like TLD states and EAP fee schedules) are managed.
Since the original boolean column was never mapped in Java (PR #2804 only added the database column), it is completely safe to drop it immediately in this migration.
To ensure backward compatibility with running servers (which are still executing the old Java code during the deployment transition), the new column is added as `NOT NULL` with a temporary `DEFAULT` constraint. This prevents constraint violations on inserts from old servers. A TODO has been left in the SQL migration to drop this default in a subsequent schema release once the Java changes have been deployed.
TAG=agy
CONV=88271e71-e272-40e0-85f8-a075a423b7c2
Refactor RdapDomainSearchActionTest to dynamically resolve all domain
and host Repository IDs (ROIDs) instead of asserting on hardcoded,
sequence-generated strings (like "2E-LOL" or "6-LOL").
When tests are executed in parallel (as is common in CI environments like
Kokoro), multiple test threads concurrently reset and allocate from the
shared database sequence 'project_wide_unique_id_seq'. This interleaves
ID allocations non-deterministically, causing any test asserting on
exact, hardcoded sequence values to flake.
To fix this, createManyDomainsAndHosts was updated to return the list of
persisted domains, allowing tests to dynamically resolve their ROIDs.
All other test cases were refactored to dynamically fetch the ROIDs of
pre-created domains and hosts (stored in fields or in hostNameToHostMap,
using punycode keys for IDN hosts) for their JSON assertions, rendering
the entire suite robust against sequence shifts.
we shouldn't have to rebuild it each time we get a request to a
different service or really ever at all -- we might get a tiny bit of
cache benefit here
Summarize all documentation updates across the repository to align with modern GKE, Cloud SQL Proxy v2, standard EPP fee v1.0, and Postgres database environments.
Key Updates:
- Prerequisites: Bump Java requirement to Java 25.
- Architecture & Scaling: Document GKE workloads, Cloud Tasks queues, and scheduled tasks. Replace App Engine references with GKE deployment restart commands (kubectl rollout restart).
- Configuration: Update Cloud SQL Proxy instructions to v2, fix keyring verification commands, and document IAP configuration.
- Escrow (RDE/BRDA): Fix manual generation and download procedures to match the Dataflow job ID folder structure, and correct deposit encryption/verification command parameters.
- Monitoring: Correct metric names and expand the documented metrics list with caching, locking, and reserved list metrics.
- Fixes: Standardize lists formatting across markdown files, fix broken webdriver links, and resolve various typos.
- Cleanup: Remove leftover cloud scheduler configurations for the deleted wipeOutContactHistoryPii task, and update ICANN reporting documentation to reflect open-sourced DNS query coordinator.
TAG=agy
CONV=88271e71-e272-40e0-85f8-a075a423b7c2
Normal comparison exits out once a difference is found so theoretically
a time variance can leak information. This isn't really a huge deal but
is probably still worth doing.
Introduced overloaded helper methods `persistFeatureFlag` in `DatabaseHelper` to simplify the creation and persistence of `FeatureFlag` entities in test setups.
1. `persistFeatureFlag(FeatureName, FeatureStatus)`: Persists a feature flag with a single status starting from the Unix Epoch (`START_INSTANT`).
2. `persistFeatureFlag(FeatureName, FeatureStatus, Instant, FeatureStatus)`: Persists a feature flag with an initial status at `START_INSTANT` and a subsequent transition at a specified time.
Refactored 11 occurrences of manual 1-transition flag creation and 5 occurrences of 2-transition flag creation across the test suite to use these new helpers, significantly reducing boilerplate and improving test readability.
TAG=agy
CONV=583b8a23-9fe5-476d-ac35-aeba7b218eb0
This is similar to PR #3069 but for the algorithms themselves rather
than the digest data. This forbids algorithms, that, according to RFC
9904, should not be used.
This commit reverts changes from PR #3068 that swapped 'npm run build' for 'npx ng build' while attempting to dynamically set the '--output-path' via the CLI.
Passing '--output-path' on the command line overrides the entire 'outputPath' configuration object in angular.json. Because the new Angular 18 Application Builder (esbuild) nests outputs inside a 'browser/' directory by default, overriding the configuration bypassed the 'browser: ""' flattening property, causing all client assets to be nested deeper than expected.
This resulted in empty deployments because downstream tasks (like Jetty's copyConsole and the deployment tar scripts) expected the assets to be completely flat. By removing the '--output-path' override from the 'npx ng build' calls, the Angular CLI once again respects angular.json, flattens the output into 'staged/dist/', and the restored 'doLast' block successfully copies the artifacts where they belong.
This commit reverts changes from 5599a0eb3d and most of 5286b1a0dc (PR #3068) that stripped essential dependencies (buildConsoleForAll, buildNomulusImage, buildToolImage, fragileTest) from the default './gradlew build' target, which broke downstream deployment pipelines. It restores the default build to correctly generate all necessary production artifacts and Docker images.
It introduces a new 'fastBuild' target designed explicitly for local developers and CI checks. This lightweight target disables the execution of heavy Docker image builds, Angular compilations, and fragile tests to provide rapid feedback. Sequential execution constraints for parallel Angular builds are maintained to prevent cache corruption.
It updates the ':core:generateSqlSchema' task to execute using the 'unittest' environment instead of 'alpha'. The 'alpha' configuration is a private, internal environment config that is not distributed in the open-source repository, which caused the task to fail for public contributors. By switching to 'unittest', the generator can successfully run using the public test configuration. With this fixed, it also includes the newly generated 'db-schema.sql.generated' file, which now correctly tracks the 'FORBID_INSECURE_ALGORITHMS_RFC_9904' feature flag that was recently added.
Finally, it implements a split-runner execution strategy for the 'sqlIntegrationTest' task to permanently resolve 'failed to discover tests' and 'NoSuchMethodError' exceptions on Kokoro. Because Kokoro tests cross-version compatibility against both legacy deployed artifacts (compiled with JUnit 4 @RunWith wrappers) and modern artifacts (compiled with JUnit 5 @Suite annotations), we cannot statically configure a single test runner. We now dynamically run both the legacy 'useJUnit()' and modern 'useJUnitPlatform()' runners sequentially with 'failOnNoDiscoveredTests' disabled, allowing the appropriate engine to discover and execute the suite without causing classpath collisions.
This commit adds the buildAll task to restore the existence of a target that builds everything, which was unintentionally removed when the default build was stripped down in PR #3068. It also introduces necessary sequential constraints to the console-webapp build tasks to prevent parallel execution from corrupting the Angular CLI cache. Finally, it addresses paths for the newer Angular esbuild output and hardens the style injection in ConsoleScreenshotTest to prevent fragile test failures.
We can't change digest types that are already in the database but that's
fine (since we just store them as integers). But we forbid them as part
of domain creates/updates.
This commit introduces several security hardening improvements across the codebase:
1. XML Processing: Hardened `TransformerFactory` and `SchemaFactory` instantiations in `EppMessage.java` by explicitly enabling `XMLConstants.FEATURE_SECURE_PROCESSING` and disabling external schema access.
2. Randomness: Replaced instances of `java.util.Random` with `java.security.SecureRandom` in `SelfSignedCaCertificate.java` for stronger entropy. (Added documentation in `ProxyModule.java` explaining why `java.util.Random` is intentionally retained there for metrics sampling).
3. Deserialization: Hardened `SerializeUtils.java` by injecting an `ObjectInputFilter` into the `ObjectInputStream`, restricting deserialization strictly to expected `google.registry` classes and standard Java collections.
This commit relaxes the upper bounds on several dependencies that were previously hardcapped to specific versions:
- com.google.protobuf to [3.25.5,) and [3.17.3,)
- org.apache.beam to [2.72.0,)
- io.github.ss-bhatt to [1.0.0,)
- io.protostuff to [1.8.0,)
- redis.clients:jedis to [7.4.1,)
- org.junit.jupiter and org.junit.platform to [5.6.2,) and [1.6.2,)
- org.jcommander to [2.0,)
- org.jline to [3.0,)
- jakarta.servlet to [6.0,)
Upgrading to the modern versions of jline introduced a breaking change where DefaultParser().parse(line, line.length()) strips trailing spaces when using the default ParseContext.UNSPECIFIED. This caused the autocompletion to misbehave and tests to fail. This commit fixes ShellCommandTest.java by explicitly passing ParseContext.COMPLETE when parsing test strings to perfectly mimic the real-world JLine completion context.
Additionally, SqlIntegrationTestSuite was migrated to JUnit 5's @Suite annotation, fixing a NoClassDefFoundError introduced by uncapping the JUnit Platform dependencies, and the test suite was re-integrated into the standard :build lifecycle.
The following dependencies remain explicitly capped:
1. Hibernate & Jakarta Persistence (Blocked by -Werror):
These are held back because newer Jakarta Persistence versions deprecate executeUpdate(), setMaxResults(), and getResultStream() on Query.
- org.hibernate.orm:hibernate-core:7.3.4.Final
- org.hibernate.orm:hibernate-hikaricp:7.3.4.Final
- org.hibernate.orm:hibernate-ant:7.3.4.Final
- jakarta.persistence:jakarta.persistence-api:[3.2.0,4.0.0)
2. Netty (Blocked by abandoned v5):
Netty 5.0.0 was an experimental release abandoned in 2015. We explicitly cap beneath 5.0.0 so Gradle doesn't resolve dead-end alphas.
- io.netty:netty-codec-http:[4.1.59.Final, 5.0.0)!!
- io.netty:netty-codec:[4.1.59.Final, 5.0.0)!!
- io.netty:netty-common:[4.1.59.Final, 5.0.0)!!
- io.netty:netty-handler:[4.1.59.Final, 5.0.0)!!
- io.netty:netty-transport:[4.1.59.Final, 5.0.0)!!
- io.netty:netty-buffer:[4.1.59.Final, 5.0.0)!!
3. Google API Services:
Capped beneath their respective unstable beta/v1b4 versions:
- com.google.apis:google-api-services-dataflow:[v1b3-rev20240430-2.0.0, v1b4)!!
- com.google.apis:google-api-services-dns:[v1-rev20240419-2.0.0, v2beta)
The lockfiles have been fully regenerated and all test suites ran successfully against the latest available transitive versions.
This commit dramatically optimizes the local Gradle build time, shaving over 5 minutes off a full build execution:
- Instrumented the build to identify fragileTest taking > 3 minutes.
- Refactored TestServer.java to dynamically bind to ephemeral port 0, resolving race conditions.
- Updated UploadBsaUnavailableDomainsActionTest to use the thread-safe TestServer, allowing it to run in parallel.
- Removed outdated exclusions for HostInfoFlowTest and RegistryPipelineWorkerInitializerTest.
- Moved these tests to the highly parallelized standardTest suite.
- Removed the redundant sqlIntegrationTest execution from the standard test phase.
- Stripped heavy Docker (buildNomulusImage) and 5x frontend (buildConsoleForAll) staging dependencies from the standard build task, ensuring they are only run when explicitly deployed.
- pr-polisher: Relaxed the package-lock.json strictness. If package.json or dependencies.gradle are modified, changes to package-lock.json now correctly trigger a WARNING rather than a fatal ERROR, streamlining intentional dependency updates.
- java-ast-refactoring: Replaced the reliance on a local google-java-format binary with the project's native ./gradlew javaIncrementalFormatApply task for post-AST format fixes.
- Updated GEMINI.md and skill instructions to explicitly authorize and mandate the agent to proactively propose systemic infrastructure fixes to the user when it encounters recurring friction, false positives, or brittle workarounds.
- Overhauled the PR polisher "When to Use" instructions in GEMINI.md and SKILL.md into a critical mandate explicitly tying the execution of the polisher to the action of making or amending a commit to prevent agent forgetfulness.
- Relaxed Google Cloud and gRPC upper bounds to dynamic versions in dependencies.gradle.
- Relaxed JUnit testing framework bounds to < 6.0.0 and < 2.0.0.
- Aligned beam-runners-core-construction-java to 2.72.0!!
- Regenerated all gradle lockfiles across the project.
- Updated GEMINI.md and MEMORY.md with the optimized lockfile/dependency update workflow.
- Fixed presubmit errors in check_diff.py (added Apache 2.0 license header and fixed syntax).
This commit:
- Centralizes all creation validation checks into Fee.create().
- Inlines the redundant private createWithCustomDescription() method.
- Removes the awkward ternary operator for null type evaluation.
- Adds FeeTest.java to completely cover the Fee instance creation logic.
Enhances the `pr-polisher` skill to enforce stricter PR pre-flight checks:
- Added checks for extraneous files (package-lock.json).
- Added checks for missing license headers on new files across multiple languages.
- Added regex checks for codebase anti-patterns (FQNs, package visibility, UTC ZoneId, un-injected clocks, redundant transactions, etc.).
- Added regex checks for test anti-patterns (generic Exception catching, Truth Optional assertions, Thread.sleep).
- Enforced commit message body presence.
- Added a workflow step to explicitly verify commit message accuracy against the diff.
- Updated GEMINI.md to mandate the usage of the pr-polisher skill.
This commit updates several dependencies in dependencies.gradle and resolves the resulting test failures:
- Netty: Updated strict upper bound from <4.2.0 to <5.0.0 (resolved to 4.2.14.Final).
- OkHttp: Removed strict upper bound of <5.0.0 and moved to dynamic versions section (resolved to 5.3.2).
- Cleaned up dependencies.gradle by moving strict dependencies (secretmanager, common-protos, gax) out of the dynamic versions section.
- Updated all gradle.lockfile files.
Resulting test fixes:
- Netty: Replaced NioEventLoopGroup with DefaultEventLoopGroup in NettyExtension and ProbingStepTest. In Netty 4.2+, LocalServerChannel no longer supports registration with NIO-backed event loops, throwing an IllegalArgumentException.
- OkHttp: Refactored ServiceMonitoringClient to safely handle empty responses using Optional.orElse(). In OkHttp 5.x, a 204 No Content response returns a non-null ResponseBody with an empty string, causing an assertion failure where MosApiException was incorrectly bypassed.
* Address technical debt and improve safety in domain flows and models
- Addressed unhandled empty lists and swallowed exceptions in DomainFlowTmchUtils.
- Improved null safety and immutability guarantees in Fee and LaunchPhase.
- Applied defensive copying in FeeTransformResponseExtension.
Note: This uses the forceEmptyToNull(nullToEmptyImmutableCopy(...))
pattern. This defensive copy ensures immutability, while forceEmptyToNull
is required because JAXB will serialize an empty collection as an empty
XML tag (which violates EPP XML schemas). Setting it to null ensures
JAXB omits the tag entirely.
- Corrected JAXB property suppression in FeeCheckResponseExtensionItemStdV1.
* Add pr-polisher skill for automated PR pre-flight checks
* Enhance pr-polisher with more GEMINI.md constraints
Added checks for:
- Incorrect @Nullable imports.
- Unstatically imported utility methods (DateTimeUtils/CacheUtils).
- Redundant transaction wrapping (tm().transact -> tm().reTransact).
- Mutable collection instantiations (ArrayList/HashMap).
This adds a Gemini CLI skill that leverages OpenRewrite to perform Abstract Syntax Tree (AST) based refactoring on Java codebases. It is highly preferred over text-based regex or python scripts because it understands Java semantics, correctly updates imports, and preserves formatting. A custom Python script is also included as a fallback for renaming fields and local variables.
* add step to create cd release in cb-nomulus job
* add variable for pipeline and region
* pass image with digest to release
* add source and skaffold to release command
* remove new lines from script
- Replace deprecated Soy templates for EPP XML with JAXB models and a refined Fluent DSL.
- Migrate Spec11 and administrative emails to FreeMarker with HTML auto-escaping.
- Remove Soy compiler, Gradle tasks, and library dependencies.
- Address PR feedback regarding shadowing, version locking, and security warnings.
- Enhance tests with comprehensive XML equality assertions using Java 15 text blocks.
- Improve Javadocs and maintain strict temporal consistency using java.time.
FreeMarker replaces Soy for email templating, providing native HTML auto-escaping and allowing the removal of the complex 'soyToJava' compilation step from the build process. This significantly simplifies the build system and reduces maintenance overhead. For EPP XML, migrating to JAXB allows tool-generated commands to use the same model classes as the server-side EPP flows. This ensures that tool-generated XML is always schema-compliant and eliminates the risk of divergence between tool templates and actual server-side implementation. This unified approach provides compile-time type safety and improves developer ergonomics via a refined fluent DSL.
The base ImmutableObject class now provides a public clone() override that correctly resets the cached hashCode to null. This centralizes the custom cloning logic previously handled by a static helper and ensures that all subclasses—including the newly added JAXB models—satisfy CodeQL security requirements without needing redundant per-class overrides. The legacy static clone(T) helper has been updated to delegate to this instance method to maintain compatibility and architectural consistency.
Creation of Gson objects is nontrivial and it's thread-safe so we might
as well just use some singleton objects as much as possible rather than
recreating them.
* generate kubernetes partial/canary manifests in release job
* rename partial phase labels
* replace container name value to be stage dependent
* just keep the new partial deployment update in the manifest generation
The replyTo header works in manual tests. Add code emulating
BillingEmailUtils behavior and see if that causes the problem.
Also experimenting not to set the From header in GmaiClient, since
whatever we set is overridden anyway.
After the public-access removal from GCS buckets, the Kokoro tests can
no longer use our private repo for resolve dependencies. And breakage is
discovered only during build.
This PR lets Github to create review comment, which triggers on PRs that
contain *.lockfile changes and asks the PR author to confirm that the
update_dependency script has been executed.
This requires moving phone numbers from the contact XSD file to the
eppcommon XSD file (they're still used by registrars).
The remaining changes are related to removing the XML infrastructure
that allows for contacts and any uses of contacts.
We shouldn't merge this until
https://github.com/google/nomulus/pull/2954 is deployed to production
and has had a little bit to bake and make sure that nothing is wrong.
gbrodman
Ben McIlwain
Juan Celhay
Pavlo Tkach
Weimin Yu