This commit reverts changes from PR #3068 that swapped 'npm run build' for 'npx ng build' while attempting to dynamically set the '--output-path' via the CLI.
Passing '--output-path' on the command line overrides the entire 'outputPath' configuration object in angular.json. Because the new Angular 18 Application Builder (esbuild) nests outputs inside a 'browser/' directory by default, overriding the configuration bypassed the 'browser: ""' flattening property, causing all client assets to be nested deeper than expected.
This resulted in empty deployments because downstream tasks (like Jetty's copyConsole and the deployment tar scripts) expected the assets to be completely flat. By removing the '--output-path' override from the 'npx ng build' calls, the Angular CLI once again respects angular.json, flattens the output into 'staged/dist/', and the restored 'doLast' block successfully copies the artifacts where they belong.
This commit reverts changes from 5599a0eb3d and most of 5286b1a0dc (PR #3068) that stripped essential dependencies (buildConsoleForAll, buildNomulusImage, buildToolImage, fragileTest) from the default './gradlew build' target, which broke downstream deployment pipelines. It restores the default build to correctly generate all necessary production artifacts and Docker images.
It introduces a new 'fastBuild' target designed explicitly for local developers and CI checks. This lightweight target disables the execution of heavy Docker image builds, Angular compilations, and fragile tests to provide rapid feedback. Sequential execution constraints for parallel Angular builds are maintained to prevent cache corruption.
It updates the ':core:generateSqlSchema' task to execute using the 'unittest' environment instead of 'alpha'. The 'alpha' configuration is a private, internal environment config that is not distributed in the open-source repository, which caused the task to fail for public contributors. By switching to 'unittest', the generator can successfully run using the public test configuration. With this fixed, it also includes the newly generated 'db-schema.sql.generated' file, which now correctly tracks the 'FORBID_INSECURE_ALGORITHMS_RFC_9904' feature flag that was recently added.
Finally, it implements a split-runner execution strategy for the 'sqlIntegrationTest' task to permanently resolve 'failed to discover tests' and 'NoSuchMethodError' exceptions on Kokoro. Because Kokoro tests cross-version compatibility against both legacy deployed artifacts (compiled with JUnit 4 @RunWith wrappers) and modern artifacts (compiled with JUnit 5 @Suite annotations), we cannot statically configure a single test runner. We now dynamically run both the legacy 'useJUnit()' and modern 'useJUnitPlatform()' runners sequentially with 'failOnNoDiscoveredTests' disabled, allowing the appropriate engine to discover and execute the suite without causing classpath collisions.
This commit adds the buildAll task to restore the existence of a target that builds everything, which was unintentionally removed when the default build was stripped down in PR #3068. It also introduces necessary sequential constraints to the console-webapp build tasks to prevent parallel execution from corrupting the Angular CLI cache. Finally, it addresses paths for the newer Angular esbuild output and hardens the style injection in ConsoleScreenshotTest to prevent fragile test failures.
We can't change digest types that are already in the database but that's
fine (since we just store them as integers). But we forbid them as part
of domain creates/updates.
This commit introduces several security hardening improvements across the codebase:
1. XML Processing: Hardened `TransformerFactory` and `SchemaFactory` instantiations in `EppMessage.java` by explicitly enabling `XMLConstants.FEATURE_SECURE_PROCESSING` and disabling external schema access.
2. Randomness: Replaced instances of `java.util.Random` with `java.security.SecureRandom` in `SelfSignedCaCertificate.java` for stronger entropy. (Added documentation in `ProxyModule.java` explaining why `java.util.Random` is intentionally retained there for metrics sampling).
3. Deserialization: Hardened `SerializeUtils.java` by injecting an `ObjectInputFilter` into the `ObjectInputStream`, restricting deserialization strictly to expected `google.registry` classes and standard Java collections.
This commit relaxes the upper bounds on several dependencies that were previously hardcapped to specific versions:
- com.google.protobuf to [3.25.5,) and [3.17.3,)
- org.apache.beam to [2.72.0,)
- io.github.ss-bhatt to [1.0.0,)
- io.protostuff to [1.8.0,)
- redis.clients:jedis to [7.4.1,)
- org.junit.jupiter and org.junit.platform to [5.6.2,) and [1.6.2,)
- org.jcommander to [2.0,)
- org.jline to [3.0,)
- jakarta.servlet to [6.0,)
Upgrading to the modern versions of jline introduced a breaking change where DefaultParser().parse(line, line.length()) strips trailing spaces when using the default ParseContext.UNSPECIFIED. This caused the autocompletion to misbehave and tests to fail. This commit fixes ShellCommandTest.java by explicitly passing ParseContext.COMPLETE when parsing test strings to perfectly mimic the real-world JLine completion context.
Additionally, SqlIntegrationTestSuite was migrated to JUnit 5's @Suite annotation, fixing a NoClassDefFoundError introduced by uncapping the JUnit Platform dependencies, and the test suite was re-integrated into the standard :build lifecycle.
The following dependencies remain explicitly capped:
1. Hibernate & Jakarta Persistence (Blocked by -Werror):
These are held back because newer Jakarta Persistence versions deprecate executeUpdate(), setMaxResults(), and getResultStream() on Query.
- org.hibernate.orm:hibernate-core:7.3.4.Final
- org.hibernate.orm:hibernate-hikaricp:7.3.4.Final
- org.hibernate.orm:hibernate-ant:7.3.4.Final
- jakarta.persistence:jakarta.persistence-api:[3.2.0,4.0.0)
2. Netty (Blocked by abandoned v5):
Netty 5.0.0 was an experimental release abandoned in 2015. We explicitly cap beneath 5.0.0 so Gradle doesn't resolve dead-end alphas.
- io.netty:netty-codec-http:[4.1.59.Final, 5.0.0)!!
- io.netty:netty-codec:[4.1.59.Final, 5.0.0)!!
- io.netty:netty-common:[4.1.59.Final, 5.0.0)!!
- io.netty:netty-handler:[4.1.59.Final, 5.0.0)!!
- io.netty:netty-transport:[4.1.59.Final, 5.0.0)!!
- io.netty:netty-buffer:[4.1.59.Final, 5.0.0)!!
3. Google API Services:
Capped beneath their respective unstable beta/v1b4 versions:
- com.google.apis:google-api-services-dataflow:[v1b3-rev20240430-2.0.0, v1b4)!!
- com.google.apis:google-api-services-dns:[v1-rev20240419-2.0.0, v2beta)
The lockfiles have been fully regenerated and all test suites ran successfully against the latest available transitive versions.
This commit dramatically optimizes the local Gradle build time, shaving over 5 minutes off a full build execution:
- Instrumented the build to identify fragileTest taking > 3 minutes.
- Refactored TestServer.java to dynamically bind to ephemeral port 0, resolving race conditions.
- Updated UploadBsaUnavailableDomainsActionTest to use the thread-safe TestServer, allowing it to run in parallel.
- Removed outdated exclusions for HostInfoFlowTest and RegistryPipelineWorkerInitializerTest.
- Moved these tests to the highly parallelized standardTest suite.
- Removed the redundant sqlIntegrationTest execution from the standard test phase.
- Stripped heavy Docker (buildNomulusImage) and 5x frontend (buildConsoleForAll) staging dependencies from the standard build task, ensuring they are only run when explicitly deployed.
- pr-polisher: Relaxed the package-lock.json strictness. If package.json or dependencies.gradle are modified, changes to package-lock.json now correctly trigger a WARNING rather than a fatal ERROR, streamlining intentional dependency updates.
- java-ast-refactoring: Replaced the reliance on a local google-java-format binary with the project's native ./gradlew javaIncrementalFormatApply task for post-AST format fixes.
- Updated GEMINI.md and skill instructions to explicitly authorize and mandate the agent to proactively propose systemic infrastructure fixes to the user when it encounters recurring friction, false positives, or brittle workarounds.
- Overhauled the PR polisher "When to Use" instructions in GEMINI.md and SKILL.md into a critical mandate explicitly tying the execution of the polisher to the action of making or amending a commit to prevent agent forgetfulness.
- Relaxed Google Cloud and gRPC upper bounds to dynamic versions in dependencies.gradle.
- Relaxed JUnit testing framework bounds to < 6.0.0 and < 2.0.0.
- Aligned beam-runners-core-construction-java to 2.72.0!!
- Regenerated all gradle lockfiles across the project.
- Updated GEMINI.md and MEMORY.md with the optimized lockfile/dependency update workflow.
- Fixed presubmit errors in check_diff.py (added Apache 2.0 license header and fixed syntax).
This commit:
- Centralizes all creation validation checks into Fee.create().
- Inlines the redundant private createWithCustomDescription() method.
- Removes the awkward ternary operator for null type evaluation.
- Adds FeeTest.java to completely cover the Fee instance creation logic.
Enhances the `pr-polisher` skill to enforce stricter PR pre-flight checks:
- Added checks for extraneous files (package-lock.json).
- Added checks for missing license headers on new files across multiple languages.
- Added regex checks for codebase anti-patterns (FQNs, package visibility, UTC ZoneId, un-injected clocks, redundant transactions, etc.).
- Added regex checks for test anti-patterns (generic Exception catching, Truth Optional assertions, Thread.sleep).
- Enforced commit message body presence.
- Added a workflow step to explicitly verify commit message accuracy against the diff.
- Updated GEMINI.md to mandate the usage of the pr-polisher skill.
This commit updates several dependencies in dependencies.gradle and resolves the resulting test failures:
- Netty: Updated strict upper bound from <4.2.0 to <5.0.0 (resolved to 4.2.14.Final).
- OkHttp: Removed strict upper bound of <5.0.0 and moved to dynamic versions section (resolved to 5.3.2).
- Cleaned up dependencies.gradle by moving strict dependencies (secretmanager, common-protos, gax) out of the dynamic versions section.
- Updated all gradle.lockfile files.
Resulting test fixes:
- Netty: Replaced NioEventLoopGroup with DefaultEventLoopGroup in NettyExtension and ProbingStepTest. In Netty 4.2+, LocalServerChannel no longer supports registration with NIO-backed event loops, throwing an IllegalArgumentException.
- OkHttp: Refactored ServiceMonitoringClient to safely handle empty responses using Optional.orElse(). In OkHttp 5.x, a 204 No Content response returns a non-null ResponseBody with an empty string, causing an assertion failure where MosApiException was incorrectly bypassed.
* Address technical debt and improve safety in domain flows and models
- Addressed unhandled empty lists and swallowed exceptions in DomainFlowTmchUtils.
- Improved null safety and immutability guarantees in Fee and LaunchPhase.
- Applied defensive copying in FeeTransformResponseExtension.
Note: This uses the forceEmptyToNull(nullToEmptyImmutableCopy(...))
pattern. This defensive copy ensures immutability, while forceEmptyToNull
is required because JAXB will serialize an empty collection as an empty
XML tag (which violates EPP XML schemas). Setting it to null ensures
JAXB omits the tag entirely.
- Corrected JAXB property suppression in FeeCheckResponseExtensionItemStdV1.
* Add pr-polisher skill for automated PR pre-flight checks
* Enhance pr-polisher with more GEMINI.md constraints
Added checks for:
- Incorrect @Nullable imports.
- Unstatically imported utility methods (DateTimeUtils/CacheUtils).
- Redundant transaction wrapping (tm().transact -> tm().reTransact).
- Mutable collection instantiations (ArrayList/HashMap).
This adds a Gemini CLI skill that leverages OpenRewrite to perform Abstract Syntax Tree (AST) based refactoring on Java codebases. It is highly preferred over text-based regex or python scripts because it understands Java semantics, correctly updates imports, and preserves formatting. A custom Python script is also included as a fallback for renaming fields and local variables.
* add step to create cd release in cb-nomulus job
* add variable for pipeline and region
* pass image with digest to release
* add source and skaffold to release command
* remove new lines from script
- Replace deprecated Soy templates for EPP XML with JAXB models and a refined Fluent DSL.
- Migrate Spec11 and administrative emails to FreeMarker with HTML auto-escaping.
- Remove Soy compiler, Gradle tasks, and library dependencies.
- Address PR feedback regarding shadowing, version locking, and security warnings.
- Enhance tests with comprehensive XML equality assertions using Java 15 text blocks.
- Improve Javadocs and maintain strict temporal consistency using java.time.
FreeMarker replaces Soy for email templating, providing native HTML auto-escaping and allowing the removal of the complex 'soyToJava' compilation step from the build process. This significantly simplifies the build system and reduces maintenance overhead. For EPP XML, migrating to JAXB allows tool-generated commands to use the same model classes as the server-side EPP flows. This ensures that tool-generated XML is always schema-compliant and eliminates the risk of divergence between tool templates and actual server-side implementation. This unified approach provides compile-time type safety and improves developer ergonomics via a refined fluent DSL.
The base ImmutableObject class now provides a public clone() override that correctly resets the cached hashCode to null. This centralizes the custom cloning logic previously handled by a static helper and ensures that all subclasses—including the newly added JAXB models—satisfy CodeQL security requirements without needing redundant per-class overrides. The legacy static clone(T) helper has been updated to delegate to this instance method to maintain compatibility and architectural consistency.
Creation of Gson objects is nontrivial and it's thread-safe so we might
as well just use some singleton objects as much as possible rather than
recreating them.
* generate kubernetes partial/canary manifests in release job
* rename partial phase labels
* replace container name value to be stage dependent
* just keep the new partial deployment update in the manifest generation
The replyTo header works in manual tests. Add code emulating
BillingEmailUtils behavior and see if that causes the problem.
Also experimenting not to set the From header in GmaiClient, since
whatever we set is overridden anyway.
After the public-access removal from GCS buckets, the Kokoro tests can
no longer use our private repo for resolve dependencies. And breakage is
discovered only during build.
This PR lets Github to create review comment, which triggers on PRs that
contain *.lockfile changes and asks the PR author to confirm that the
update_dependency script has been executed.
This requires moving phone numbers from the contact XSD file to the
eppcommon XSD file (they're still used by registrars).
The remaining changes are related to removing the XML infrastructure
that allows for contacts and any uses of contacts.
We shouldn't merge this until
https://github.com/google/nomulus/pull/2954 is deployed to production
and has had a little bit to bake and make sure that nothing is wrong.
Change the CannedScriptExecutionAction to send a email
message as a user-specified G workspace user.
This change is part of b/510340944, to verify that a newly
added dedicated sender is properly set up for sending emails.
Once the new sender is tested, the changes in this PR can be
dropped.
This completes the exhaustive refactoring of foundational temporal types from Joda-Time to the native java.time API across the entire codebase.
- Replaced org.joda.time.DateTime, Instant, LocalDate, and Duration with java.time equivalents.
- Audited and updated Clock implementations (FakeClock, SystemClock). Added nowMillis(), nowDate(), and nowDateTime() to eliminate repetitive conversions and maintain parallel naming.
- Replaced ZonedDateTime with OffsetDateTime globally per go/avoid-zdt. OffsetDateTime is a better fit as we use a hardcoded ZoneOffset.UTC throughout the system, making geographical time zone rules (like daylight saving time) irrelevant and preventing serialization ambiguities. Added a presubmit check.
- Completely removed all transitional bridge methods from DateTimeUtils and deleted obsolete converters (e.g., DateTimeConverter).
- Updated testing infrastructure, Apache Beam pipelines, custom JCommander parameters, and networking modules to solely rely on java.time primitives.
- Retained the lone necessary org.joda.time.Instant usage in SafeBrowsingTransforms required by the Apache Beam API.
- Cleared Gradle lockfiles and removed the joda-time dependency entirely from the build configuration.
We mostly had the xsd code for this already, we just never actually used
it and added it to the output. Note that I had to make some changes to
allow the expected format (where some of the elements are empty).
The fix for https://hibernate.atlassian.net/browse/HHH-20276 was
included in 7.3.4 and will be included in version 8 in the future. 8.x
is still in alpha though so we don't want to use it yet.
Note that this primarily affects domain lookups. We choose to use the
remote cache for hosts based on repo ID (not host name), so the remote
caches are not particularly useful for host lookups. We chose this
because the number of domain queries is orders of magnitude higher than
the number of host queries.
This only applies to the CacheModule-provided caches because we don't
want to have to deal with all the various other caches. We'll want to
know the various ratios between types of cache hits/misses when
evaluating the usefulness of the remote caching.
* Migrates core classes (Clock, Sleeper, TransactionManager) and extensive domain models from Joda-Time to java.time.
* Restores original public API method names while substituting parameters/return values with `java.time.Instant`.
* Updates JAXB XJC `bindings.xjb` to natively generate `java.time.Instant` and `java.time.LocalDate`, eliminating `toDateTime` wrapper methods.
* Fixes XML serializers (`DateAdapter`) to robustly convert OffsetDateTime timezone strings to UTC.
* Cleans up redundant imports and Checkstyle failures across the codebase.
Remaining Joda-Time surface area to migrate in future tasks:
* Command-line parameters (e.g. `DateTimeParameter`, `DateParameter`, `IntervalParameter`) in `google.registry.tools.params`.
* EPP/RDAP flow testing infrastructure (`EppTestCase`, `RdapActionBaseTestCase`, `FlowTestCase`).
* Beam pipelines and Load Testing modules (`Spec11PipelineTest`, `RdePipelineTest`, `RegistryJpaReadTest`, `EppClient`).
* Utility bridges and converters (`DateTimeUtils.toDateTime/toInstant`, `DateTimeConverter`, `UtcDateTimeAdapter`).
* Remaining UI Console tests and Actions.
Created a smoke test to cover unit test gaps wrt BEAM:
- The Java and SDK compatibility in the pipeline container image
- The JPA setup in the pipelines
Both issues above can only be tested in a real pipeline.
This PR defines a new pipeline that performs a lightweight SQL
query and minimal processing. The build process can launch it
in a test environment to verify that the pipelines in the build
can run. The run script is also provided.
Found this out while testing metrics. In hindsight, not the best idea to
handle prefixing outside of the client itself. Instead, we'll enforce
the prefixing closer to Valkey, all in one place.
This commit migrates the BSA, DNS, batch, and reporting packages from Joda-Time
to java.time. Key changes include:
- Updated Sleeper, Clock, and BigqueryUtils to use java.time types natively.
- Refactored models like RdeRevision and Tld to eliminate redundant Joda
conversions, utilizing new DateTimeUtils static utilities for LocalDate.
- Improved test safety by replacing dynamic Instant.now() calls with static
parsed constants.
- Migrated temporal arithmetic in test suites to use DateTimeUtils convenience
methods (plusDays, minusDays).
- Updated BigqueryUtils serialization to preserve millisecond precision and
formatting for large years, ensuring consistency with previous Joda behavior.
- Enhanced code readability by converting long concatenated strings to Java
text blocks in LordnLogTest.
- Resolved environmental test failures in SyncRegistrarsSheetTest by
synchronizing the FakeClock with the JPA extension.
- Updated project engineering standards (GEMINI.md) to prefer Truth's
.hasValue() for Optional assertions.
Verified with a clean full build and all relevant test suites passing.
This uses two cursors (one for hosts and one for domains) to track our
progress in "catching up", or syncing recent changes to the database,
using the update-timestamp field. When the cache is in use and fully
caught up, these
cursors should be kept relatively up-to-date to the actual time, i.e.
less than one hour behind
Migrates the massive DomainBase, DomainHistory, and Registrar models to use java.time.Instant natively, removing all deprecated DateTime accessors and fixing their associated JPA converters.
Migrates the Tld entity and its entire JSON/YAML testing ecosystem, including implementing an InstantKeySerializer to preserve millisecond precision in Jackson.
Migrates the entire Epp Flow ecosystem (Create, Update, Delete, Renew, Transfer for Domains/Hosts/Contacts) to use native Instant parameters and operations.
Migrates EppTestCase and EppLifecycleDomainTest to use Instant natively instead of constantly double-wrapping timestamps.
Rebases against upstream and standardizes all Duration and DateTimeUtils static imports across the test suites.
gbrodman
Juan Celhay
Ben McIlwain
Pavlo Tkach
Weimin Yu