This commit relaxes the upper bounds on several dependencies that were previously hardcapped to specific versions:
- com.google.protobuf to [3.25.5,) and [3.17.3,)
- org.apache.beam to [2.72.0,)
- io.github.ss-bhatt to [1.0.0,)
- io.protostuff to [1.8.0,)
- redis.clients:jedis to [7.4.1,)
- org.junit.jupiter and org.junit.platform to [5.6.2,) and [1.6.2,)
- org.jcommander to [2.0,)
- org.jline to [3.0,)
- jakarta.servlet to [6.0,)
Upgrading to the modern versions of jline introduced a breaking change where DefaultParser().parse(line, line.length()) strips trailing spaces when using the default ParseContext.UNSPECIFIED. This caused the autocompletion to misbehave and tests to fail. This commit fixes ShellCommandTest.java by explicitly passing ParseContext.COMPLETE when parsing test strings to perfectly mimic the real-world JLine completion context.
Additionally, SqlIntegrationTestSuite was migrated to JUnit 5's @Suite annotation, fixing a NoClassDefFoundError introduced by uncapping the JUnit Platform dependencies, and the test suite was re-integrated into the standard :build lifecycle.
The following dependencies remain explicitly capped:
1. Hibernate & Jakarta Persistence (Blocked by -Werror):
These are held back because newer Jakarta Persistence versions deprecate executeUpdate(), setMaxResults(), and getResultStream() on Query.
- org.hibernate.orm:hibernate-core:7.3.4.Final
- org.hibernate.orm:hibernate-hikaricp:7.3.4.Final
- org.hibernate.orm:hibernate-ant:7.3.4.Final
- jakarta.persistence:jakarta.persistence-api:[3.2.0,4.0.0)
2. Netty (Blocked by abandoned v5):
Netty 5.0.0 was an experimental release abandoned in 2015. We explicitly cap beneath 5.0.0 so Gradle doesn't resolve dead-end alphas.
- io.netty:netty-codec-http:[4.1.59.Final, 5.0.0)!!
- io.netty:netty-codec:[4.1.59.Final, 5.0.0)!!
- io.netty:netty-common:[4.1.59.Final, 5.0.0)!!
- io.netty:netty-handler:[4.1.59.Final, 5.0.0)!!
- io.netty:netty-transport:[4.1.59.Final, 5.0.0)!!
- io.netty:netty-buffer:[4.1.59.Final, 5.0.0)!!
3. Google API Services:
Capped beneath their respective unstable beta/v1b4 versions:
- com.google.apis:google-api-services-dataflow:[v1b3-rev20240430-2.0.0, v1b4)!!
- com.google.apis:google-api-services-dns:[v1-rev20240419-2.0.0, v2beta)
The lockfiles have been fully regenerated and all test suites ran successfully against the latest available transitive versions.
- Relaxed Google Cloud and gRPC upper bounds to dynamic versions in dependencies.gradle.
- Relaxed JUnit testing framework bounds to < 6.0.0 and < 2.0.0.
- Aligned beam-runners-core-construction-java to 2.72.0!!
- Regenerated all gradle lockfiles across the project.
- Updated GEMINI.md and MEMORY.md with the optimized lockfile/dependency update workflow.
- Fixed presubmit errors in check_diff.py (added Apache 2.0 license header and fixed syntax).
This commit updates several dependencies in dependencies.gradle and resolves the resulting test failures:
- Netty: Updated strict upper bound from <4.2.0 to <5.0.0 (resolved to 4.2.14.Final).
- OkHttp: Removed strict upper bound of <5.0.0 and moved to dynamic versions section (resolved to 5.3.2).
- Cleaned up dependencies.gradle by moving strict dependencies (secretmanager, common-protos, gax) out of the dynamic versions section.
- Updated all gradle.lockfile files.
Resulting test fixes:
- Netty: Replaced NioEventLoopGroup with DefaultEventLoopGroup in NettyExtension and ProbingStepTest. In Netty 4.2+, LocalServerChannel no longer supports registration with NIO-backed event loops, throwing an IllegalArgumentException.
- OkHttp: Refactored ServiceMonitoringClient to safely handle empty responses using Optional.orElse(). In OkHttp 5.x, a 204 No Content response returns a non-null ResponseBody with an empty string, causing an assertion failure where MosApiException was incorrectly bypassed.
- Replace deprecated Soy templates for EPP XML with JAXB models and a refined Fluent DSL.
- Migrate Spec11 and administrative emails to FreeMarker with HTML auto-escaping.
- Remove Soy compiler, Gradle tasks, and library dependencies.
- Address PR feedback regarding shadowing, version locking, and security warnings.
- Enhance tests with comprehensive XML equality assertions using Java 15 text blocks.
- Improve Javadocs and maintain strict temporal consistency using java.time.
FreeMarker replaces Soy for email templating, providing native HTML auto-escaping and allowing the removal of the complex 'soyToJava' compilation step from the build process. This significantly simplifies the build system and reduces maintenance overhead. For EPP XML, migrating to JAXB allows tool-generated commands to use the same model classes as the server-side EPP flows. This ensures that tool-generated XML is always schema-compliant and eliminates the risk of divergence between tool templates and actual server-side implementation. This unified approach provides compile-time type safety and improves developer ergonomics via a refined fluent DSL.
The base ImmutableObject class now provides a public clone() override that correctly resets the cached hashCode to null. This centralizes the custom cloning logic previously handled by a static helper and ensures that all subclasses—including the newly added JAXB models—satisfy CodeQL security requirements without needing redundant per-class overrides. The legacy static clone(T) helper has been updated to delegate to this instance method to maintain compatibility and architectural consistency.
This completes the exhaustive refactoring of foundational temporal types from Joda-Time to the native java.time API across the entire codebase.
- Replaced org.joda.time.DateTime, Instant, LocalDate, and Duration with java.time equivalents.
- Audited and updated Clock implementations (FakeClock, SystemClock). Added nowMillis(), nowDate(), and nowDateTime() to eliminate repetitive conversions and maintain parallel naming.
- Replaced ZonedDateTime with OffsetDateTime globally per go/avoid-zdt. OffsetDateTime is a better fit as we use a hardcoded ZoneOffset.UTC throughout the system, making geographical time zone rules (like daylight saving time) irrelevant and preventing serialization ambiguities. Added a presubmit check.
- Completely removed all transitional bridge methods from DateTimeUtils and deleted obsolete converters (e.g., DateTimeConverter).
- Updated testing infrastructure, Apache Beam pipelines, custom JCommander parameters, and networking modules to solely rely on java.time primitives.
- Retained the lone necessary org.joda.time.Instant usage in SafeBrowsingTransforms required by the Apache Beam API.
- Cleared Gradle lockfiles and removed the joda-time dependency entirely from the build configuration.
The fix for https://hibernate.atlassian.net/browse/HHH-20276 was
included in 7.3.4 and will be included in version 8 in the future. 8.x
is still in alpha though so we don't want to use it yet.
We add optional Valkey caching of hosts and domains for future use. Eventually, this will allow us to pre-warm large amounts of data in Valkey for quick retrieval during actions like RDAP.
Note: this doesn't actually use the caches yet.
We use Jedis instead of Redisson for speed purposes
(https://www.instaclustr.com/blog/redis-java-clients-and-client-side-caching/)
which means that we have to implement our own multilayer cache but
that's not the worst thing in the world.
Tested on crash with logging and RDAP code that's not included in this
PR -- it behaves as you'd expect, where the local cache works for
immediate re-lookups and the remote cache works after a restart.
We want to make sure that we use the same XML factories no matter what,
so we use "newDefaultFactory" instead of "newFactory" (to avoid picking
up some random thing on the classpath).
This also fixes an exception that occurs if you haven't synced the
internal repo with the public repo.
I think this may have been introduced as part of Gradle 9? Not sure why
it's not showing up in the remote builds but without this, I can't run
any "devTool" commands.
Note: the fixes were suggested by gemini-cli
Error-prone introduced many more checks in Java 25. We fixed a few
and suppressed most. A follow-up bug is opened to clean this up.
An ai agent should be able to clean up most of it.
This PR is created with gemini-cli. Summary of experience:
* The good: AI caught most compatibility issues, and with permission,
suppressed them through compiler flags and errorprone options.
It also caught many versio references in scripts.
* Where it didn't shine:
- It did not find and update the target version spec in the custome
VKey annotation processor source file.
- It did not flag eclipse-temurin:21 docker image for upgrade.
- When running into failure, its first instinct is to disable checks
e.g., -Werror instead of fixing them.
* More Gradle 9 preparations
Fix additional compatibility warnings after upgrading to Gradle 8.14.3
from 8.13.
* More Gradle 9 compatibility fix
More fixes after upgrading Gradle from 8.13 to 8.14.3.
Upgraded the gradle-license-report plugin, and handled config leaking
issues.
Researched using gemini web and manually applied the fixes. Gemini-cli
could not find the right solution.
This ended up being wayyyy more complicated than expected due to
issues with Hibernate, various dependencies having conflicts with the
proto dependency version, and other breaking changes.
Notes:
- Hibernate 7 switches up the user type / converter system and for us,
this means we must be / want to be more explicit with how we convert
and store things. For example, we need to add Postgres types to @Column
definitions.
- Hibernate 7.3 has an issue with generic MappedSuperclasses -- we have
issues with BaseDomainLabelList. I'll investigate that, but for now
let's stick with 7.2.x
- H7 is more strict with annotations and prevents us from storing mapped
superclasses embedded within other objects. This kinda makes sense but
makes the History objects a bit more difficult. We had to add "concrete"
embeddable DomainBase and HostBase objects that we can store/retrieve
from the DB.
- We convert some of the calls to "Query" to "TypedQuery" -- in
Hibernate 8 / JPA 4.0 these will be super-deprecated and we'll need to
shift everything over, so this is necessary.
- You aren't supposed to put callback listeners on embedded entities
(because it can be not obvious what's happening). We don't like that,
so we add our own annotations that are processed recursively for
embedded entities, so we get things like the update / create
timestamps.
- Hibernate doesn't allow for multiple converters to be auto-applied to
the same "type" and it counts all VKey converters as one type.
Unfortunately, this means we have to explicitly mark each one.
- A bunch of other dependency changes were required to keep from having
the proto 3/4 conflict
I obtained access to an IBM s390x VM so I thought I'd see how multi-arch
Nomulus is.
Our main application is in Java so it is already multi-arch, but several
tests use docker images that are by default x64. Luckily postgres has an
s390x port, but selenium does not. So I had to disable Screenshot tests
when the arch is not amd64.
Several jars in our dependencies are now multi-release, including
dnsjava and snakeyaml, and a few more. Such jars include
jvm-version-specific classes that will only be loaded by the vm that can
handle them. All it takes is a new manifest attribute.
This change allows us to upgrade to dnsjava3.6+: the base (java 8) version of
this jar breaks java21. The correct manifest allows java21 to find the
classes it needs.
This is the last remaining GAE API that we depend on. By removing it, we are able to remove all common GAE dependencies as well.
To merge this PR, we need to create console User objects that have the same email address as the RegistrarPoc objects' login_email_address and copy over the existing registry lock hashes and salts.
We are also able to simply the code base by removing some redundant logic like AuthMethod (API is now the only supported one) and UserAuthInfo (console user is now the only supported one)
There are several behavioral changes that are worth noting:
The XsrfTokenManager now uses the console user's email address to mint and verify the token. Previously, only email addresses returned by the GAE Users service are used, whereas a blank email address will be used if the user is logged in as a console user. I believe this was an oversight that is now corrected.
The legacy console will return 401 when no user is logged in, instead of redirecting to the Users service login flow.
The logout URL in the legacy console is changed to use the IAP logout flow. It will clear the cookie and redirect the users to IAP login page (tested on QA).
The screenshot changes are mostly due to the console users lacking a display name and therefore showing the email address instead. Some changes are due to using the console user's email address as the registry lock email address, which is being fixed in Add DB column for separate rlock email address #2413 and its follow-up RPs.
* Add log traces to Nomulus service on GKE
Add request-scope log traces to Nomulus on GKE which, unlike
AppEngine and Cloud Run etc, does not generate traces for hosted
applications. This change only affects the GKE image. It does not affect
the AppEngine services.
Log traces are added to Nomulus-generated logs in request-processing
threads. Forked threads are not covered yet. The single relevant use
case (TimeLimiter) will be addressed in a followup PR.
The main change is in the logging configuration:
* Use gcp-cloud-logging's LoggingHandler
* Add gcp-cloud-logging's TraceLoggingEnhancer to the handler.
* Set a thread-local trace id through the TraceLoggingEnhancer in
ServletBase on request's entry and clear it on completion.
Also removed an unused class (`RequestLogId`).
* CR
* CR
Console users need IAP to inject the necessary OIDC tokens into their
request headers and therefore need to be bound to appropriate roles. Note
that in environments managed by latchkey, the bindings will need to be
present in latchkey config files as well, otherwise the changes made by
the nomulus tool will be reverted.
TESTED=ran the nomulus command against alpha and verified that the
bindings are created/removed upon console user creation/deletion.
gbrodman
Ben McIlwain
Weimin Yu
Lai Jiang