Postgresql-17.6 introduces two new lines in pg_dump output as a
security feature: `\restricted {HASH}` and `\unrestricted {HASH}`.
We filter out lines starting with these two prefixes when comparing
schemas.
The db upgrade also adds two empty lines to the pg_dump output. We
know ignore all empty lines when comparing schemas.
We now pin to postgreSQL v17 when running tests, which means that minor
version might increase without our intervention. This causes (at least)
the comment in the golden schema to change, and failing the test as a
result.
This PR adds the ability to strip lines that we deem as comment from the
comparison, so we don't have to do trivial upgrades to the gold schema
whenever there's minor version upgrade.
I obtained access to an IBM s390x VM so I thought I'd see how multi-arch
Nomulus is.
Our main application is in Java so it is already multi-arch, but several
tests use docker images that are by default x64. Luckily postgres has an
s390x port, but selenium does not. So I had to disable Screenshot tests
when the arch is not amd64.
* Add index for domainRepoId to PollMessage and DomainHistoryHost
* Add flyway fix for Concurrent
* fix gradle.properties
* Modify lockfiles
* Update the release tool and add IF NOT EXISTS
* Test removing transactional lock from deploy script
* Add transactional lock flag to actual flyway commands in script
* Remove flag from info command
* Add configuration for integration test
Make the necessary changes for the code base to compile with JDK 21.
Other changes:
1. Upgraded testcontainer version and the SQL image version (to be the
same as what we use in Cloud SQL). This led to some schema changes and
also changed the order of results in some test queries (for the
better I think, as the new order appears to be alphabetical).
2. Remove dependency on Truth8, which is deprecated.
3. Enable parallel Gradle task execution and greatly increased the
number of parallel tests in standardTest. Removed outcastTest.
This PR makes it possible to build the Nomulus code base using Java 17.
Building with Java 11 continue to be possible and the resulting bytecodes are
still at Java 8 level. Also upgraded Gradle to 8.5.
There are several necessary changes to make this happen:
1. Some Gradle plugins need to be upgraded to support Java 17, notably
errorprone. As a result, a lot more "errors" were caught and corrected.
2. All test code are now built and run at Java 8 level. Previously it was left
undefined (which defaults to the version of the compiler) and had led to
situations where we inadvertently called Java 8+ features in production that
are not caught by tests. The change also made the java8compatibility subproject
obsolete, which is therefore removed.
3. Removed the docs subproject. Its main use is to generate flows.md, but it
relies heavily on Java internal APIs that have changed significant with each
version. Upgrading to Java 11 required extensive refactoring of the code there,
and Java 17 again removed many APIs that were used. I don't think it is worth
the maintenance effort just to have a tool to generate flows.md which no one
actually reads.
4. Capped a few GCP dependencies because the latest version depends on
grpc-java >= 1.59.0, which includes a runtime incompatibility
(https://github.com/grpc/grpc-java/releases/tag/v1.59.0).
Use a system property to specify whether this check should be executed.
We will update the presubmit test script to run this check only during
foss-pr.
* Use Gradle dependency dynamic versioning
Use dynamic versioning for Gradle dependencies when possible.
Please refer to go/dr-dependency-upgrade for more information about the
automation plan.
This PR calls out all dependencies that must be pinned to specific
versions for various reasons. The remaining ones are converted to
open-ended version ranges ("[version_str,)").
* Enhance the test for forbidden Schema changes
Current test is git-based. It is difficult to maintain and does not
catch out-of-order version numbers. It is also more aggressive than
necessary, failing on changes to submitted scripts that have not been
deployed yet.
The new test starts a database, deploys the current schema to it,
then deploys the set of Flyway scripts in this repository to the database.
* Get rid of all remaining JUnit 4 usages except in prober & proxy subprojects
Caveat: Test suites aren't yet implemented in JUnit 5 so we still use the ones
from JUnit 5 in the core subproject.
* Fix some build errors
* Require explict tag when starting psql docker
Defined a util class to return docker tag of desired PSQL version.
Class is defined in ':db' and shared by ':db' and ':core'. Used
an artifact declaration to exclude unnecesary compile dependencies.
Added a presubmit check for instantiations without explicit tag.
* Upgrade to Truth 1.0
Refactored fail(...) to assertWithMessage().fail().
Upgraded com.google.monitoring-client family of dependencies to 1.0.6
Also fixed bad use of io.StringIO (on binary buffer) recently introduced to
google-java-format-diff.py.
* Add schema deployment tests
Updated flyway schema script files so that they reflect what is
currently deployed in alpha: ClaimsList and PremiumList related
elements.
Put post-schema-push pg_dump output in nomulus.golden.sql as the
authoritative schema. Also added test to verify that the schema
pushed by flyway will result in exactly the golden schema.
Upgraded testcontainers to 1.12.1.
Added a custom Truth subject for better diffing of multi-line
text blocks.
Removed claims_list.sql and premium_list.sql, as we do not have use for
them.
* Add schema deployment tests
Updated flyway schema script files so that they reflect what is
currently deployed in alpha: ClaimsList and PremiumList related
elements.
Put post-schema-push pg_dump output in nomulus.golden.sql as the
authoritative schema. Also added test to verify that the schema
pushed by flyway will result in exactly the golden schema.
Upgraded testcontainers to 1.12.1.
Added a custom Truth subject for better diffing of multi-line
text blocks.
Removed claims_list.sql and premium_list.sql, as we do not have use for
them.
* Add schema deployment tests
Updated flyway schema script files so that they reflect what is
currently deployed in alpha: ClaimsList and PremiumList related
elements.
Put post-schema-push pg_dump output in nomulus.golden.sql as the
authoritative schema. Also added test to verify that the schema
pushed by flyway will result in exactly the golden schema.
Upgraded testcontainers to 1.12.1.
Added a custom Truth subject for better diffing of multi-line
text blocks.
Removed claims_list.sql and premium_list.sql, as we do not have use for
them.
