Commit Graph
100 Commits
Author SHA1 Message Date
Shicong HuangandGitHub a7e1bd800b Add a nomulus command to resave entity with unique id (#656)
* Add a nomulus command to resave entity with unique id

* Remove PollMessage

* Remove logic for PollMessage and resolve comments

* Resolve comments
2020-08-10 16:34:24 -04:00
Shicong HuangandGitHub 7b2b49dd53 Fix vkey construction in PollMessage and BillingEvent (#747)
* Fix vkey construction in PollMessage and BillingEvent

* Remove null check
2020-08-07 15:38:53 -04:00
Shicong HuangandGitHub 486bf32353 Add SQL schema for GracePeriod (#709)
* Add SQL schema for GracePeriod

* Remove the join table

* Add a domainRepoId in GracePeriod

* Move the clone logic to GracePeriod

* Rebase on HEAD
2020-08-06 10:26:19 -04:00
Shicong HuangandGitHub 1022817384 Fix broken RegistryCli (#728) 2020-07-30 14:10:09 -04:00
Shicong HuangandGitHub 0ecc20b48c Rename a V40 flyway file to V41 to resolve conflict (#719) 2020-07-27 15:16:01 -04:00
Shicong HuangandGitHub c65af4b480 Add remaining columns to Domain's SQL schema (#702) 2020-07-27 13:32:39 -04:00
Shicong HuangandGitHub f5186f8476 Merge two PremiumList entities (#690) 2020-07-21 18:18:52 -04:00
Shicong HuangandGitHub e1db357fc3 Merge two reserved list entities (#616)
* Merge reserved list

* Replace INSTANCE with getInstance()

* Fix broken test

* Rebase on master

* Simplify class
2020-07-13 13:40:34 -04:00
Shicong HuangandGitHub 58618a274e Add two folders of auto-generated Java classes to .gitignore (#679) 2020-07-13 10:09:56 -04:00
Shicong HuangandGitHub 391929b518 Expand AckPollMessagesCommand to ack PollMessage.Autorenew (#647)
* Expand AckPollMessagesCommand to ack PollMessage.Autorenew

* Rebase on master and address comment

* Resolve comments
2020-07-01 15:06:35 -04:00
Shicong HuangandGitHub 75659671db Fix a warning from Hibernate (#645)
Hibernate complained that ContactResource.getTransferData should not be
final:
WARN: HHH000305: Could not create proxy factory
for:google.registry.model.contact.ContactResource
org.hibernate.HibernateException: Getter methods of lazy classes cannot
be final: google.registry.model.contact.ContactResource#getTransferData
2020-06-24 10:02:42 -04:00
Shicong HuangandGitHub e7db9b3c1a Add a test to verify JPA supports polimorphic VKey (#642) 2020-06-22 15:13:38 -04:00
Shicong HuangandGitHub 2f600e3e69 Implement remaining methods in JpaTransactionManager (#633) 2020-06-17 12:39:34 -04:00
Shicong HuangandGitHub 26e2a51180 Refactor TransferData to remove unused fields in Contact table (#623)
* Add DomainTransferData and ContactTransferData

* Refactor TransferData to remove unused fields in Contact table

* Add scope for TransferData's type parameter
2020-06-16 10:42:57 -04:00
Shicong HuangandGitHub fdac686250 Add columns for TransferData in Domain and Contact (#577)
* Add columns for TransferData in Domain and Contact

* Rename flyway file and foreign key

* Rebase on master and address comment

* Compileable commit

* Fix unit test

* Refactor TransferServerApproveEntity

* Use tm().delete(vkeys)

* Rename transfer_period fields

* Rename client_id to registrar_id

* Rebase on master

* Resolve comment

* Rebase on master
2020-06-09 16:39:55 -04:00
Shicong HuangandGitHub f0765dc893 Make JpaUnitTestRule not depend on the nomulus schema (#613) 2020-06-09 14:51:13 -04:00
Shicong HuangandGitHub b1241b98b2 Generate sql schema for PollMessage (#582)
* Generate sql schema for PollMessage

* Rework columns and resolve comments

* Fix datastore schema
2020-06-04 18:24:59 -04:00
Shicong HuangandGitHub b42ded9451 Add test to verify the behavior of @DualDatabaseTest (#606) 2020-06-03 14:55:37 -04:00
Shicong HuangandGitHub 472503541b Add deleteAll method to TransactionManager (#604)
* Add deleteAll method to TransactionManager

* Rename deleteAll to delete

* Add bucket.getLastWrittenTime() before second mutation
2020-06-03 10:02:48 -04:00
Shicong HuangandGitHub 26fb5388a4 Generate sql schema for BillingEvent (#565)
* Generate sql schema for BillingEvent

* Change to use sequence

* Address comments

* Resolve warnings and remove duplicate cost related fields

* Increase the flayway file version to V25

* Remove extra space

* Split to 3 tables, merge VKey

* Rename talbes

* Rename repoId to domainRepoId

* Exclude VKey in schema.txt

* Rename target_id to domain_name

* Fix javadoc

* Resolve comments
2020-05-27 15:59:19 -04:00
Shicong HuangandGitHub 0f174d9ce0 Add all existing entities to VKeyTranslatorFactory (#595)
EntityClasses.ALL_CLASSES has all of our registered entities so
we can use it to initialize VKeyTranslatorFactory.classRegistry
to avoid adding them one by one.

Also, this PR changed to use Key.getKind() to get the kind of
the entity to solve the problem that when the entity class
is an inner class, its kind should still be the class name
instead of OuterClass$InnerClass.
2020-05-20 14:24:45 -04:00
Shicong HuangandGitHub a0f4013d53 Add JUnit5 extension to run test twice against different databases (#588)
* Add JUnit5 extension to run test against different databases

* Fix typos

* Add some explanation
2020-05-18 11:06:21 -04:00
Shicong HuangandGitHub 01e2d24658 Revert "Remove minimumIdle config in HikariCP (#557)" (#573)
This reverts commit d8066ca752.
2020-05-01 16:21:10 -04:00
Shicong HuangandGitHub 19bc1c9c9c Add annotation processor to generate converter for VKey (#566) 2020-04-29 17:29:05 -04:00
Shicong HuangandGitHub d8066ca752 Remove minimumIdle config in HikariCP (#557)
* Remove minimumIdle config for HikariCP

* Add comment

* Resolve comment
2020-04-22 19:35:02 -04:00
Shicong HuangandGitHub 295251ee78 Add JPA annotations to ContactResource and generate schema (#547)
* Add JPA annotations to ContactResource and generate schema

* Resolve comments

* Resolve comments

* Manually add foreign key constraints

* Run with junit5

* Rebase on HEAD

* Fix DomainBaseSqlTest
2020-04-21 15:40:16 -04:00
Shicong HuangandGitHub 9db4d1a082 Add a listener to invoke entity callbacks (#551)
* Add a listener to invoke entity callbacks

* Resolve comments

* Add test
2020-04-16 14:30:43 -04:00
Shicong HuangandGitHub 1ded33ecea Resolve warnings in the Hibernate log (#542) 2020-04-03 18:04:55 -04:00
Shicong HuangandGitHub bf4659f11c Auto-apply JPA converters for map type (#520)
* Add map converter

* Delete old map usertype

* Refactor bind

* Change to use map entry

* Use Map.Entry
2020-04-03 10:47:42 -04:00
Shicong HuangandGitHub bac1998d6a Auto-apply JPA converters for map type (#520)
* Add map converter

* Delete old map usertype

* Refactor bind

* Change to use map entry

* Use Map.Entry
2020-04-02 16:43:08 -04:00
Shicong HuangandGitHub db7d49801d Supress exccesive logging message from Cloud SQL (#540)
* Supress exccesive logging message from Cloud SQL

* Upgrade package versions that were downgraded before
2020-03-31 17:57:18 -04:00
Shicong HuangandGitHub fa9400ebc5 Set postgres package back to runtime dependency (#522) 2020-03-20 15:43:30 -04:00
Shicong HuangandGitHub d01f1f7604 Make jpaTm for nomulus tool use local credential (#515)
* Make jpaTm for nomulus tool use local credential

* Remove unused methods in RegistryToolEnvironment

* Fix order of annotations

* Remove unused method in PersistenceComponent

* Move the creation of credential to the module

* Move creadential creation to AuthModule

* Add a TODO
2020-03-17 20:16:42 -04:00
Shicong HuangandGitHub 0545375eba Change cloud sql SDK to compile level dependency (#516) 2020-03-16 10:24:19 -04:00
Shicong HuangandGitHub 3098048fdb Enable Cloud SQL when Datastore is enabled for unit test (#502)
* Enable Cloud SQL when Datastore is enabled for unit test

* Add explanation for why add a ETA field in GenerateEscrowDepositCommand

* Fix line length

* Ignore membershipt test but bring back test suite

* Fix tiny issue
2020-03-10 12:26:25 -04:00
Shicong HuangandGitHub adafab60c4 Add common CRUD operations to TransactionManager (#487)
* Add BasicDao

* Refactor RegistrarDao to extend BasicDao

* Introduce VKey and rewrite BasicDao

* Move CRUD methods to TransactionManager

* Refactor code to simplify the way to get id from entity and sqlKey

* Assert in transaction

* Fix broken test

* Change methods name
2020-03-05 14:03:03 -05:00
Shicong HuangandGitHub be395611ca Move all JPA converters to a dedicated package (#480)
* Move all JPA converters to a dedicated package

* Move tests as well
2020-02-19 13:54:27 -05:00
Shicong HuangandGitHub b9c40648d0 Add dual write for Registrar (#474)
* Add dual write for Registrar

* Use @AlsoLoad to set street field for Cloud SQL

* Change email body to use the new streetLine field

* Refactored the logic to handle street fields

* Simplify postLoad function

* Address comments

* Add a TODO to remove street

* Add test for onLoad and postLoad

* Rebase on master
2020-02-13 16:00:21 -05:00
Shicong HuangandGitHub 90945bcc30 Add a test to verify persistence.xml (#482) 2020-02-12 12:39:03 -05:00
Shicong HuangandGitHub 594ce30122 Auto-apply JPA converters for collection type (#469)
* Auto-apply JPA converters for collection type

* Extract common logic to a base class

* Remove extra lines

* Rebase on master
2020-02-10 10:33:43 -05:00
Shicong HuangandGitHub d6f49f5c08 Add JPA annotations to RegistrarContact (#432) 2020-02-06 16:58:15 -05:00
Shicong HuangandGitHub 9ea06ebf35 Use JPA @Enumerated to convert enum to string (#466) 2020-02-04 10:50:03 -05:00
Shicong HuangandGitHub 955c3d9aeb Fix money conversion for JPY in PremiumListDao (#463) 2020-01-29 13:10:58 -05:00
Shicong HuangandGitHub d03cea2443 Add JPA annotations to class Registrar (#430)
* Add JPA annotations to class Registrar

* Use array for Java list

* Exclude parent field

* Use 3 columns for address and use text for enum

* Use EnumParameter and 3 properties in Address

* Rename columns and rebase on HEAD
2020-01-29 11:06:23 -05:00
Shicong HuangandGitHub 43682aa7f4 Enable Cloud SQL for nomulus command implementing CommandWithRemoteApi (#460) 2020-01-27 10:28:38 -05:00
Shicong HuangandGitHub 7c3f685946 Relax active connection check to reduce flakiness (#458) 2020-01-23 18:09:20 -05:00
Shicong HuangandGitHub ad2cf933c2 Add dual read for ReservedList (#423)
* Add dual read for ReservedList

* Extract loading cloud sql list to a method
2020-01-17 15:49:58 -05:00
Shicong HuangandGitHub 851465a89d Relax reserved list existence check to allow Cloud SQL migration (#429) 2020-01-15 13:10:50 -05:00
Shicong HuangandGitHub 6aa7c19344 Add MapUserType to support converstion between Map and hstore (#443) 2020-01-15 10:37:14 -05:00
Shicong HuangandGitHub 4eae10800d Add user types to support Array in Cloud SQL (#439)
* Add user types to support Array in Cloud SQL

* Remove duplicate null check

* Simplify class structure
2020-01-10 14:54:21 -05:00
Shicong HuangandGitHub a28754dc5d Add dual read claims list (#413)
* Add dual read claims list

* Improve warning log and use longer duration for cache

* Extract the comparison logic to a method

* Move cache to DAO
2020-01-03 10:59:34 -05:00
Shicong HuangandGitHub f76030f7f0 Add initial support to write/update reserved lists to Cloud SQL (#388)
* Add initial support to write reserved list to Cloud SQL

* Add support to update reserved list in Cloud SQL

* Fix wrong check when override is enabled in create command

* Add sql dependent tests to the suite

* Address comment

* Make invocation of super.execute more readable

* Add test to check upper case and non-puny code label

* Move ReservedListDao related classes to schema package

* Simplify a function name
2019-12-19 12:51:48 -05:00
Shicong HuangandGitHub f301314d97 Move ClaimsListDao related classes to new package (#416) 2019-12-19 10:50:41 -05:00
Shicong HuangandGitHub 988f78274e Check SQL dependency in super class for SqlIntegrationMembershipTest (#398)
In some cases, we define JpaTransactionManagerRule in a TestCase
class which is extended by the concrete test class. So, we need
to check if JpaTransactionManagerRule is also defined in the super
class.
2019-12-05 13:12:56 -05:00
Shicong HuangandGitHub 9be5091c84 Add entity for reserved list (#381)
This PR added the Cloud SQL entity for reserved list.
2019-11-26 16:51:41 -05:00
Shicong HuangandGitHub 98414cb7cb Add a test to verify generated schema (#377) 2019-11-21 13:37:37 -05:00
Shicong HuangandGitHub 9e0368b77c Enable JpaTransactionManager in all environment (#352)
* Enable JpaTransactionManager in all environment

* Refactor to have a single place to create JpaTm
2019-11-18 14:53:49 -05:00
Shicong HuangandGitHub 09aef04117 Add PersistenceXmlUtility and refactor related code (#357)
* Refactor GenerateSqlSchemaCommand

* Add and throw UncheckedClassNotFoundException
2019-11-12 15:08:54 -05:00
Shicong HuangandGitHub 3a47fa2fe9 Fix beam pipeline deployment issue (#360) 2019-11-11 15:57:33 -05:00
Shicong HuangandGitHub f0345ddf89 Resolve test flakiness caused by connection leak (#355) 2019-11-08 15:40:27 -05:00
Shicong HuangandGitHub 2c6d71f04c Prevent test from changing golden schema (#337)
When we add the extra test entity to the current JpaTransactionManagerRule by calling jpaRule.withEntityClass(TestEntity.class) and jpaRule.withProperty(Environment.HBM2DDL_AUTO, "update"), the rule would override the golden database scheme with the schema derived from the current entity class(This is an expected behavior by enabling HBM2DDL_AUTO). This behavior prevents us from detecting breaking changes in ORM classes.

This PR fixed the issue by adding HibernateSchemaExporter to export the DDL script for given extra entity class, and make JpaTransactionManagerRule execute the DDL script to create the corresponding table for the extra entity when initializing the database. This PR also simplified the code when adding extra entity class for testing. For now, you don't need to(and shouldn't) call jpaRule.withProperty(Environment.HBM2DDL_AUTO, "update").
2019-11-05 11:36:03 -05:00
Shicong HuangandGitHub ee4e6ace2a Remove jdbcUrl config for nomulus tool (#330) 2019-11-05 10:33:53 -05:00
Shicong HuangandGitHub d00ade8ae0 Enable JpaTransactionManager in sandbox (#323)
* Enable JpaTransactionManager in sandbox
2019-10-31 14:23:38 -04:00
Shicong HuangandGitHub 6a3bd9418f Upgrade org.apache.beam related packages to latest version (#333) 2019-10-30 11:44:26 -04:00
Shicong HuangandGitHub 2d7f80baaf Fix issues related to Cloud SQL connection (#321)
* Reenable JpaTransactionManager for Alpha and Crash

* Make UploadClaimsListCommand implement CommandWithCloudSql

* Fix wrong call to get password

* Use Cloud SQL Socket library to provision TransactionManager

* Change to use dataSource configs
2019-10-25 12:40:43 -04:00
Shicong HuangandGitHub 85c11ca889 Use a single database container for all tests (#313) 2019-10-15 15:09:03 -04:00
Shicong HuangandGitHub 6c220567c8 Write ClaimsList to Cloud SQL (#223)
* Rewrite ClaimsListShard with new API

* Write ClaimsList to Cloud SQL

* Add creationTimestamp
2019-10-11 12:31:34 -04:00
Shicong HuangandGitHub a694e247cd Add support for nomulus tool to connect to Cloud SQL (#303) 2019-10-09 10:30:35 -04:00
Shicong HuangandGitHub 022f397cd9 Implement ZonedDateTimeConverter (#287)
* Implement ZonedDateTimeConverter

* Use dedicated TestEntity for ZonedDateTimeConverterTest
2019-10-08 16:46:07 -04:00
Shicong HuangandGitHub 36ab0cb45c Add Cloud SQL configs for nomulus tool (#288)
We will use a different user for nomulus tool to connect to Cloud SQL. 
This PR added corresponding configurations for that.
2019-09-27 15:58:18 -04:00
Shicong HuangandGitHub 34da498958 Add sql scripts to create other types of user (#283)
1. User with read-only permission to all tables
2. User with read-write permission to schema and all tables
2019-09-27 15:12:59 -04:00
Shicong HuangandGitHub 9c7872dbb7 Add socket factory as runtime dependency (#286)
* Add Cloud SQL socket factory as runtime dependency

* Lock update from update_dependency.sh
2019-09-27 11:41:59 -04:00
Shicong HuangandGitHub ff439f598d Add JpaTransactionManagerRule (#277)
* Add RegistryRuntime and JpaTransactionManagerRule

* Revert RegistryJavaRuntime change

* Add JpaTransactionManager interface
2019-09-24 15:38:53 -04:00
Shicong HuangandGitHub cc314d4838 Make project core depend on db at test runtime (#278)
This PR also upgraded the version of Guava to 28.1-jre to
resolve missing stream package issue.
2019-09-23 14:12:24 -04:00
Shicong HuangandGitHub b35c813ae6 Add sql files to manage nomulus user (#274) 2019-09-20 14:46:02 -04:00
Shicong HuangandGitHub d031dedad6 Implement JpaTransactionManager (#268) 2019-09-19 10:01:40 -04:00
Shicong HuangandGitHub 48d8b1274f Add configurations for Cloud SQL secrets (#266) 2019-09-11 12:20:08 -04:00
Shicong HuangandGitHub 2967256766 Consolidate EntityManagerFactoryProviderTest (#253)
Check if EntityManagerFactory is null before calling close() to avoid NullPointerException.
2019-09-04 15:46:13 -04:00
Shicong HuangandGitHub 487b695a10 Set up database connection pool (#234)
Set up database connection pool
2019-08-29 16:12:28 -04:00
Shicong HuangandGitHub dcceb0d49a Add sql schema and entity class for ClaimsList (#227) 2019-08-26 18:30:51 -04:00
Shicong HuangandGitHub 3ba6e7bd06 Remove hardcoded Java home path (#235) 2019-08-20 10:54:45 -04:00
Shicong HuangandGitHub d143cc83a0 Add Hibernate and PostgreSQL packages (#224) 2019-08-14 11:22:01 -04:00
Shicong HuangandGitHub a68b1a12fd Implement TransactionManager for datastore (#207)
This PR created the new interface named TransactionManager which defines
methods to manage transaction. Also, the access to all transaction related
methods of Ofy.java are restricted to package private, and they will be exposed
by DatastoreTransactionManager which is the datastore implementation of
TransactionManager.
2019-08-07 12:59:08 -04:00
Shicong HuangandGitHub 8e00f85f60 Use OpenJDK8 for Gradle build (#210) 2019-08-02 17:00:35 -04:00
Shicong HuangandGitHub 3cc8d83396 Hard code webdriver docker version (#208) 2019-08-01 14:02:57 -04:00
Shicong HuangandGitHub 110bd9c057 Fix deploying to alpha from workstation (#198)
appengineDeployAll requires appengine.deploy.version to be set
otherwiese the deployment would fail.
2019-07-26 14:37:49 -04:00
Shicong HuangandGitHub e2a673d914 Bring back the old GoogleCredential for Drive API (#187)
Using the new GoogleCredentials to access Drive API caused 403 forbidden
exception. So, this PR brought back the old GoogleCredential to
temporarily resolve the production issue while we are figuring out the
long term fix.

TESTED=Deployed to alpha and verified exportPremiumTerms succeeded, see
https://paste.googleplex.com/6153215760400384.
2019-07-23 11:31:35 -04:00
Shicong HuangandGitHub 633dd887f4 Fix permission issue in Beam pipeline deployment (#170) 2019-07-15 16:13:42 -04:00
Shicong HuangandGitHub 8371cb838c Add a 30s timeout for all webdriver tests (#161)
Sometimes, the webdriver tests get stuck forever for no reason. It could
be some issue in the test container but it is hard to root cause it. So,
adding a 30s timeout can either trigger the retry earlier or let the
test just fail.
2019-07-09 14:42:32 -04:00
Shicong HuangandGitHub 8dd6797614 Read golden images from src directly (#159)
This PR prevents Gradle from copying the golden images
to build/resources/test, so the screenshot test would
read golden images from src/test/resources directly and
display the path in test log if the test fails. Because
the path pointing to the actual file in src/ folder,
the engineer can easily find it.
2019-07-09 10:49:20 -04:00
Shicong HuangandGitHub 6536631857 Remove injected credentials from invoice pipeline (#155)
We got non-serialization object error when deploying the invoicing
pipeline. It turns out that Beam requires every field in the pipeline
object is serilizable. However, it is non-trivial to make
GoogleCredentialsBundle serilizable because almost all of its
dependency are not serilizable and not contraled by us. Also,
it is non-necessary to inject the credential as the spec11
pipeline also writes output to GCS without having injected
credential. So, removing the injected variable can solve the
problem.

TESTED=First reproduced the problem locally by deploying the invoicing pipeline with the previous code; applied this change and successfully deploy the pipeline without having any issue.
2019-07-03 15:12:48 -04:00
Shicong HuangandGitHub 82fa3d7349 Change button color to blue style (#150) 2019-07-02 15:29:17 -04:00
Shicong HuangandGitHub 6daf72a54e Replace deprecated GoogleCredential with new auth lib (#129)
Replace deprecated GoogleCredential with new lib

This PR also introduced a CredentialsBundle class to carry
HttpTransport and JsonFactory object which are needed by
most of the GCP library to instantiate client.
2019-07-02 10:29:51 -04:00
Shicong HuangandGitHub 661e348fcb Stop provisioning RegistryEnvironment from Dagger (#140)
We found that some webdriver tests failed because RegistryEnvironment
was set to 'production' by other test and was carried over to the
webdriver test, and it was nontrivial to fix them because the instance
of RegistryEnvironment was injected into the testing web server.

Also, to eliminate this problem from potential victims, we stopped
provisioning RegistryEnvironment from Dagger. Instead, we just use
RegistryEnvironment.get() function to get the instance, which indeed
retrives the value from system property every time. If any test case
needs to run the test with other environment, it can just simply use
the existing SystemPropertyRule and RegistryEnvironment.${ENV}.setup
to change it.
2019-06-26 16:57:19 -04:00
Shicong HuangandGitHub c6f2448d63 Merge pull request #122 from hstonec/upgrade-google-auth-lib
Upgrade google-auth-library-java to latest version
2019-06-18 17:54:24 -04:00
Shicong Huang 0630b371df Upgrade google-auth-library-java to latest version
This is a preparation for replacing the deprecated GoogleCredential
library.
2019-06-18 17:27:21 -04:00
Shicong HuangandGitHub 3df11e7849 Merge pull request #105 from hstonec/deploy-beam-from-gcb
Add a CloudBuild task to deploy Beam pipeline
2019-06-18 15:34:28 -04:00
Shicong HuangandGitHub 6ca5cab66f Consolidate testcontainer used by WebDriver test (#112)
* Consolidate testcontainer used by WebDriver test

Previously, we hard coded the version of the docker image for
provisioning Chrome browser and WebDriver server because the
version of the Chrome browser has to match the version of
the webdriver client, otherwise the screenshot test will fail.

Changing to use BrowserWebDriverContainer can delegate the match
to the library itself because it chooses the correct docker image
version based on the WebDriver version on our classpath.

* Increase maxColorDiff to 20

This is to supress the test flakness after switching to use
BrowserWebDriverContainer to provision browser and webdriver
service.
2019-06-18 11:51:55 -04:00
Shicong Huang 15622c2f84 Add a CloudBuild task to deploy Beam pipeline 2019-06-18 10:48:41 -04:00
Shicong HuangandGitHub 8fa45e8c76 Merge pull request #96 from google/sc/fix-uber-jar
Use ShadowJar plugin to generate uber jar
2019-06-12 23:07:28 -04:00